Lead Software Engineer
Job Description
🚀 Hiring: Lead Software Engineer (Azure Databricks) – Toronto, Canada
Client: Canadian Imperial Bank of Commerce (CIBC)
Duration: (Contract)
Rate: Hourly (CAD)
(Permanent Canadian Residents and Work permit holders are only eligible to apply)
We’re looking for a Lead Software Engineer / Azure Databricks Consultant to join a high-impact data engineering initiative in Toronto. This role is ideal for someone experienced in building modern data platforms and scalable ETL pipelines on Azure.
🔹 Key Responsibilities
Design and implement ETL pipelines using Azure Databricks (batch & streaming)
Build real-time and batch data solutions using Spark (Structured Streaming)
Manage data governance & access control with Unity Catalog
Develop SCD (Type 1 & 2) and CDC pipelines
Work with Lakehouse architecture & external data integrations
Optimize performance using partitioning & clustering techniques
Collaborate with cross-functional teams and contribute to CI/CD & DevOps
🔹 Required Skills
Strong experience with Azure Databricks / Delta Lake
Proficiency in Python, SQL, and data modeling
Hands-on experience with ETL pipeline development
Solid understanding of data warehousing concepts
Experience with Azure ecosystem (ADF, ADLS, Azure SQL, Azure Functions)
Strong problem-solving and analytical skills
🔹 Preferred Skills
Experience with Apache Spark (PySpark/Scala)
Knowledge of data governance & security practices
Familiarity with CI/CD pipelines and DevOps workflows
🔹 Requirements
7+ years of experience in data engineering
Bachelor’s degree in Computer Science, IT, or related field
Similar Jobs
Agentforce QA Engineer
North Carolina
Senior Palantir Foundry Data Engineer
California
Senior Palantir Foundry Data Engineer
California
Senior Palantir Foundry Data Engineer
California
Data Engineer
Texas