Azure Databricks Data Engineer

🔍 Key Responsibilities:
Design and develop data pipelines using Azure Data Factory (ADF) / Azure ADB
Ingest data from multiple structured and unstructured data sources
Perform data transformation using Azure Databricks, Delta Lake, and PySpark
Implement and manage CI/CD pipelines using Azure DevOps (Stories, Bugs, Issue tracking)
Build and support integration solutions within the Microsoft Azure ecosystem
Analyze and troubleshoot data issues to ensure data reliability and accuracy
Collaborate with cross-functional teams to gather requirements and design data solutions
Perform data cleansing and transformation for high-quality data outputs
Optimize and monitor data pipelines for performance and scalability
Ensure compliance with data governance, security, and best practices
Work closely with data architects to align data models with business needs
Document technical designs, processes, and best practices