Data Engineer

Responsibilities:
Build and optimize pipelines in Databricks (PySpark, SQL, Delta Lake).
Ingest data from databases, APIs, PDFs, Excel, flat files, and web scraping.
Implement data quality checks, validation, and testing frameworks.
Deploy, monitor, and optimize pipelines in Databricks/AWS.
Collaborate with data science/analytics teams and document data processes.

Qualifications :
8+ years of Data Engineering experience.
Hands-on expertise in Databricks (PySpark, SQL, Delta Lake).
Strong experience with structured & unstructured data.
Knowledge of data quality/testing tools (Great Expectations, Deequ, dbt tests).
Proven ability to optimize large-scale Spark/Databricks workloads.