QA With Databricks

Job Title: QA Databricks

Location: Charlotte,NC

Experience:12+

The Security Data Operations team is seeking a QA Data Engineer to design and develop a QA framework and pipeline for CyberDW within the Client’s Data Lakehouse. This role will ensure data accuracy and quality across ETL processes using Medallion Architecture (raw to target layers).

Key Responsibilities:

• Develop QA ETL pipelines in Azure Databricks.
• Validate and test data transformations and ingestion processes.
• Identify bugs in ETL code and create JIRA tickets for remediation.
• Ensure data quality, accuracy, and optimized storage solutions.
• Collaborate with QA analysts, IT, and Data Engineering teams.
• Follow Agile SDLC processes and provide sprint updates.

Required Skills:

• Strong experience in Python and SQL.
• Hands-on experience with Azure Databricks, Azure Data Factory, or AWS Glue.
• Experience working with ETL pipelines and cloud data tools.