Azure Data Engineer

Hello,

We do have a requirement for Azure Data Engineer on C2C/W2 basis. Please free to contact us. 

Job Title: Azure Data Engineer

Location: Jersey City, NJ, Charlotte, NC

Position: Hybrid, Twice a week 

PRINCIPAL DUTIES & RESPONSIBILITIES:

Design and develop code and configurations for Azure Databricks applications, ensuring alignment with business and technical requirements.

Build and optimize data integration pipelines for ingestion, transformation, and storage using Databricks and Azure Data Lake.

Conduct detailed analysis of requirements, incidents, and process flows; prepare clear technical documentation for development and implementation.

Troubleshoot and resolve issues across environments, performing root cause analysis to improve reliability and prevent recurrence.

Deliver timely, high-quality solutions and production support to maintain compliance and customer satisfaction.

Communicate effectively with business users, support teams, vendors, and stakeholders to ensure transparency throughout the project lifecycle.

Manage multiple projects and tasks simultaneously, balancing priorities in a fast-paced environment.

Develop and execute unit and integration test cases to validate changes and releases.

Coordinate and perform system upgrades in collaboration with end users and technical teams.

Act as an escalation point for complex production issues, ensuring minimal business impact.

Participate in system implementations and go-lives, including occasional evening or weekend support for deployments.

Collaborate with cross-functional teams to resolve high-priority incidents and contribute to post-implementation reviews and continuous improvement efforts.

JOB SPECIFICATIONS:

5+ years of experience implementing enterprise technologies or vendor platforms.

Bachelor's degree in Computer Science or related field preferred.

Core Technologies: Azure Databricks, Azure Data Lake, Python, Power BI, Power Apps, SQL Server, Oracle/PL-SQL, SQL stored procedures, Actimize (preferred)

Proven experience in IT development, system design, integration, and data analysis across complex environments.

Strong understanding of relational and SQL database platforms, with proficiency in Oracle or similar technologies.

Familiarity with surveillance and corporate compliance systems.

Ability to manage and prioritize multiple tasks independently in a dynamic environment.

Strong analytical and problem-solving skills with a focus on innovation and efficiency.

RESPONSIBILITIES

Build large-scale batch and real-time data pipelines with data processing frameworks in Azure cloud platform.

Designing and implementing highly performant data ingestion pipelines from multiple sources using Azure Databricks.

Direct experience of building data pipelines using Azure Data Factory and Databricks.

Developing scalable and re-usable frameworks for ingesting of datasets

Lead design of ETL, data integration and data migration.

Partner with architects, engineers, information analysts, business, and technology stakeholders for developing and deploying enterprise grade platforms that enable data-driven solutions.

Integrating the end to end data pipeline - to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times

Working with event based / streaming technologies to ingest and process data

Working with other members of the project team to support delivery of additional project components (API interfaces, Search)

Evaluating the performance and applicability of multiple tools against customer requirements

REQUIREMENTS

Experience on ADLS, Azure Databricks, Azure SQL DB and Datawarehouse

Strong working experience in Implementation of Azure cloud components using Azure Data Factory , Azure Data Analytics, Azure Data Lake, Azure Data Catalogue, LogicApps and Function Apps

Have knowledge in Azure Storage services (ADLS, Storage Accounts)

Expertise in designing and deploying data applications on cloud solutions on Azure

Hands on experience in performance tuning and optimizing code running in Databricks environment

Good understanding of SQL, T-SQL and/or PL/SQL

Should have experience working in Agile projects with knowledge in Jira

Good to have handled Data Ingestion projects in Azure environment

Demonstrated analytical and problem-solving skills particularly those that apply to a big data environment

Experience on Python scripting, Spark SQL PySpark is a plus