Lead Data Engineer Databricks Manufacturing/MES)

Job Title: Lead Data Engineer (Databricks – Manufacturing/MES)

Location: San Jose, CA: Onshore – Preferably PST Time Zone

Preferably US Citizen, Good profile will go all visas.

Role Overview:

We are seeking a hands-on Lead Data Engineer with strong expertise in Databricks and deep experience in Manufacturing environments, particularly working with MES (Manufacturing Execution Systems) data and systems. This role requires a technical leader who can architect solutions while remaining actively involved in development (“hands-on keyboard”).

Key Responsibilities:

• Design, build, and optimize scalable data pipelines using Azure Databricks (PySpark, Spark SQL, Delta Lake)
• Integrate and process MES data, shop-floor data, IoT/sensor data, and ERP systems
• Architect modern data solutions supporting manufacturing analytics and operational reporting
• Lead data engineering best practices including performance tuning, optimization, and governance
• Work closely with manufacturing stakeholders to translate MES data requirements into scalable data models
• Ensure high data quality, reliability, and production-grade deployments

Required Skills & Experience:

• Strong hands-on expertise in Azure Databricks (Delta Lake, PySpark, Spark architecture)
• Proven experience in Manufacturing domain, especially working with MES systems and production data
• Experience building real-time or near-real-time data pipelines
• Strong SQL and data modelling skills
• Experience with cloud platforms (Azure preferred)
• Ability to function as both technical architect and hands-on developer
• Excellent communication skills and ability to work in PST time zone

Preferred:

• Experience with IoT data ingestion and streaming technologies
• Exposure to data warehousing and BI integration
• Experience leading small data engineering teams