Data Architect / Staff Data Engineer - Databricks

Software Design for Data Systems

• Design and implement scalable data services and APIs for data consumption
• Build production-grade data applications using modern software engineering practices
• Develop microservices for data ingestion, transformation, and serving layers
• Create robust error handling, retry mechanisms, and monitoring for data systems

Outcome: Production-ready data services with 99.9% uptime and comprehensive observability

Data Pipeline Development

• Build and maintain complex ETL/ELT pipelines processing healthcare data at scale
• Implement real-time streaming data pipelines for time-sensitive healthcare events
• Design orchestration workflows for dependencies across multiple data systems
• Optimize data processing jobs for performance and cost efficiency

Outcome: Reliable, efficient data pipelines that scale automatically with data volume

Technical Leadership

• Provide technical guidance on software design patterns for data applications
• Lead code reviews ensuring software engineering best practices in data code
• Mentor junior engineers on both software development and data engineering concepts
• Drive adoption of test-driven development and CI/CD for data systems

Outcome: High-quality, maintainable codebases with comprehensive test coverage

Data Product Engineering

• Build data products as full-stack applications with APIs, SDKs, and documentation
• Implement data quality frameworks with automated testing and validation
• Create self-service data platforms with proper authentication and authorization
• Develop data catalogs and discovery tools as software products

Outcome: Resilient data systems that gracefully handle failures and scale demands

Collaboration and Communication

• Work with software engineers to integrate data services into applications
• Partner with platform engineers on infrastructure and deployment strategies
• Collaborate with data scientists to productionize models as services
• Communicate technical designs and trade-offs to stakeholders

Outcome: Seamless integration of data systems with broader application ecosystem

Compliance and Security Standards

• Implement secure coding practices for data applications handling PHI
• Build data access controls and audit logging into all data services
• Ensure HIPAA compliance in all data processing and storage systems
• Maintain security scanning and vulnerability management for data applications