Cloud AI Engineer

Title: Cloud AI Engineer

Location: Austin, TX 78751 (Hybrid 2 or 3 Days Onsite)

Duration: 5 Months with possible extension hire

Job Description:

Understands business objectives and problems, identifies alternative solutions, performs studies and cost/benefit analysis of alternatives. Analyzes user requirements, procedures, and problems to automate processing or to improve existing computer system: Confers with personnel of organizational units involved to analyze current operational procedures, identify problems, and learn specific input and output requirements, such as forms of data input, how data is to be; summarized, and formats for reports. Writes detailed description of user needs, program functions, and steps required to develop or modify computer program. Reviews computer system capabilities, specifications, and scheduling limitations to determine if requested program or program change is possible within existing system.

Additional job details and special considerations:

The Databricks Administrator is responsible for administering, configuring, and supporting the Databricks platform to enable scalable data engineering, analytics, and AI/ML workloads. This role ensures platform reliability, security, cost optimization, and compliance while supporting data engineers, analysts, and data scientists across the organization. The administrator will manage workspace configuration, cluster policies, user access, integrations with cloud services, and ongoing platform monitoring to support enterprise data and analytics initiatives.
 

Must have skills:

• 8+ years of experience required in administering Databricks workspaces in a cloud environment - AWS

• 8+ years of experience required with strong understanding of Databricks cluster configuration, job scheduling, and workspace management.

• 8+ years of experience required in managing user access, roles, and permissions using IAM, SCIM, and role-based access control (RBAC).

• 8+ years of proficiency with Apache Spark concepts, including performance tuning and troubleshooting.

• 8+ years of experience required in integrating Databricks with cloud storage services (e.g., S3,)

• 8+ years of experience required in implementing and enforcing cluster policies and workspace governance standards.

• 8+ years of experience required with familiarity with Databricks SQL, notebooks, and job orchestration.

• 8+ years of experience required in monitoring platform health, performance, and availability.

• 8+ years of experience required in understanding of data security, encryption, and compliance requirements.

• 8+ years of experience required with DevOps or automation tools (Terraform, CI/CD pipelines, scripting).

• 4+ years of experience preferred in administering Databricks in an enterprise or government environment.

• 4+ years of experience preferred with Databricks Unity Catalog for data governance and access control.

• 4+ years of experience preferred in knowledge of cost management and optimization for Databricks workloads.

• 4+ years of experience preferred in supporting AI/ML workloads using Databricks ML and MLflow.

• 4+ years of experience preferred with data lake and lakehouse architectures.

• 4+ years of experience preferred with knowledge of Python, SQL, or Scala for administration and troubleshooting.

Best Regards,
Tarik Khanna
T a r i k . k h a n n a @ i n f o j i n i c o n s u l t i n g . c o m