Job Title : Databricks Platform Admin
Location : St. Louis, Missouri (Hybrid)
Duration : 12 Months
Job Description :
Key Responsibilities
- Design, implement, and manage scalable and secure Databricks platforms to ensure optimal performance and reliability.
- Create and manage Databricks workspaces and clusters, including right-sizing and lifecycle management of resources.
- Automate infrastructure provisioning and deployment using Infrastructure-as-Code tools like Terraform, GitLab CI/CD, and scripting languages (Python, Bash, PowerShell).
- Monitor platform health, performance, and cost by building dashboards and integrating with monitoring tools.
- Troubleshoot and resolve issues related to cluster configuration, job failures, and performance bottlenecks within Databricksenvironments.
- Collaborate with cross-functional teams including data scientists, ML engineers, and product teams to deploy and maintain data pipelines and services on Databricks.
- Implement security and governance policies, including user management, access controls, and compliance adherence using Unity Catalog and other Databricks security features.
- Optimize compute and storage resources for cost efficiency without compromising performance, leveraging features like Delta Lake and Photon.
- Stay updated with new Databricks features and coordinate with product teams to evaluate and adopt them.
- Document platform architecture, best practices, troubleshooting guides, and provide mentorship to junior engineers on platform usage and optimization
What you'll need
- Required hands-on experience in architecting, deploying, managing, and supporting Azure Databricks Platform, Azure Datalake and related solutions.
- Required hands-on experience working with Cloud provider tools and architecture (Azure) .
- Required hands-on experience using scripting languages and command-line (CLI) tools such as Ansible, SQL, Terraform, Azure CLI, Databricks CLI, PowerShell and/or Bash.
- Preferred experience in management and tuning of data platforms like Azure ADLS storage and Databricks.
- Knowledge working with and maintaining CI/CD pipelines and versioning tools such as Git preferred.
- Knowledge with a broad set of data ingestion and data pipeline patterns, and associated tools like (Qlik Replicate, Qlik Compose, Databricks, Kafka, Matillion ELT and Snowflake Services)
- Experience with working in an agile product development team
- Excellent communication skills and ability to articulate system designs and patterns to varying levels of leadership.
Thanks & Regards
Bhavanesh Eamani
Lead Technical Recruiter
PH : 9724400073
Agile Enterprise Solutions Inc.
2591 Dallas Parkway Suite 300 Frisco, TX 75034
