Azure Databricks Lead/ Architect
Location Dallas TX, Charlotte NC Preferred Location Onsite
Contract
Must Have
12 years of experience of working on Azure and Databricks Lakehouse implementation with grasp of following
o Translate business requirement into technical solution for data curation and consumptions
o Have experience in building Databricks Lakehouse from a modelling and transformation ELT perspective
Experience in SQL programming Pyspark ADLS
2 or more years experience working as a Data Engineer Experience of developing in the Azure Modern Data Warehouse platform e g Azure Data Lake Azure Data Factory Azure Databricks Databricks Delta Lake etc
Good to Have
3 or more years experience in SQL
Experience handling structured and unstructured datasets
Good knowledge of DevOps tools and processes
Expertise on Databricks delta lake implementation Databricks architecture and configuration
Expertise in Azure Data Factory building pipelines and orchestration Should have prior experience doing SQL server migration to cloud Experience in effort estimation for the new projects Proven ability to drive complex design and development efforts in an agile environment Prior experience of migrating on premise SQL Server to Delta lake Automating data pipelines through a CI CD delivery methodology Expertise in Databricks Expert in ETL Pipelines Typical azure services current future used ADX ADF Databricks ADLS use in data ETL and storage
In depth hands on implementation knowledge on Azure Databricks Delta Lake Delta table Managing Delta Tables Databricks Cluster Configuration Cluster policies
Expertise in Azure Data factory Data Flow Azure Data Bricks and Data Lake Storage
Design and implement ETL components for multiple applications
Good Understanding of data modelling for data lake
Hands on experience in using secure file transfers tools like Kiteworks
Oracle PL SQL knowledge a plus
Soft Skills
As this is an onsite role must have strong written and oral communication skills
Insurance Domain Knowledge
Strong ability to work with client stakeholders
Requirement Review and Work Effort Estimation
Qualifying questions
1 Do you have relevant hands on experience on Azure Data Services like Databricks Data Lake etc
2 Do you hands on Unity Catalog work
3 Any reporting tool such as Power BI used and or aware of Mosaic AI tools
|
Neha Chaudhary |
