Lead Data Engineer
Location: Remote
Exp: 13+ years
Duration: Long Term
Phone and Video call
Requirements
We are seeking a Lead Data Engineer with strong expertise in Databricks and a solid understanding of Microsoft Fabric Data Engineering concepts. The ideal candidate will work closely with SanDisk’s internal teams, playing a critical role in designing and developing scalable data solutions that meet complex business requirements.
Key Responsibilities
Work hands-on with Databricks to build, optimize, and maintain large-scale data pipelines.
Collaborate directly with multiple SanDisk employees and technical stakeholders to gather requirements and implement data engineering solutions.
Leverage Databricks features including:
Different types of computes (clusters/jobs)
Unity Catalog for data governance and access control
Spark / PySpark for distributed data processing
Implement scalable data solutions using modern data lake and medallion architecture principles.
Demonstrate good understanding and implementation of Fabric Data Engineering features such as:
Mirroring
Direct Lake
Real-time and batch data integration strategies
Required Skills & Experience:
10+ years of hands-on experience in data engineering roles, with 2+ years specifically in Databricks.
Proven experience with Spark and PySpark in enterprise environments.
Experience implementing Unity Catalog for data security and lineage.
Familiarity with Microsoft Fabric Data Engineering, including Direct Lake, Mirroring, Lakehouse architecture, etc.
Strong problem-solving skills, performance tuning experience, and ability to write efficient code.
Excellent communication skills to work effectively with business and technical stakeholders.
Preferred Qualifications:
Experience working with cloud platforms like Azure or AWS.
Knowledge of Delta Lake, MLflow, or other advanced Databricks services.
Exposure to CI/CD for data pipelines and DevOps practices
Contact Information
Email: navneet@fasthireinc.com
Click the email address to contact the job poster directly.