Job Description – Databricks Data Engineer
Position Details
- Role: Databricks Data Engineer
- Location: Seattle, WA
-
Duration: Long Term
- Experience Required: 6–8 Years
-
Pay Rate : Depends on Profile
- MSP Owner: Michelle Lee
- GBaMS Req ID: 10735832
- Client Name: TCS/Confidential
Job Summary
We are seeking an experienced Databricks Data Engineer with strong expertise in Apache Spark, ETL/ELT development, and scalable cloud-based data engineering solutions. The ideal candidate will be responsible for designing, developing, and optimizing modern data pipelines and analytics platforms using Databricks and Delta Lake technologies. This role requires hands-on experience with large-scale data ingestion, Spark optimization, production support, and data governance best practices.
Key Responsibilities
- Design, develop, and maintain scalable and high-performance data pipelines using Databricks and Apache Spark
- Build and optimize ETL/ELT workflows for processing structured and unstructured data
- Implement and manage Delta Lake architectures for reliable and efficient data lake solutions
- Develop robust data ingestion frameworks from APIs, RDBMS, streaming platforms, cloud storage, and flat files
- Collaborate with data scientists, analysts, architects, and business stakeholders to deliver analytics-ready datasets
- Optimize Spark jobs for scalability, performance tuning, and cost efficiency
- Implement data quality validations, monitoring frameworks, and error-handling mechanisms
- Support production deployments, troubleshooting, incident resolution, and ongoing maintenance activities
- Ensure compliance with data security, governance, and regulatory standards
- Participate in code reviews, technical discussions, and mentor junior team members
- Document technical designs, workflows, and operational procedures
Required Skills
- Strong hands-on experience with Databricks
- Expertise in Apache Spark and distributed data processing
- Experience building ETL/ELT pipelines and workflow orchestration
- Strong knowledge of Delta Lake implementation and optimization
- Experience with data ingestion from:
- APIs
- RDBMS
- Streaming sources
- Flat files
- Cloud storage platforms
- Spark performance tuning and optimization experience
- Experience implementing data quality checks, monitoring, and logging frameworks
- Strong troubleshooting and production support skills
- Understanding of data security, governance, and compliance standards
- Experience participating in code reviews and mentoring junior engineers
Preferred Qualifications
- Experience working in cloud-based data engineering environments
- Knowledge of scalable data lake and lakehouse architectures
- Exposure to streaming data processing frameworks
- Strong analytical and problem-solving skills
- Excellent communication and stakeholder management abilities
Skill Matrix
|
Skill
|
Experience Required
|
|---|---|
|
Databricks
|
6+ Years
|
|
Apache Spark
|
6+ Years
|
|
ETL / ELT Development
|
6+ Years
|
|
Delta Lake
|
3+ Years
|
|
Data Engineering
|
6+ Years
|
|
Spark Optimization
|
4+ Years
|
|
Production Support
|
4+ Years
|
Lucky Vegi
Senior Technical Recruiter
732-289-2689 | lucky@mediit.io
Kendall Park, New Jersey – 08824
