Job Title: Senior Data Engineer
Location: Madisson WI – Onsite
Contract
Senior Data Engineer – Microsoft Fabric & Snowflake
(ADF, Databricks)
Role Summary
Own end-to-end data platform design and implementation across Lakehouse and Cloud Data Warehouse environments. Deliver POCs and production-grade solutions with architecture evaluation, performance tuning, governance, and cost optimization.
Key Responsibilities
- Architecture: Design scalable Lakehouse/Warehouse solutions; implement Medallion (Bronze/Silver/Gold); evaluate Fabric vs Databricks vs Snowflake.
- Ingestion & Orchestration: Build batch/CDC pipelines (MERGE, Streams, watermark); metadata-driven workflows; scheduling and monitoring.
- Transformation: Develop ELT using PySpark and SQL; optimize storage (partitioning, clustering, Delta optimize); tune compute/concurrency.
- Data Modeling: Star schema, surrogate keys, layered schemas (Raw → Curated), data quality enforcement.
- Governance & Security: RBAC, lineage, masking, Unity Catalog/Fabric governance.
- Performance & Cost: Throughput benchmarking; SKU/DBU/Credit optimization; workload isolation and scalability testing.
Core Skills
- Advanced: Microsoft Fabric, Azure Data Factory, Databricks (Delta, Unity Catalog), Snowflake (Warehouses, Streams/Tasks), PySpark, SQL
- Strong: Medallion architecture, dimensional modeling, governance & security
- Working: CI/CD, Infrastructure as Code (Terraform/ARM), CDC/streaming, capacity planning
- Experience: Cross-platform architecture comparison and high-concurrency tuning
| Sundram Jha Senior Recruiter e: sundram.jha o: (+1) 609-606-9010 ext. 2460 m: (+1) 732-592-3874 |
::
:
:
🔔 Get our daily C2C jobs / Hotlist notifications on
