Sr. Data Engineer with MS Fabric & Azure Must || Chicago, IL (Hybrid) || Long Term Contract

Location- Chicago, IL (Hybrid)

Long Term Contract

 

Core Responsibilities

• Design and build end-to-end data platforms using Microsoft Fabric

o Lakehouse, Warehouse, OneLake, Dataflows Gen2

• Develop and optimize Spark workloads using PySpark and SparkSQL

• Develop MLOps pipelines for Advanced Analytics & AI

• Build scalable ETL/ELT pipelines using:

o Azure Data Factory (ADF)

o MS Fabric Data pipeline

o Dataflow gen 2

o SSIS (on-prem, Azure-SSIS IR, and migration scenarios)

• Implement data modeling patterns:

o Medallion (Bronze / Silver / Gold)

o Dimensional modeling (Star/Snowflake)

o Different data file management experience – Parquet, JSON, XML

• Integrate Microsoft Purview for:

o Data cataloging & classification

o Automated data lineage (ADF, Fabric, SQL, ADLS)

• Enforce data security and access controls:

o RBAC, column-level security, masking

o Fabric & Purview policy alignment

• Optimize performance, reliability, and cost across Fabric capacities

• Implement CI/CD and IaC for data pipelines and governance artifacts

• Partner with security, compliance, and BI teams to ensure trusted data delivery

 

Required Technical Skills

Microsoft Fabric (Must-Have)

• Fabric Data Engineering workloads

• Lakehouse & Warehouse

• OneLake architecture

• Fabric pipelines & notebooks

• Capacity planning and performance optimization

• Advanced PySpark (joins, windows, UDFs, optimization)

• Strong SparkSQL

• Strong MLOps & Feature Engg.

• Partitioning strategies, shuffle tuning, caching

• Large-scale data processing (TB+)

Azure Data Platform

• Azure SQL Database / SQL Server

• Azure Data Factory (ADF)

• SSIS / Azure-SSIS Integration Runtime

• ADLS Gen2

Data Lineage & Security (Microsoft Purview)

• Purview data catalog & scanning

• Automated lineage across ADF, Fabric, SQL, ADLS

• Business glossary management

• Integration with Azure RBAC & security policies

 

 

Your sincerely,
Ajay Sharma | Sr. Technical Recruiter.
Net2Source Inc.
Fax: (201) 221-8131 | Email: ajays@net2source.com
Global HQ Address: 270 Davidson Ave, Suite 704, Somerset, NJ 08873, USA