Role: Azure Data Engineer Silver and Gold layer data models
Location: Norwell, MA- onsite
Duration: 6+Months C2H
Mode: Video
Job Details:Â Â Â Â Â Â Â Â
Job Summary:
We are seeking a skilled Data Engineer to design, build, and maintain high-quality, scalable data pipelines with a strong focus on Silver and Gold layer transformations, data quality automation, and CI/CD-driven deployments. The ideal candidate will have hands-on experience with Python, PySpark, Git-based CI/CD pipelines, ARM templates, and data quality libraries such as Great Expectations.
Key Responsibilities:
      Design and implement Silver and Gold layer data models following medallion architecture best practices.
      Perform data cleansing, standardization, enrichment, and aggregation to support analytics and reporting.
      Build optimized PySpark-based transformations for large-scale data processing.
      Ensure data consistency, performance, and scalability across datasets.
 Build and maintain CI/CD pipelines using Git-based workflows (Azure DevOps / GitHub).
      Use ARM templates (or IaC equivalents) for automated infrastructure provisioning.
      Enable automated deployment of data pipelines, notebooks, and configurations.
      Follow DevOps best practices for version control, branching, and release management.
      Develop reusable Python-based automation scripts for pipeline orchestration, validation, and monitoring.
      Create modular, maintainable, and testable Python code.
      Support automation of metadata, logging, alerting, and operational tasks.
      Implement data quality libraries such as Great Expectations.
      Define and automate data quality rules (completeness, accuracy, freshness, consistency).
      Integrate data quality checks into data pipelines and CI/CD workflows.
      Monitor, log, and troubleshoot data quality issues proactively.
      Work closely with data architects, analysts, QA, and business stakeholders.
      Translate business and analytical requirements into robust data engineering solutions.
      Participate in Agile ceremonies and support sprint-based delivery.
      Provide production support, performance tuning, and troubleshooting
Required Skills and Qualifications:
5–10+ years of experience in data engineering or analytics engineering.
Strong hands-on experience with Silver & Gold layer development, Python (automation and data processing) and PySpark. Also with Great Expectations (data quality framework) and Experience with CI/CD pipelines using Git-based tools. Hands-on experience with ARM templates or infrastructure-as-code concepts. Strong understanding of data modelling and medallion architecture. Experience working with large datasets in distributed environments.
Good to Have:
Microsoft Certified: Azure Data Engineer Associate or Azure Enterprise Data Analyst Associate.
Wast management or oil and gas domain knowledge
Best Regards,
Navneet Jha
D : 848-999-0314Â |Â Email:Â navneet.jha@siriinfo.com
Siri InfoSolutions Inc, 3 Ethel Rd, Suite # 302, Edison NJ 08817.
