Urgent need for Databricks Architect with HR Domain

Location: Remote
 

Job Summary

We are seeking an experienced Databricks Architect with strong expertise in data engineering, cloud data platforms, and HR data analytics. The ideal candidate will design and implement scalable, secure, and high-performance data solutions using Databricks, integrating data from various HR systems to enable workforce analytics, reporting, and data-driven decision-making.

The candidate should have a strong understanding of HR business processes, employee lifecycle data, HR systems, data governance, and privacy requirements.

Key Responsibilities

• Design and implement end-to-end data architecture using Databricks and cloud-based data platforms.
• Develop scalable data pipelines using PySpark, Python, SQL, and Delta Lake.
• Architect Lakehouse solutions using the Medallion Architecture (Bronze, Silver, and Gold layers).
• Integrate HR data from platforms such as Workday, SAP SuccessFactors, Oracle HCM, ADP, and other HR systems.
• Build unified HR data models supporting employee demographics, recruitment, onboarding, compensation, payroll, benefits, performance management, learning, attrition, and workforce planning.
• Design curated data products and analytical datasets for HR dashboards, workforce analytics, and executive reporting.
• Implement data quality, validation, reconciliation, lineage, and monitoring frameworks.
• Establish data security and governance controls, including role-based access, PII protection, data masking, and row-level/column-level security.
• Ensure compliance with applicable data privacy regulations and organizational policies for sensitive employee information.
• Optimize Databricks workloads for performance, scalability, reliability, and cost efficiency.
• Collaborate with HR business stakeholders, data engineers, BI developers, data scientists, and enterprise architects to translate business requirements into technical solutions.
• Define architecture standards, reusable design patterns, development guidelines, and deployment practices.
• Implement CI/CD pipelines, automated testing, and infrastructure-as-code practices for data platform deployments.
• Mentor engineering teams and provide technical leadership throughout solution design, development, testing, and production deployment.

Required Technical Skills

• Strong hands-on experience with Databricks, Apache Spark, PySpark, Python, and SQL.
• Expertise in Delta Lake, Delta tables, and Lakehouse architecture.
• Experience designing and implementing ETL/ELT pipelines and batch or streaming data processing.
• Strong knowledge of data modeling, dimensional modeling, data warehousing, and data lake design.
• Experience with at least one major cloud platform: Azure, AWS, or GCP.
• Experience with cloud storage and data integration services, such as Azure Data Lake Storage, Amazon S3, or equivalent services.
• Knowledge of orchestration, workflow scheduling, monitoring, and error-handling frameworks.
• Experience with data governance, access control, metadata management, lineage, and sensitive-data protection.
• Familiarity with Git, CI/CD, automated testing, and deployment automation.
• Experience with Power BI, Tableau, or similar BI tools is an advantage.

HR Domain Expertise

• Strong understanding of HR business processes and employee lifecycle management.
• Experience integrating data from HRIS, HCM, payroll, recruitment, benefits, learning management, and talent management systems.
• Familiarity with HR platforms such as Workday, SAP SuccessFactors, Oracle HCM Cloud, or ADP.
• Experience designing HR data models and workforce analytics solutions.
• Understanding of HR metrics and KPIs, including:
  • Headcount and workforce composition
  • Employee turnover and attrition
  • Hiring and recruitment effectiveness
  • Time-to-hire and time-to-fill
  • Compensation and benefits analysis
  • Absenteeism and workforce productivity
  • Diversity and workforce planning metrics, subject to applicable policies
• Experience handling sensitive employee data with appropriate access restrictions, privacy controls, retention policies, and auditability.

Preferred Qualifications

• Databricks certifications, such as Databricks Data Engineer or Data Engineer Professional.
• Relevant cloud certifications in Azure, AWS, or GCP.
• Experience implementing enterprise-scale HR data platforms or people analytics solutions.
• Knowledge of Unity Catalog, Delta Live Tables or Lakeflow Spark Declarative Pipelines, and Databricks Workflows.
• Familiarity with machine learning and predictive analytics use cases, such as attrition prediction and workforce forecasting.
• Experience working in large enterprise environments with distributed data engineering teams.

Key Competencies

• Enterprise data architecture and solution design
• Strong analytical and problem-solving skills
• Stakeholder management and communication
• Technical leadership and mentoring
• Ability to translate HR business needs into scalable data solutions
• Focus on security, governance, performance, and cost optimization

Expected Deliverables

• Enterprise HR data architecture and solution design documents
• Scalable Databricks data pipelines and Lakehouse implementation
• Integrated HR data models and curated analytical datasets
• Data quality, governance, and security frameworks
• Production-ready deployment and monitoring processes
• Technical documentation and architecture standards