Hi
Greetings,
This is Arunima Sharma from VBeyond Corp. We are a global recruitment company. One of our clients is looking Data Engineer lead
Job Title: Data Engineer Lead
Location: Dallas, TX – REMOTE
Duration: Long term Contract
Exp: 10 +
• Experience of End-to-End data platform implementation on Azure platform
• Understand business processes and how they are modelled in various systems
• Extensive development experience on Azure Synapse analytics, Azure Data Factory, Data Bricks and Spark
• Must have good Data analysis skills (MS SQL Server)
• Work experience on advanced programming languages Scala, Python and Spark
• Database development skills for Azure Synapse
• Experience and expertise on Design Document build and Architecture diagram build
• Experience of doing impact analysis on existing data source systems
• Experience and expertise in designing and implementing modular pipelines and reusable/configurable frameworks
• Work with Data Engineering Lead, technology teams to understand requirements to create data pipelines
• Implement data structures, workflows, and integrations between enterprise platforms to ensure the accurate and timely execution of business processes.
• Develop and maintain scalable data pipelines and build out new API integrations to support continuing increases in data volume and complexity.
• Managing and improving the performance of database, queries, tools, and solutions
• Creating and maintaining data warehouse, databases, tables, SQL queries, and ingestion pipelines to power BI reports
• Writing complex and efficient queries to transform raw data sources into easily accessible models for our teams and reporting platforms
• Prepare data for predictive and prescriptive modelling
• Identify and analyse data patterns
• Identify ways to improve data reliability, efficiency, and quality
• Work with analytics, data science, and wider engineering teams to help with automating data analysis and visualization needs, advise on transformation processes to populate data models, and explore ways to design and develop data infrastructure
• Strong experience in data curation/aggregation/preparation using Azure Data Lake, Azure Data Factory and distilling data into meaningful insights with hands-on expertise on Azure and Snowflake/Databricks.
• Create and maintain ETL specifications and process documentation to produce the required data deliverables data profiling source to target maps ETL flows
• Performance tuning of ETL workflows and SQL scripts
• Strong experience in Azure based EDW projects
• Experience with Agile development methodologies
• Good Verbal and Written communication skills Mandatory
• Azure Synapse analytics, Azure Data Factory, Data Bricks and Spark
• MS SQL Server
• Scala, Python and Spark
• Work experience preferably on Azure Cloud
