AWS Data Engineer

Job Title: AWS Data Engineering
Location: New York, NY (Onsite/Hybrid)
Contract
Experience: 13+years

Job Description:
Primary Skills:
PySpark
Python
Unit Testing
Skill Cluster:
Data Engineering – AWS (Glue, EMR, Spark, Redshift, Kinesis, S3, AWS Native Data Services)
Role Overview:
We are seeking an experienced Lead Software Engineer (Data Engineering) with strong expertise in AWS data services and big data technologies. The ideal candidate will have a proven track record of designing and building scalable data pipelines and systems for high-volume environments.
Key Responsibilities:
Design, develop, and maintain scalable data pipelines using PySpark and Python
Implement batch and real-time data processing solutions using Spark, Kafka, and Kinesis
Work with AWS data services such as Glue, EMR, S3, Redshift, and Lake Formation
Optimize performance for large-scale data processing applications
Build reusable frameworks, tools, and data products
Ensure data governance, security, and compliance standards are met
Collaborate with cross-functional teams in Agile environments
Perform code reviews, unit testing, and ensure CI/CD best practices
Required Qualifications:
8–10 years of experience in Data Engineering or related roles
Strong hands-on programming experience in Python and PySpark
Expertise in Kafka, Kinesis, Spark, AWS Glue, and Lake Formation
Experience with performance tuning in batch and real-time processing systems
Solid understanding of data governance and security implementations
Experience building scalable and high-volume data processing systems
Familiarity with microservices architecture and distributed systems
Hands-on experience with Agile methodologies and CI/CD practices
Preferred Qualifications:
Experience with AWS services: ECS, EKS, EMR, DynamoDB, Aurora, Redshift, QuickSight
Knowledge of serverless technologies such as AWS Lambda, MSK, MWAA, Kinesis Analytics
Experience working with high-throughput, real-time data systems