Data Engineer (Python / PySpark / AWS) || McLean, VA (Onsite) (Need Only Ex-Capital One Employee) ||

Work Location: McLean, VA (Onsite) (Need Only Ex-Capital One Employee)

Duration: Long term contract

End Client: Capital One

Max & Final Rate: $65/hr. on C2C

Immediate Interview 

 

Responsibilities:

·       Develop and optimize scalable data pipelines using Python and PySpark, leveraging AWS services such as Glue, EMR, and Step Functions.

·       Design, implement, and maintain robust data processing systems using AWS Glue for ingestion, transformation, and orchestration.

·       Utilize EMR clusters for distributed data processing and analytics, ensuring optimal performance and resource utilization.

·       Build and manage serverless workflows using AWS Step Functions to orchestrate complex data pipelines.

·       Collaborate with cross-functional teams to gather data requirements and deliver scalable, efficient solutions.

·       Ensure data quality, governance, and security best practices across the data lifecycle.

·       Troubleshoot and resolve production data issues using AWS monitoring and logging tools.

·       Stay current with emerging big data technologies, AWS services, and data engineering best practices.

 

Required Qualifications:

·       Bachelor’s degree in Computer Science, Engineering, or related field.

·       Proven expertise in building data pipelines using Python and PySpark, with strong hands-on experience in AWS Glue, EMR, and Step Functions.

·       Strong experience with Relational Databases and Snowflake (Mandatory).

·       Hands-on experience with workflow orchestration tools such as Apache Airflow and CI/CD pipelines.

·       Proficiency in SQL for complex data manipulation and querying.

·       Experience with distributed computing frameworks such as Hadoop and Spark.

·       Strong understanding of cloud computing principles and AWS ecosystem.

·       Excellent problem-solving, analytical, and communication skills.

·       Experience working in agile environments is a plus.

·       This is a mandatory onsite role in McLean, VA.

 

Preferred:

·       AWS Solution Architect Certification (nice to have).