Location: Columbus, OH – Onsite
Contract
Job Summary:
Contribute to building brand new data platforms in AWS, using Java and Spark. Be part of a dynamic team, building data solutions in a supportive and hybrid work environment. You will design and implement micro services and data pipelines using Java, Sprint Boot, Kafka, Spark and AWS services. Success in this role requires technical expertise, strong problem-solving skills, and the ability to collaborate effectively within an agile team.
Must Have Tech Skills:
- Demonstrable experience as a senior data engineer.
- Expert in Java and Spark, with a deep focus on data transformations, processing and data engineering practices.
- Expert in micro service implementation using Java, Sprint Boot, Kafka and Kubernetes.
- Good knowledge on AWS services like EKS, S3, SQS, Lambda, SNS, MSK.
- Experience in implementing tests using Junit, Mockito, Cucumber and Karate.
- Good knowledge in performance tuning Spark jobs and micro services.
Nice To Have Tech Skills:
- Experience in solution architecture and technical design, allowing for the creation of scalable, reliable data architectures that meet both technical and business requirements
- A master’s degree or equivalent experience or relevant certifications (e.g., AWS Certified Solutions Architect, Certified Data Analytics) is advantageous
Key Accountabilities:
- Provides guidance on standard methodologies in design, development, and implementation, ensuring solutions meet business requirements and technical standards.
- Drive the migration of existing data processing workflows to the Lakehouse architecture, using Iceberg capabilities.
- Communicates complex technical information clearly, tailoring messages to the appropriate audience to ensure alignment.
- Good ability in debugging, problem solving and performance tuning of micro services and high volume spark applications.
Key Skills:
Deep technical knowledge of data engineering solutions and practices. Implementation of data pipelines using Kafka, Spark and AWS services.
Highly proficient in Java, Spark and familiar with a variety of development technologies. This knowledge enables the Senior Data Engineer to adapt solutions to project-specific needs. Proficient in creating clear, comprehensive documentation. Ensures that documentation supports technical teams' knowledge and compliance, making it accessible and valuable for future reference Proficient in quality assurance practices, including code reviews, automated testing, and standard methodologies for data validation.
Experience in using automation tools and Continuous Integration/Continuous Deployment (CI/CD) pipelines to streamline development, testing, and deployment.
Thanks & Regards.
Simranjeet Kaur
(+1) 609-606-9010 ext.1750
Simranjeet.kaur@compunnel.com
