Senior Data Engineer Only GC/USC

Role: Senior Data Engineer

Location: Remote 

Only GC/US Citizens

Basic Qualifications

. Think and communicate critically about architecture, design, and best practices and

guide your team to adopting them.

. Design data systems that allow managed growth of the data model to minimize risk and

cost of change.

. Write transformation and validation code that applies complex data aggregation and

calculation using SQL and Python

. Drive implementation of automated testing for data pipelines within a CI environment

. Create new pipelines or rewrite existing pipelines and build reusable components at

scale to support accounting functions, as well as reporting & analytics.

. Collaborate with other Disney and Hulu teams to identify and document shifting data

requirements while also advocating for a minimal change set for your team.

. Solve complex data issues and perform root cause analysis to proactively resolve

product and operational issues.

. Collaborate with leadership and other engineers to develop technical story backlog

derived from high level business requirements and design collaboration and estimating

story points.

. BS or MS in Computer Science, a related field, or equivalent industry experience

. 3 years of professional experience engineering complex, high-volume data pipelines

using SQL, Python, and Airflow

. 3 years of experience building cloud scalable and high-performance data lake / data

warehouse solutions using AWS products – S3, Athena, Glue, and EMR

. Experience with binary data serialization formats such as Parquet

. Deep understanding of data structures and algorithms

. Understanding of code versioning tools such as GIT

. Have a passion for data solutions

Preferred Qualifications Nice to Have:

. Exposure to AWS cloud data pipeline tools such as Managed Airflow and Glue

. Experience integrating with Ad Tech platforms such as Operative and STAQ

. Exposure and opinions regarding alternate orchestration tooling beyond Airflow

. Understanding of SOX compliance needs and how they affect system design.

. Have worked with a variety of Airflow Operator types, including REST, Lambda, ECS

. Can flex between Python and Javascript/Typescript.

Technical Environment

. Aurora/Hive (databases)

. Spark (large-scale data processing)

. Airflow (workflow management)

. Docker (software packaging and delivery)

. AWS (development and hosting)