Get all C2C Jobs / hotlists 🔥 Alerts

GCP Data Engineer-with Java

Contract

VL INFOTECH INC

CORE TECHNICAL REQUIREMENTS 

 

·                GCP Core Data Stack: Hands-on production experience building, maintaining, and supporting scalable batch and streaming data pipelines on Google Cloud Platform, specifically BigQuery and Google Cloud Storage (GCS).

·                Cloud Composer / Apache Airflow: Proven ability to author, schedule, troubleshoot, and optimize complex Airflow DAGs in a managed Cloud Composer environment (custom operators, task dependencies, retries, backfills, log diagnostics, and runtime/SLA tuning).

·                Google Cloud Dataflow (Java): Hands-on experience building, maintaining, and debugging Apache Beam pipelines written in Java on Google Cloud Dataflow, including job monitoring and performance troubleshooting.

·                Dataform (SQLX): Practical experience developing and debugging SQLX ELT transformations, data quality assertions, and dependency graphs in Dataform.

·                Advanced BigQuery SQL & Optimization: Deep SQL proficiency in BigQuery, including partitioned (_PARTITIONTIME) and clustered tables, query performance tuning, slot/cost optimization, and querying operational metadata/telemetry tables.

·                Python for Data Engineering: Strong python3 skills for writing and debugging production-grade Airflow tasks, API integrations, and automation scripts.

·                Production Support, RCA & Data Validation: Demonstrated ability to own production reliability—monitoring daily DAG/task runs, performing deep root-cause analysis (RCA) on failures or data discrepancies, building automated data validation/reconciliation checks, and deploying permanent fixes within SLA.

·                Data Modeling & Onshore/Offshore Collaboration: Solid grasp of dimensional modeling and modern ELT patterns for high-volume datasets (e.g., e-commerce or consumer/audience data), paired with clear communication for daily onshore/offshore handoffs and stakeholder alignment.

To apply for this job email your details to jags.kamma@vlinfotec.com

×

Post your C2C job instantly

Quick & easy posting in 10 seconds

Keep it concise - you can add details later
Please use your company/professional email address
Simple math question to prevent spam