DATASTAGE DEVELOPER WITH STRONG SPARK & HIVE

Required Skills : Technical Skills : IBM Infosphere D ,Oracle ,Talend OpenStudio for Big Data Domain Skills : Healthcare Payer

Nice to have skills : Technical Skills : Unix Shell Script ,SQL Scripting ,Scala ,Spark Domain Skills : Enrollment ,Claims
Roles & Responsibilities :
Requirement gathering analysis design coding and implementation of ETL IBM Datastage, Talend, Oracle, UNIX Shell Script and Big Data skills Spark Scala Hadoop Hive Kafka in Hadoop Oracle-Facets ecosystem.
Understanding the requirement and converting into Spark code by using Scala language
Create Spark framework by using Scala for data load from Landing to Staging and Staging to EDL layers
Create complex DDL scripts for Hive tables/Oracle Tables.
Create Data Frames by using Hive Context for data filtering.
Create Spark Hive Queries for maintaining and filtering Duplicate Data
Create Hive complex data type tables analyze and perform Hive queries Stored Data Frame as Parquet file format into Hive.
Involved in defect fixes Participate in calls of daily interaction with clients.
Experience in Agile with CICD delivery models.
Ensure compliance with applicable code standards and architectures.
Review troubleshoot and remediate potential coding issues Enhancement of Datastage/Talend and Big Data process using Spark Scala Hadoop Hive and Kafka
Good Analytical skills critical thinker must have good communication skills must be a team player and individual contributor AgileDevOps background