Location: San Antonio
Contract
Job Description:
· Identify and select the data sources, Tables, Relationships between different entities in Snowflake.
· Establish a secure connection between Snowflake and AWS Neptune via AWS service stack by
· Securing extracted data storing in S3 for transformation before loading into AWS Neptune.
· Collaborate with current platform team to understanding data structure to implement data
· Extraction processes from Snowflake.
· Load the extracted data into AWS S3 for graph model transformation.
· Analyze the data, relationships, entities mappings to determine the necessary graph schema.
· Design nodes, edges, and properties for the graph schema using entities definitions.
· Implement the graph schema in AWS Neptune.
· Creating indexes to improve query performance
· Review and refine schema based on query patterns and performance based on mutually agreed
· Design
· Define the transformation logic using DBT.
· Develop DBT models for data transformation.
· Schedule and automate the ELT pipeline using DBT and Snowflake.
|
M: +1 213-668-5856 | E: Javed.m@twsol.com |