Title : Cloud Data engineer
Work Remote
Position Type- Contract
Â
Visa – H1B, H4-EAD, GC-EAD, L2 EAD, TN EAD, USC only
GC on W2 only
Â
Experience:Â Â 9+ Years
Job Description :Â
Â
Tech Stack:Â Scala, Apache Spark, Azure Data Factory, Azure Cosmos DB, Cosmos Scope, TypeScript, c#, Kusto/KQL (plus), Azure Fabric (plus), PySpark (plus)
• Onsite (10+ yrs): Own and maintain production Scala/Spark pipelines; lead technical decisions independently
• Ramp on Cosmos Scope language and TypeScript
• Kusto/KQL proficiency strongly preferred; Azure Fabric and PySpark are nice-to-haves for ad hoc analysis
Â
Â
Expectations for the Role (Evaluation Criteria)
Candidates for this role should demonstrate strong hands-on knowledge in the following areas:
|
# |
Area |
Expectation |
|
1 |
Spark Internals |
Performance tuning and debugging — understanding of DAG, shuffle, partitioning, caching, etc. |
|
2 |
Medallion Architecture |
Practical understanding of Bronze/Silver/Gold layered data architecture. |
|
3 |
Schema Evolution |
Handling schema changes in evolving datasets without breaking pipelines. |
|
4 |
Photon / Vectorization |
Awareness of implicit Photon engine or vectorized query execution support and its performance impact. |
|
5 |
Slowly Changing Dimensions (SCD) |
Ability to design and implement SCD Type 1/2/3 in modern pipelines. |
|
6 |
Design Process Thinking |
Example: designing a job to calculate unique monthly active users — covering logic, deduplication, and scalability. |
|
7 |
Resource Allocation & Debugging |
Executor/resource tuning, methodical troubleshooting and debugging approach for Spark jobs. |
Â
Â
Skill Matrix
|
Skills |
Years of Experience |
Candidate Self Rating (Scale of 1 to 5) |
|
SCALA |
 |
 |
|
Apache Spark |
 |
 |
|
Azure data factory |
 |
 |
|
Azure CosmosDB |
 |
 |
|
Pyspark |
 |
 |
|
Typescript |
 |
 |
Â
Â
Â
Â
Thanks & Regards
Vivek Verma | Executive Recruitment
TekIntegral Inc | 555 Republic Drive, Suite 240 Plano, TX USA 75074
