Databricks Architect Urgent C2C Requirement – Databricks Architect – Raritan, NJ (Onsite day 1)

Responsibilities:

1. Architecture Design:
   

   • Design end-to-end Databricks architectures that align with business goals and data requirements.
     
   • Collaborate with data scientists, analysts, and engineers to understand project objectives.
     

2. Cloud Integration:
   

   • Implement Databricks solutions on both Azure and AWS platforms.
     
   • Ensure seamless integration with cloud-based data storage, processing, and analytics services.
     

3. Data Pipelines:
   

   • Develop robust data pipelines using Databricks for efficient data processing.
     
   • Optimize and tune Spark jobs to enhance performance and reliability.
     

4. Security and Compliance:
   

   • Implement security best practices to safeguard sensitive data processed by Databricks.
     
   • Ensure compliance with relevant data governance and regulatory requirements.
     

5. Collaboration:
   

   • Work closely with cross-functional teams to understand analytics and machine learning requirements.
     
   • Collaborate with cloud architects to optimize overall cloud infrastructure.
     

6. Performance Optimization:
   

   • Identify and resolve performance bottlenecks in Databricks workflows.
     
   • Implement caching and partitioning strategies for optimal data processing.
     

7. Documentation:
   

   • Create and maintain comprehensive documentation for Databricks architectures and processes.
     
   • Provide training and support to other team members as needed.
     

8. Monitoring and Troubleshooting:
   

   • Implement monitoring solutions to track Databricks job performance.
     
   • Troubleshoot and resolve issues related to data processing and analytics.
     

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field.
  
• Proven experience (5+ years) as a Databricks Architect with expertise in both Azure and AWS.
  
• Strong proficiency in Apache Spark and Databricks environments.
  
• Experience with cloud-native data storage and analytics services (e.g., Azure Data Lake, AWS S3, Athena).
  
• Solid understanding of data security, compliance, and governance principles.
  
• Excellent problem-solving and troubleshooting skills.
  
• Certifications in Databricks, Azure, or AWS are a plus.