Role: Databricks solution Architect with Workspace hands on
Location: Remote
Â
CONTEXT CURRENT STATE
Bains Polaris 20 program is a firmwide data platform transformation built on Databricks with a medallion architecture Bronze Silver Gold and Unity Catalog governance The platform is in active execution multiple data domains are progressing through silver ingestion with Gold layer certification underway for priority business products including the Commercial Dashboard Revenue Reporting and GROM
While the platform infrastructure is established and delivery is accelerating, we do not yet have consistent repeatable operational processes in place to support the growing footprint The program needs an experienced Databricks practitioner to bring structure and standards to how the workspace is managed and monitored creating the operational backbone that the team and downstream business users will rely on
Currently we have one custom application in production pulling from Databricks with several data owners accessing the gold layer to certify data We have already seen early requests to get into the platform and the transparency and enablement doesn’t seem to be sustainable Our business goal by end of the year is to have all functional teams on the workspace and support requests by report writers and data consumers to leverage our certified enterprise data We need a clean foundation that is scalable and low overhead to manage
Â
SCOPE OF WORK
The Workspace Enablement Lead will design and implement the operational standards and tooling required to run Polaris 20 at scale Core deliverables include
Â
Workspace governance setupÂ
Define and enforce naming conventions folder structures and access control patterns within Unity Catalog
Set up and standardize cluster policies compute configurations and cost allocation tagging
Provide best practices around data request access processes data consumer data owner
Â
Data quality standards
Implement automated quality checks integrated into the ingestion pipeline
Establish escalation and remediation workflows when quality checks fail
Â
Monitoring observability dashboardsÂ
Build operational dashboards covering pipeline health SLA performance compute costs and data freshness
Set up proactive alerting for platform engineering and data ownership teams
Define KPIs and SLAs for platform operations e.g. ETL run time uptime targets
Â
Handoff knowledge transfer
Produce comprehensive documentation for all standards templates and configurations
Conduct structured handoff to the internal team and/or incoming maintenance resource
Â
IDEAL PROFILE
- 7 years Hands-on Databricks platform oversight experience Unity Catalog Delta Lake DLT SQL Warehouses
- Demonstrated experience building operational standards and governance frameworks not delivering pipelines
- Experience with monitoring and observability in data platforms alerting dashboards SLA management
- Excellent documentation and communication skills this role produces artifacts others will use long after the engagement ends
- Prior professional services or management consulting environment experience is a plus
Â
|
Neha Chaudhary
Survey Link:Â The survey is anonymous, and the information collected will be analysed in aggregate to support initiatives that improve the total experience across Compunnel.
|
Â
—