Platform Architecture & Engineering responsibilities:
-
Design and implement scalable Databricks Lakehouse platforms on AWS and/or Azure aligned to client requirements
-
Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers
-
Build and configure cloud infrastructure using infrastructure-as-code (e.g. Terraform & Declarative Automation Bundles(DAB's))
-
Establish secure, compliant environments including networking (VNet/VPC, Private Link), identity (IAM/Entra ID), data governance (Unity Catalog), and access controls
-
Define environment strategies (dev/test/prod), CI/CD pipelines, and release processes for Databricks deployments
-
Implement monitoring, logging, cost optimisation, and performance tuning across the platform
-
Design and implement data pipelines using Delta Live Tables, Auto Loader, and Databricks Workflows for both batch and streaming workloads
Client Delivery & Enablement responsibilities:
-
Work directly with clients to translate business and technical requirements into scalable platform designs
-
Lead technical workshops, architecture sessions, and whiteboarding engagements with client stakeholders
-
Support rapid prototyping and proof-of-concept builds within Databricks to demonstrate platform capabilities and accelerate client adoption
-
Provide best practice guidance on Lakehouse architecture, data modelling, workload optimisation, and cost management
-
Produce high-quality technical documentation including architecture diagrams, architecture decision records (ADRs), runbooks, and deployment guides
-
Enable client teams through structured knowledge transfer, training, and platform handover
-
Collaborate with data engineers, data scientists, and product teams to ensure successful delivery outcomes
Governance & Security:
-
Implement Unity Catalog for centralised data governance, including access control (RBAC/ABAC), data lineage, audit logging, and compliance enforcement
-
Apply security best practices across platform design: network isolation, secret management, encryption at rest and in transit, and identity federation
-
Ensure platform designs meet client regulatory and compliance requirements (e.g. GDPR, ISO 27001, sector-specific standards)
What success looks like in the role:
-
Delivery of robust, secure, and scalable Databricks platforms that meet client performance and cost expectations
-
Clear, well-architected solutions that balance flexibility, governance, and operational efficiency
-
Strong client relationships built on trust, technical credibility, and effective communication
-
Accelerated client adoption of the Lakehouse platform through well-designed enablement and documentation
-
Reduced deployment time through reusable infrastructure patterns and automation
-
Proactive identification of risks, trade-offs, and optimisation opportunities across platform design and delivery
-
Contribution to the organisation's growing body of reusable platform accelerators, reference architectures, and internal knowledge
Competencies and Behaviours:
-
3+ years experience in data platform engineering, cloud engineering, or similar roles
-
Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows
-
Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/VPC, IAM)
-
Experience with infrastructure-as-code tools (e.g. Terraform preferred) and CI/CD pipelines (e.g. Azure DevOps, GitHub Actions)
-
Solid understanding of data architecture concepts including Lakehouse medallion architecture and dimensional modelling
-
Familiarity with security and governance frameworks (e.g. RBAC, ABAC, data masking, audit, compliance standards)
-
Excellent communication skills with the ability to explain complex technical concepts to non-technical stakeholders
-
Comfortable working in a client-facing consultancy environment with multiple concurrent engagements
-
Proactive, self-driven, and able to take ownership of end-to-end platform delivery
-
Willingness to travel within the UK as required
-
Right to work in the UK
