Lead Data Engineer (Azure | Databricks | Fabric | PySpark | SQL)
Permanent | Hybrid | UK Public Sector Project
Overview
We are seeking a hands-on Lead Data Engineer to join a growing Data & Analytics practice. This role focuses on designing, building, optimising, and maintaining enterprise-level data solutions.
You will work across the full data lifecycle, delivering scalable, high-performance solutions using Azure and tools such as Databricks and Fabric.
Who We’re Looking For
A strong Data Engineer with deep technical expertise in Azure, Databricks and Fabric, who enjoys working in a collaborative, client-facing environment and thrives at designing and delivering enterprise-level data solutions, spanning initial requirements gathering, proof-of-concept development, architecture planning, deployment and delivery.
Key Responsibilities
- Develop solutions using Azure Databricks and/or Fabric
- Implement Medallion Architecture (Bronze, Silver, Gold)
- Develop data warehouse and lakehouse solutions
- Support BI, analytics, and AI workloads
- Optimise performance, cost, and reliability
- Collaborate with technical and functional stakeholders
Mandatory Requirements
- Proven ability to make both high-level strategic and day-to-day operational decisions with confidence
- Strong leadership and interpersonal skills, with a track record of motivating and influencing stakeholders at all levels
- Sharp analytical mindset with the ability to proactively spot emerging opportunities and resolve challenges before they escalate
- Strong hands-on experience with Azure Databricks and Fabric
- Strong SQL and data warehousing expertise
- Experience with data modelling
- Experience with Databricks AI/BI or Power BI
- Experience with CI/CD and DevOps practices
Desirable
- Experience in consulting / professional services
- Exposure to AI and advanced analytics solutions
- Exposure to Agentic AI systems built on top of Large Language Models
- Experience with real-time / streaming data
- Experience with Infrastructure as Code (Terraform or Bicep)
- Experience in regulated industries
- Passionate about staying at the forefront of industry developments, with a habit of actively monitoring emerging trends, tools, and technologies
- Databricks/Microsoft Azure certifications
Soft Skills
- Strong problem-solving skills
- Collaborative and team-oriented mindset
- Clear communication and documentation skills
- Ability to work in fast-paced delivery environments
Contract Details
- Role Type: Permanent & Hybrid (2 days per week)
- Clearance: Candidates must be eligible for UK Security Clearance (SC) or active SC
