Senior Data Engineer – eDV Cleared
Locations: Manchester, Gloucester or London (Full-Time, Permanent, Onsite)
We are partnering with a leading defence and technology organisation to recruit an experienced Senior Data Engineer to join their highly specialised engineering teams across Manchester, Gloucester or London.
This is an excellent opportunity to work on complex, mission-critical programmes within the National Security sector, delivering innovative solutions across AI, machine learning, cyber security and geospatial intelligence.
As a Senior Data Engineer, you will join a collaborative Agile environment, playing a key role in designing, building and maintaining scalable data platforms and processing pipelines. You will work alongside software engineers, architects and stakeholders to transform and optimise data for advanced analytical and operational use.
The team is actively hiring Data Engineers across multiple levels, offering opportunities for both experienced professionals seeking technical leadership and individuals looking to further develop their expertise within a high-performing engineering function.
*Please note: Due to the nature of the work, candidates must hold current and active eDV (Enhanced Developed Vetting) clearance.
Key Responsibilities
- Design, develop and maintain robust data pipelines to ingest, clean, transform and aggregate data from a variety of sources.
- Collaborate closely with engineers, technical leads and stakeholders to deliver project objectives.
- Support the successful delivery of project milestones within Agile teams.
- Contribute to continuous improvement initiatives, engineering best practices and team capability development.
- Influence technical direction and contribute to architectural and design decisions.
Required Skills & Experience
- Strong analytical skills with experience working with large-scale and unstructured datasets.
- Python development experience, including technologies such as PySpark, Pandas and PyArrow.
- Expertise in distributed data processing frameworks, particularly Apache Spark.
- Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi.
- Cloud platform experience across AWS, Azure and/or GCP.
- Knowledge of messaging and streaming technologies including Kafka, AWS SQS or equivalent cloud-native services.
- Strong understanding of SQL and NoSQL databases.
- Experience with modern data storage technologies including HDFS, Iceberg, Elasticsearch, S3 and Data Lake architectures.
- Containerisation and orchestration experience using Docker, Kubernetes and/or OpenShift.
- Knowledge of testing frameworks, engineering standards and software development best practices.
- Experience working with source control systems, branching strategies, pull requests and code review processes.
What's on Offer
- Opportunity to work on highly secure, cutting-edge programmes with real-world impact.
- Exposure to advanced technologies across AI, machine learning, cyber security and data engineering.
- Structured training and development support to help broaden your technical capabilities.
- Clear progression opportunities within a growing and highly regarded engineering function.
- Collaborative Agile environment working alongside some of the industry's leading technical specialists.
Benefits:
- Competitive salary and excellent benefits
- eDV bonus on top of salary and annual bonus
- Company bonus scheme
- Contributory Pension Scheme (up to 10.5% company contribution)
- 6 times salary ‘Life Assurance’ with pension
- 25 days holiday (increasing with service) + statutory public holidays, plus opportunity to buy and sell up to 5 days
If you're an eDV-cleared Data Engineer looking to work on some of the UK's most challenging and rewarding technology programmes, we'd be keen to discuss this opportunity further.
