About Scrumconnect
Scrumconnect Consulting is a UK-based digital transformation consultancy delivering agile, secure, and scalable technology solutions for public and private sector clients. This is a fully remote role based in India, supporting UK operations and client engagements. The successful candidate will work closely with UK leadership teams and must be comfortable operating within UK regulatory, governance, and delivery frameworks.
About the Role
We are looking for an experienced Test Manager to lead quality assurance and testing activities within a large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services.
This role combines hands-on technical leadership with test strategy ownership. You will be responsible for defining and driving the overall testing approach across automated pipelines, data workflows, cloud infrastructure, and release delivery while managing risks, improving testing standards, and mentoring engineering teams.
The ideal candidate will have strong experience in Big Data and Data Lake environments, including testing both structured and unstructured datasets, and will be comfortable operating in highly automated CI/CD-driven delivery environments.
You will work closely with engineering, DevOps, architecture, and stakeholder teams to ensure quality is embedded throughout the software delivery lifecycle.
Key Responsibilities
Test Strategy & Leadership
Define, implement, and manage the overall test strategy across cloud-native data platforms and applications
Lead end-to-end testing activities across data pipelines, cloud services, APIs, and distributed systems
Establish quality standards, governance processes, and testing best practices across teams
Drive a quality-first engineering culture across the programme
Provide leadership, mentoring, and coaching to Test Engineers and QA team members
Automated Testing & Frameworks
Design, implement, and oversee scalable automated test frameworks for data and application testing
Build and maintain automated test suites across AWS cloud infrastructure and data workflows
Enhance existing automation frameworks to improve reliability, coverage, maintainability, and reporting
Ensure automated testing is fully integrated within GitLab CI/CD pipelines for rapid feedback and release confidence
Big Data & Data Lake Testing
Lead testing activities across Big Data and Data Lake solutions containing structured and unstructured datasets
Validate large-scale data ingestion, transformation, and processing workflows
Ensure data integrity, transformation accuracy, reconciliation, and performance under load
Oversee PySpark-based data validation and interrogation activities on AWS EMR
Support testing and monitoring of workflows orchestrated through Apache Airflow
Drive root cause analysis for production data issues and platform defects
Risk Management & Reporting
Identify, assess, and communicate quality risks across releases and environments
Analyse test execution metrics, defects, trends, and quality indicators
Translate technical findings into clear, prioritised reporting for technical and non-technical stakeholders
Provide regular status reporting to programme leadership and client stakeholders
CI/CD & DevOps
Collaborate with DevOps and engineering teams to improve release quality and deployment confidence
Ensure automated test execution within GitLab CI/CD pipelines
Support infrastructure testing and validation within Terraform-managed cloud environments
Contribute to continuous improvement initiatives across testing and delivery processes
Technology Stack
Apache Airflow, Amazon Athena, AWS S3, AWS Glue, AWS EMR, AWS EC2, AWS ECR, AWS DynamoDB, AWS CloudWatch, AWS IAM, Python, SQL, Java, Bash, GitLab CI/CD, Jupyter Notebooks, Apache Spark / PySpark, Terraform, Docker
Key Skills
Strong proficiency in Python, SQL, Java, and scripting languages such as Bash
Hands-on experience testing Big Data platforms and Data Lake solutions
Strong experience with AWS EMR, Apache Airflow, and Jupyter Notebooks
Experience performing PySpark-based data interrogation, validation, and testing
Strong knowledge of AWS services including S3, Glue, Athena, EMR, CloudWatch, IAM, DynamoDB, EC2, and ECR
Experience designing, implementing, and managing enterprise-scale automated test frameworks
Strong understanding of CI/CD practices using GitLab pipelines
Experience with infrastructure-as-code tools such as Terraform
Familiarity with Docker and containerised testing environments
Understanding of dimensional data models and slowly changing dimensions
Experience validating structured and unstructured data workflows
Strong analytical and root cause analysis skills
Ability to communicate technical risks clearly to stakeholders at all levels
Experience leading and mentoring QA/Test Engineering teams
Skills & Experience Required
Proven experience as a Test Manager, Lead Test Engineer, or QA Lead within cloud and data engineering environments
Strong background in automated testing for cloud-native and data-centric platforms
Experience testing large-scale data pipelines, transformation processes, and distributed systems
Experience managing or leading QA/testing teams in Agile delivery environments
Strong stakeholder management and communication skills
Experience establishing test governance, reporting, and quality metrics
Strong problem-solving and decision-making capability
Experience working within regulated or security-conscious delivery environments
SFIA Level 5 capability — provides leadership, influences strategy, and drives quality improvements
Active SC clearance (mandatory)
Diversity and Inclusion
At Scrumconnect Consulting, we believe diversity drives innovation and stronger outcomes for our teams and clients. We are committed to creating an inclusive environment where every individual is respected, valued, and supported. We welcome applications from candidates of all backgrounds, experiences, and perspectives.
