Job Overview
We are seeking a dynamic and experienced Lead Data Engineer to spearhead our data infrastructure initiatives and drive innovative data solutions. In this pivotal role, you will lead a talented team of data engineers, architect scalable data pipelines, and ensure the integrity and security of our enterprise data ecosystem. Your expertise will enable data-driven decision-making across the organization, fostering a culture of continuous improvement and technological excellence. If you thrive on solving complex data challenges and possess a passion for building robust systems, this opportunity is for you! This is a hybrid position with 3 days working in the London office.
Duties
- Lead the design, development, and maintenance of large-scale data pipelines and architectures utilizing technologies such as Hadoop, Spark, and Apache Hive.
- Collaborate with cross-functional teams to understand data requirements and translate them into efficient database designs and scalable solutions.
- Oversee the integration of diverse data sources including Oracle, Microsoft SQL Server, and cloud platforms like Azure to support analytics and reporting needs.
- Develop and optimize SQL queries, stored procedures, and scripts using VBA, Bash (Unix shell), Shell Scripting, Python, and Java to automate processes and enhance system performance.
- Ensure data quality, security, and compliance by implementing best practices in database management and governance.
- Mentor junior team members in technical skills such as database design, big data processing, and scripting languages.
- Drive continuous improvement initiatives by evaluating emerging technologies like Informatica for ETL processes or Spark for real-time analytics.
Skills
- Deep expertise in Azure cloud services for scalable data storage and processing solutions.
- Strong programming skills in Java, Python, VBA, Bash (Unix shell), and Shell Scripting for automation and application development.
- Extensive experience with big data frameworks including Hadoop, Spark, Apache Hive, and related tools for processing large datasets efficiently.
- Proficiency in designing robust data warehouses using Oracle, Microsoft SQL Server, or similar relational databases.
- Solid understanding of database design principles to create optimized schemas supporting analytical workloads.
- Excellent analysis skills to interpret complex datasets and derive actionable insights.
- Familiarity with ETL tools such as Informatica to streamline data integration workflows.
Join us as a Lead Data Engineer to shape the future of our enterprise data landscape! Bring your passion for innovation, your technical mastery across diverse platforms, and your leadership skills to empower teams that transform raw data into strategic assets. Together, we’ll build systems that propel our organization forward with agility and precision!
Pay: £80,000.00-£100,000.00 per year
Benefits:
- Company pension
- Cycle to work scheme
- Private medical insurance
Experience:
- Azure: 5 years (required)
Work authorisation:
- United Kingdom (required)
Work Location: Hybrid remote in Camden Town NW1
