Job Summary
We are seeking an experienced Senior Data Engineer to join our dynamic team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems and data warehouses. This role requires a strong technical background in big data technologies, database design, and programming languages, with a focus on delivering high-quality data solutions that support strategic decision-making and business insights. The ideal applicant will possess excellent analysis skills and a proactive approach to problem-solving within a collaborative environment.
Duties
- Develop and optimise scalable data pipelines using technologies such as Apache Spark, Hadoop, and Apache Hive.
- Design and implement efficient database schemas for data warehouses using Oracle, Microsoft SQL Server, and other relational databases.
- Build robust ETL processes employing tools like Informatica to ensure accurate and timely data integration.
- Utilise programming languages including Python, Java, VBA, Bash (Unix shell), and Shell Scripting to automate workflows and enhance system functionality.
- Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions.
- Maintain and improve existing data infrastructure by troubleshooting issues, performing system upgrades, and implementing best practices.
- Analyse large datasets to identify trends, patterns, and anomalies that inform strategic business decisions.
- Ensure compliance with data governance standards and optimise system performance through regular monitoring.
Requirements
- Proven experience as a Data Engineer or similar role within a big data environment.
- Strong proficiency in AWS cloud services for scalable data storage and processing solutions.
- Extensive knowledge of SQL databases including Oracle, Microsoft SQL Server, as well as experience with Data Warehouse design principles.
- Hands-on experience with big data frameworks such as Hadoop, Spark, Apache Hive, and related tools.
- Proficiency in programming languages including Python, Java, VBA, Bash (Unix shell), and Shell Scripting.
- Solid understanding of database design concepts and ETL processes using Informatica or comparable tools.
- Excellent analysis skills with the ability to interpret complex datasets effectively.
- Familiarity with analysis tools such as VBA or other scripting languages is advantageous.
- Strong organisational skills with the ability to manage multiple projects simultaneously in a fast-paced environment.
- Relevant educational background in Computer Science, Information Technology or related fields is preferred. This role offers an exciting opportunity for a motivated professional eager to contribute to innovative data solutions within a forward-thinking organisation.
Pay: £90,000.00-£100,000.00 per year
Benefits:
- Company pension
- Cycle to work scheme
- Free parking
- Life insurance
- On-site parking
- Private medical insurance
Work Location: In person
