Overview
We are seeking a dynamic and highly skilled AZURE DATABRICKS ENGINEER to join our innovative data team. In this role, you will be at the forefront of designing, developing, and maintaining scalable data solutions leveraging Azure Databricks platform. Your expertise will drive the transformation of complex data into actionable insights, empowering our organization to make data-driven decisions with confidence. If you thrive in a fast-paced environment and are passionate about big data technologies, this opportunity is perfect for you to showcase your talents and grow professionally.
Duties
- Develop, optimize, and maintain large-scale data pipelines using Azure Databricks integrated with cloud services such as AWS and Azure.
- Design and implement efficient database schemas and data models for data warehouses utilizing tools like Oracle, Microsoft SQL Server, and Hadoop ecosystems.
- Write complex SQL queries, stored procedures, and scripts to extract, transform, and load (ETL) data from diverse sources including Big Data platforms, Oracle databases, and cloud storage.
- Collaborate with cross-functional teams to understand business requirements and translate them into scalable technical solutions using Java, Python, Shell Scripting, Bash (Unix shell), and VBA.
- Perform data analysis to identify trends, anomalies, and opportunities for process improvement while ensuring high levels of data quality and integrity.
- Utilize tools such as Informatica, Apache Hive, Spark, and Hadoop for processing large datasets efficiently.
- Conduct database design activities to support robust reporting and analytics capabilities within the data warehouse environment.
- Maintain comprehensive documentation of data workflows, architecture diagrams, and operational procedures to ensure transparency and ease of maintenance.
Skills
- Extensive experience with Azure Databricks platform for big data processing and analytics.
- Strong proficiency in cloud environments including AWS alongside Azure services.
- Deep understanding of database design principles for relational databases like Oracle and Microsoft SQL Server.
- Proven expertise in big data technologies such as Hadoop, Apache Hive, Spark, and related ecosystems.
- Skilled in programming languages including Java and Python for developing scalable data solutions.
- Hands-on experience with ETL tools like Informatica for seamless data integration workflows.
- Knowledge of SQL scripting for querying large datasets efficiently across multiple platforms.
- Familiarity with analysis techniques using VBA for automation within Excel or other Office applications.
- Shell scripting expertise (Bash/Unix shell) for automating routine tasks in Unix/Linux environments.
- Strong analytical skills with the ability to interpret complex datasets to support strategic decision-making.
Join us as an AZURE DATABRICKS ENGINEER to be part of a forward-thinking team committed to harnessing the power of big data! Your technical prowess will help shape innovative solutions that propel our organization into the future of analytics—empowering smarter business outcomes every day!
Pay: £250.00-£300.00 per day
Work Location: Hybrid remote in London
