As a Senior Data Engineer, you will be at the forefront of designing and building innovative data solutions within a fast-paced agile data engineering team. You will play a crucial role in modernizing our data platform to enhance product offerings, closely collaborating with stakeholders across the organization to maximize the value of data for reporting, analytics, and decision-making processes.
In this role, you'll help transform our cloud data landscape, working with streaming technologies, cloud data warehouses, and Lakehouse architectures to unlock new insights and capabilities for our business units. Additionally, you will leverage automation and generative AI to increase operational efficiency and enhance self-service capabilities across the organization.
Data Platform & Architecture
Develop and maintain robust and efficient data solutions and products including data transformation, data modelling, and reporting on both on-premises and cloud environments
Build modern data lake/lakehouse architectures that support scalable, flexible data processing
Enhance platform integration by leveraging domain events to enable more efficient and real-time data processing
Self-Service Analytics & Enablement
Champion the adoption of generative AI features such as Amazon Q and Copilot to deliver intuitive analytics experiences
Build sophisticated solutions to deliver actionable insights, with a strong focus on performance, scalability, and data accessibility
Engineering Excellence & Automation
Enhance data engineering processes through automation, implementing CI/CD pipelines, and optimizing ETL/ELT flows to improve efficiency and maintainability
Be a champion of best development practices on existing and new technologies
Collaborate with cross-functional teams to translate business requirements into technical solutions, documenting them as GAPs and aligning them with Architects
Data Governance & Discovery
Facilitate data discovery and enhance data management processes using platforms like Open Metadata
Ensure KPIs and documentation remain accurate and up to date
Support data quality and compliance standards across the organization
Innovation & Operations
Initiate and experiment with new ideas and approaches that support the A&I department's strategy
Proactively support and maintain data products, including handling release management, troubleshooting incidents, and providing on-call support as necessary
Required
5+ years of experience in data engineering, with demonstrated expertise in designing and implementing scalable, flexible modern data architectures
Experience building modern data lake/lakehouse architectures (e.g., Apache Iceberg) in cloud environments
Strong knowledge of AWS (S3, Redshift) or Azure data services
Experience with ETL/ELT processes and data ingestion
Proven expertise using transformation tools such as dbt
Experience with data pipeline orchestration tools (such as Apache Airflow)
Experience with best practices applied to data (e.g., testing, CI/CD)
Advantageous
Experience developing dashboards and reporting solutions using data visualization tools such as Power BI or Amazon QuickSight
Experience with generative AI applications in data analytics
Nice to have some JAVA experience
