An exciting opportunity for an experienced Data Engineer to join the Engineering team working within the Digital & Product Department.
The successful candidate will be responsible for leading a team of data engineers as they design and create the pipelines that transform data in a scalable and repeatable way. This hands-on role will involve discussing requirements, designing solutions, and producing code that automates the ingestion, cleansing and transformations of data required to achieve the target data format.
You will work with the Products, Engineering, Data & Business Analytics teams to understand the source and target data requirements and use this knowledge to develop data integration solutions to achieve the necessary migrations.
The Data Engineer will support our software developers, PowerBI developers, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. You must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products.
Duties & Responsibilities:
- Understand, build and develop ETL and data integration solutions using a wide array of technologies and data sources.
- Manage and update the Data Engineering Architecture.
- Analyse and organise raw data from different sources.
- Explore ways to enhance data quality and reliability.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, etc.
- Work with cloud-based infrastructure (Azure) for hosting data solutions and applications.
- Collaborate with architects, data analysts and data scientists to help meet the business goals.
Skills Required:
- Previous experience as a data engineer.
- Proven experience in development and maintenance of ETL/ELT processes within a medallion architecture.
- Good experience in Data Modeling within a cloud-based data platform.
- Strong experience with SQL Server and relational databases.
- Good knowledge of the Azure data engineering stack, including Azure Synapse and Azure Data Lake.
- Programming experience in Python, PySpark and T-SQL.
- Familiarity with the wider Azure Data Solutions such as Azure ML Studio.
- Previous experience with Azure DevOps and understanding of CI/CD.
- Analytical skills related to working with structured and unstructured datasets.
- Excellent written and verbal communication skills.
- Experience supporting and working with cross-functional teams in a dynamic environment.