Data Engineer

Data Engineer

The Everything Managed Group is a dynamic, innovative and technology driven business committed to leveraging data-driven insights to optimise operations and drive strategic decision-making. We are now seeking a talented Data Engineer to play a significant part in the design, delivery and ongoing evolution of the EMG data platform.

Summary

As a Data Engineer, you will play a critical role in transforming data into actionable insights and supporting our data infrastructure. You will collaborate with cross-functional teams to gather requirements, design data pipelines and transformations, and develop analytical solutions to address business challenges.

Responsibilities

• Collaborate with cross-functional teams, including system administrators, developers / engineers, external partners, and business stakeholders, to understand requirements and deliver solutions.
• Design, build and maintain scalable data pipelines to ingest, process and transform structured and unstructured data from diverse sources.
• Optimise solutions for performance, reliability, and scalability, ensuring efficient data processing and storage.
• Perform exploratory profiling and validation on new data sources to assess quality, structure and fitness for purpose prior to pipeline development.
• Implement and manage orchestration workflows to automate data pipeline scheduling monitoring and error handling.
• Design and implement data quality and consistency schemes to ensure the integrity and consistency of data, management information and reporting.
• Design and implement well-structured optimised data models that enable accurate, performant reporting and analysis by downstream stakeholders both within the team and across the wider organisation.
• Ensure the data platform is fully documented, maintaining up-to-date information requirements, data models, data dictionaries, and ETL mappings.
• Provide ad-hoc deep dive analysis, reporting requests and management information to business users.
• Communicate technical concepts and findings to non-technical audiences in a clear and understandable manner.
• Provide training and support to others, sharing best practices and fostering culture of continuous learning.
• Plan and prioritise work to ensure on time delivery of tasks.

Skills and Experience

• Degree in Mathematics, Statistics, Computer Science, Engineering or a related field or equivalent practical experience in a data engineering role. .
• Ideally experienced as a Data Engineer with hands-on experience designing and delivering data pipelines using industry-standard tools and cloud technologies.
• Knowledge of data engineering concepts, data pipeline development, ETL processes, data lake and data warehousing technologies.
• Proficiency in working with at least one major cloud data platform or lakehouse technology.
• Data modelling skills (including 3NF and dimensional modelling).
• Good knowledge of relational database technologies and concepts.
• Excellent analytical and problem-solving skills with a keen attention to detail.
• Good verbal and written communication skills with the ability to collaborate across teams and present findings to stakeholders.
• Experience with data transformation and/or distributed processing concepts, technologies and best practices is a benefit.
• Proficiency in Python for data engineering tasks (e.g. pipeline scripting, data transformation, automation)
• Experience using version control (Git) and familiarity with CI/CD practices for data engineering workflows.
• Ability to work both independently and collaboratively in a dynamic environment.
• Strong customer focus and stakeholder skills.
• Ability to prioritise tasks and work effectively in a fast-paced environment.