Data Engineer

We are seeking an experienced Data Engineer to design, build and maintain scalable data solutions on our Databricks platform. The successful candidate will be responsible for developing robust ETL/ELT processes, building high-performance data pipelines using PySpark, and integrating data from a variety of source systems into our Lakehouse environment.

This role will play a key part in enabling data-driven decision making by delivering reliable, high-quality and governed data assets. The ideal candidate will have strong experience with Databricks, modern cloud-based data platforms, Data Lakehouse architecture, Medallion Architecture principles and enterprise-scale data engineering best practices.

Key Accountabilities

• Design, develop and maintain scalable data pipelines within the Databricks platform.
• Build and optimise ETL/ELT processes using PySpark and Spark SQL.
• Implement and support Data Lakehouse and Medallion Architecture frameworks.
• Ingest, transform and integrate data from multiple internal and external source systems.
• Ensure data quality, accuracy, consistency and governance across data platforms.
• Monitor, troubleshoot and optimise data processing workloads and platform performance.
• Support the delivery of analytical, reporting and operational data solutions.
• Collaborate with the Product Owner, Business Analysts and business stakeholders to understand and deliver data requirements.
• Contribute to the development of data engineering standards, best practices and reusable frameworks.
• Support deployment, release management and CI/CD processes for data solutions.

• Bachelor’s degree in Computer Science, Information Technology, Data Science or a related discipline.
• Microsoft Certified: Azure Data Engineer Associate (preferred).
• Databricks Certified Data Engineer Associate or Professional (preferred).
• Azure Fundamentals or Azure Data Fundamentals certifications (desirable).
• Relevant cloud, data engineering or analytics certifications are advantageous.

Experience

• Minimum of 4+ years of hands-on experience specifically within Databricks and PySpark environments.
• Broad background in Data Warehousing (10+ years preferred).
• Strong hands-on experience building ETL/ELT pipelines using PySpark and Spark SQL.

With over 5,000 employees based in more than 80 countries across the globe, our teams provide industry leading expertise that spans:

• Consumer Engagement
• Parts, Accessories & Service Performance
• Actionable Insights
• Repair Optimization & Compliance
• Learning Solutions
• Distribution & Sales Performance

Our proven track record means that we now partner with almost every car manufacturer on the market.

The MSX Purpose

To empower Movers and Makers to thrive in our ever-changing world

The MSX Mission

To harness our expertise in mobility, the creativity of our global teams, and the power of technology, to craft tailored, sustainable and innovative solutions.

The MSX Vision

To be the clients’ first choice, recognized for our operational excellence and commitment to driving change and innovation in the mobility industry.

MSX is an equal opportunities employer and encourages applications from suitably qualified and eligible candidates regardless of sex, race, disability, neurodiversity or other personal characteristics and backgrounds, age, sexual orientation, gender reassignment, religion or belief, or marital and parental status. As users of the Disability Confident scheme, we interview all disabled applicants who meet the minimum criteria for the vacancy.