AI focused DataBricks Developer (Digital Engineering)

Company Background

Our client is a leading life insurance company, part of a global financial services group. The company offers a wide range of products, including annuities and life insurance, delivered through a large nationwide network of agents.

Project Description

The project focuses on a production system managing clients and life insurance programs. The role involves both support and new feature development, with a strong emphasis on building and optimizing data pipelines and working within a modern Azure-based data platform.

Technologies

• Azure
• Databricks (Architecture, Unity Catalog, Delta, Lineage, Permissions, Databricks Genie, Notebooks)
• Agentic AI (Mosaic AI)
• Python
• SQL

What You'll Do

• Develop and maintain data pipelines within the Azure-based data platform;
• Work with the Databricks ecosystem (Delta Tables, Unity Catalog, notebooks) to process and manage data;
• Build and optimize data transformations using Python and SQL;
• Develop and support AI-driven data pipelines and automation using Databricks (Mosaic AI or similar tools);
• Support and enhance existing production data systems;
• Collaborate with a team of engineers on Databricks-based solutions;
• Ensure data quality, consistency, and reliability across data pipelines;

Job Requirements

• 3+ years of experience as a Data Engineer;
• Strong experience with Databricks ecosystem (Architecture, Unity Catalog, Delta tables, lineage, permissions, Databricks Genie, notebooks);
• Strong experience with Python;
• Solid proficiency in SQL;
• Experience working in Azure cloud environment;
• Experience with Databricks pipelines AI automation (Mosaic AI or AWS Agentcore);

Nice To Have

• Experience with Azure data integration tools (Synapse Pipelines, Azure Data Factory);
• Experience with Azure SQL pools;
• Understanding of performance tuning for data workloads;
• Experience with CI/CD, DevOps practices, Infrastructure as Code (IaC);

What Do We Offer

The global benefits package includes:

• Technical and non-technical training for professional and personal growth;
• Internal conferences and meetups to learn from industry experts;
• Support and mentorship from an experienced employee to help you professional grow and development;
• Health insurance;
• English courses;
• Sports activities to promote a healthy lifestyle;
• Flexible work options, including remote and hybrid opportunities;
• Referral program for bringing in new talent;
• Work anniversary program and additional vacation days.