Data Engineer

Summary/Objective

Reporting to the IT Manager, Data & Analytics, the Data Engineer is responsible for designing, building, and maintaining scalable data pipelines, data platforms, and data warehouse/data lakehouse architectures that support enterprise analytics, reporting, automation, and AI-enabled solutions. This role ensures that data is accurate, reliable, well-governed, secure, and optimized for consumption across Microsoft Fabric, Power BI, generative AI, AI agents, and future technologies. The Data Engineer will enable trusted, AI-ready data foundations and support client- and vendor-facing technology solutions that are critical to business operations, growth, and service delivery.

Essential Functions

•

Design, build, and maintain robust ETL/ELT data pipelines to ingest, process, and integrate data from multiple source systems (e.g., CRM platforms, APIs, and external data sources) into Azure and Microsoft Fabric.

•

Develop and manage the enterprise data platform, including lakehouse architecture (bronze, silver, gold layers), ensuring data is structured, scalable, and optimized for analytics.

•

Ensure the quality, integrity, and consistency of data through validation processes, monitoring, and proactive issue resolution.

•

Optimize performance and scalability of data workflows, pipelines, and storage to support efficient data processing and reporting.

•

Implement and enforce data governance, security, and compliance standards, including data lineage, access controls, and regulatory requirements.

•

Enable AI-ready and agentic enterprise capabilities by preparing governed datasets, metadata, lineage, and integration patterns that allow AI agents, automation, and decision-support tools to operate safely and effectively.

•

Support the design, delivery, and ongoing operation of client- and vendor-facing data integrations, reporting assets, APIs, and platform services that require reliable, secure, and scalable data engineering practices.

Responsibilities & Duties

•

Build and maintain data pipelines using Azure Data Factory, Fabric Data Pipelines, and other integration tools.

•

Develop data transformation logic using SQL, PySpark, or similar technologies to standardize and prepare data for analytics use.

•

Design and manage lakehouse structures in Microsoft Fabric, including bronze, silver, and gold data layers.

•

Collaborate with BI developers to ensure data is modeled and structured appropriately for Power BI semantic models and reporting.

•

Monitor data pipelines and platform performance, troubleshooting failures and optimizing workloads for reliability and efficiency.

•

Implement data validation checks and reconciliation processes to ensure data accuracy and completeness.

•

Manage data storage strategies, including partitioning, indexing, and lifecycle management for performance and cost optimization.

•

Ensure secure data handling practices, including role-based access controls, encryption, and compliance with organizational and regulatory requirements (e.g., PCI considerations).

•

Maintain documentation of data pipelines, data models, and system architecture to support transparency, governance, and knowledge sharing.

•

Partner with business stakeholders and IT teams to understand data requirements and deliver scalable data solutions.

•

Support data governance initiatives, including metadata management, lineage tracking, and data classification (e.g., integration with Purview).

•

Prepare and expose curated, governed, and well-documented data assets for AI, generative AI, semantic search, retrieval-augmented generation, and agent-based enterprise workflows.

•

Partner with business, IT, and analytics stakeholders to identify where data engineering can enable agentic workflows, automation, and AI-assisted decision-making while maintaining appropriate human oversight and control.

•

Design, build, and support data integrations, APIs, secure data exchanges, and reporting components used in client- and vendor-facing technology solutions.

•

Establish monitoring, observability, alerting, and support practices for AI-enabled and externally facing data solutions to ensure reliability, traceability, performance, and timely issue resolution.

•

Contribute to continuous improvement of the data platform, recommending new tools, practices, and optimizations aligned with industry best practices.

This job description is not designed to cover or contain a comprehensive listing of essential functions or responsibilities that are required of the employee for this job. Essential functions and responsibilities and activities may change at any time with or without notice.