Data Engineer (Hybrid Set-up)

Data Engineer

Data Engineers are mainly tasked with transforming data into a format that can be easily analyzed. The Data Engineer will be responsible for designing and developing Sprouts data system, processing and extracting data features and deploying the data science teams machine learning models. The Data Engineer will support our software engineers, architects and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.

Responsibilities

Build, optimize and maintain conceptual, logical and physical database models

Develop database solutions to store and retrieve information

Assemble datasets that meet functional/non-functional business requirements

Build the infrastructure for optimal ETL from a wide variety of data sources

Monitor data integrity and adopt appropriate tools

Improve system performance

Optimize or re-design data architecture to support Sprouts next generation of products and data initiatives

Design, develop, test and deploy web service APIs

Work with Data Scientists to identify future needs and requirements

Deploy models and algorithms developed by the Data Science team

Requirements

Extensive knowledge on databases (SQL and/or NoSQL) and data engineering best practices

Expertise in SQL and other programming languages(e.g. Python, Java, Scala, shell scripting etc.)

Experience with data modeling (data warehouse, data lake) and designing data storage schemes

Familiarity with data engineering and ETL software tools, hadoop, spark, talend, SSAS, etc. is also helpful

Experience building and optimizing data pipelines, architecture and datasets

Experience in software development

Experience with Azure

A successful history of manipulating, processing and extracting value from large disconnected datasets

Familiarity with data visualization tools (e.g. PowerBI)

Familiarity with agile development as a project management methodology is a plus

Strong problem-solving and analytical skills

Must be self-motivated and comfortable supporting the data needs of multiple teams, systems and products

A good team player and willingness to learn

Strong innate desire and proven track record of continuous self-improvement (in learning, job expansion, extracurricular activities, etc.)