Data Engineer

Job Requirements

About the Role

The Data Engineer is responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support analytics and business intelligence initiatives. This role involves working with large volumes of data, collaborating with stakeholders to identify impactful solutions, and ensuring high-quality, error-free deliverables. The ideal candidate will have hands-on experience with big data technologies, cloud platforms, and data lake, lake house architectures.

Key Responsibilities

Primary Responsibilities

• Minimum 2 to 3 years of hands-on experience in data engineering.
• Proven expertise in SQL, Spark, and the Hadoop ecosystem.
• Experience handling multiple terabytes of data from ingestion to consumption.
• Collaborate with business stakeholders to identify and document high-impact business problems and potential solutions.
• Strong understanding of Data Lake/Lakehouse architecture and exposure to Hadoop (Cloudera, Hortonworks) and/or AWS.
• Manage the full data lifecycle: ingestion, transformation, and consumption.
• Proficient in Spark, Spark Streaming, Hive, and SQL.
• Experience with big data infrastructure including MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc.
• Deep knowledge of relational databases and data modeling.
• Technical debugging skills and experience with Git for version control.
• Create and maintain technical design documentation for data pipelines and projects.

Secondary Responsibilities

• Ability to work independently and manage personal development efforts.
• Strong oral and written communication skills.
• Learn and apply internal analytics technologies.
• Identify key performance indicators and develop strategies to achieve them.
• Use educational background in data engineering to perform data mining and analysis.
• Collaborate with BI analysts/engineers to build prototypes.
• Participate in the delivery and presentation of data solutions.

What We Are Looking For

Education

• Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related field.

Experience

• 2 to 5 years of relevant experience in data engineering.

Skills and Attributes

• Strong coding and debugging skills.
• Proficiency in big data tools and cloud platforms.
• Ability to work with large datasets and complex data architectures.
• Effective communication and stakeholder engagement.
• Self-driven with a continuous learning mindset.
• Strong documentation and organizational skills.

Key Success Metrics

• Timely and high-quality delivery of data engineering projects.
• Excellent code quality and maintainability.
• Identification and resolution of data issues.
• Effective technical solutioning and error-free execution.