Senior Consultant

Job Overview

We are seeking a skilled Data Engineer (6-8 years of experience) to join our team and build a FinOps Data Platform. You will design and optimize end-to-end pipelines for processing large volumes of cloud cost data to produce chargeback datasets.

Work Schedule: 12:00 PM - 8:00 PM Poland Time (5:00 AM - 2:00 PM EST).

Key Responsibilities

• Data Pipelines & Orchestration: Build, optimize, and monitor scalable ETL/ELT workflows using tools like Azure Data Factory, Synapse, AWS S3, or BigQuery. Manage scheduling, retries, and data validation.
• Architecture & Modeling: Design robust data structures, schemas, and cloud storage solutions (Data Lakes, Warehouses) optimized for performance and scalability.
• Integration & Infrastructure: Build integrations with internal/external APIs. Configure, secure, and troubleshoot distributed data infrastructure.
• Collaboration: Document technical workflows and collaborate with stakeholders to deliver tailored data solutions.

Technical Requirements

• Programming & SQL: Advanced Python and SQL (window functions, CTEs, tuning). Java/Scala or Bash is a plus.
• Databases & Warehousing: Practical knowledge of OLAP (Snowflake, BigQuery, Redshift) and relational databases (PostgreSQL, MySQL).
• Big Data & Cloud: Hands-on experience with cloud platforms (AWS, Azure, or GCP) and distributed processing frameworks (Spark/PySpark, Kafka, Hadoop).
• Software Engineering: Version control (Git), CI/CD, and Infrastructure as Code (Terraform) awareness.
• Data Quality & Governance: Experience with data validation, lineage, security controls, and RBAC.

Qualifications & Nice-to-Have

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Strong problem-solving skills and sharp attention to detail.
• Experience with the Cloudability Data Platform and FinOps/cost optimization awareness is a major plus.