Senior Big Data Engineer

*]:pointer-events-auto scroll-mt-[calc(var(--header-height)+min(200px,max(70px,20svh)))]" dir="auto" data-turn-id="request-WEB:5b06deab-eccc-4d1f-99f4-84cb38185bda-1" data-testid="conversation-turn-4" data-scroll-anchor="true" data-turn="assistant">Key Responsibilities

• Design and build scalable data lakes and data pipelines on AWS using cloud-native and automated approaches
• Enable federated and high-performance analytics using Amazon Athena and Trino, including query optimization at scale
• Manage metadata, schemas, and data discovery using AWS Glue Data Catalog
• Implement secure data governance using AWS Lake Formation, KMS encryption, and SSL
• Build, deploy, and operate data services on Kubernetes (Amazon EKS)
• Work with Hadoop ecosystem components (Spark, Hive, HDFS) including optimization techniques such as partitioning, bucketing, and columnar formats (Parquet, ORC)
• Troubleshoot and resolve complex issues across big data pipelines, clusters, and query engines
• Design and maintain CI/CD pipelines using Jenkins or similar tools
• Implement monitoring and observability using CloudWatch and Grafana
• Prepare curated, high-quality datasets for AI/ML use cases
• Build and configure MCP server for AI/ML integration
• Collaborate within Agile/Scrum teams and proactively identify architectural and performance gaps
• Propose scalable, innovative, and out-of-the-box engineering solutions Required Qualifications
• 8+ years of experience in Big Data / Data Engineering
• Strong hands-on experience with AWS services:• S3, Glue Data Catalog, Lake Formation, Athena, EMR, EKS
• Strong experience with Trino (or Presto) and query optimization techniques
• Hands-on experience with Kubernetes (EKS) for data workloads
• Strong proficiency in SQL, Python, and shell scripting
• Experience building CI/CD pipelines using Jenkins or similar tools
• Experience building and configuring MCP servers for AI/ML integration
• Strong ownership mindset with problem-solving ability (not just task execution) Preferred Qualifications
• Exposure to AI/ML data pipelines and workflows
• Experience in cloud-native data modernization programs
• Strong understanding of scalable data architecture in enterprise environments