Senior Software Engineer Big Data

We are seeking a skilled, experienced and self-motivated Software Engineer to design, develop, and optimize large-scale, cloud-based data processing and workflow execution systems, in a dynamic environment, to solve complex scenarios and drive customer value.

Responsibilities

• Work closely with cross-functional teams, using the agile methodology, to understand business needs and construct durable technical solutions.
• Design, develop, and maintain large-scale data processing and orchestration systems using cloud and Big Data technologies (e.g., AWS, EMR, EKS, Python, Java, SQL, Apache Airflow).
• Continually optimize and enhance technology components and systems for best performance, scalability, and reliability.
• Build and maintain automated unit, integration, and end-to-end tests (e.g. Pytest) for consistent product quality.
• Engineer, implement, and maintain CI/CD pipelines.
• Monitor and troubleshoot sophisticated scenarios in production environments to identify and resolve issues.
• Stay current with industry trends and emerging cloud and Big Data technologies to continuously improve the systems architecture.
• Build, configure, and operate an MCP server for AI/ML integration.
• Leverage AI technologies, such as Kiro, throughout the SDLC.

Required Skills

Bachelor's degree in Computer Science, Information Systems or related discipline with at least five (5) years of experience, or equivalent training and/or work experience.

• Proficiency in Python (or Java), experience with object-oriented and functional programming concepts (e.g., design patterns, immutability, higher-order functions).
• Working SQL knowledge, including, but not limited to, window functions, multi-table joins, aggregations, CTEs and analytical functions.
• Hands-on experience with cloud services such as AWS, S3, ECS, EKS, EMR and RDS.
• Ability to write clean, modular, extensible and performant code with unit and integration test cases.
• Familiarity with workflow orchestration tools (e.g. Apache Airflow)
• Experience with automated test construction and testing frameworks such as Pytest.
• Agile methodology and peer reviews.
• Familiarity with RESTful API development.
• Strong problem-solving and analytical skills.
• Technical documentation composition.

Preferred Skills

• UX technologies: JavaScript, Angular/React frameworks.
• Experience with CI/CD pipelines (Jenkins, GitLab CI, GitHub Actions, ArgoCD).
• Experience with Docker/ECS, EKS and container image optimization.
• Experience with Infrastructure as Code (Terraform, CloudFormation).
• Experience with monitoring and observability tools (Prometheus, Grafana, ELK stack).
• AWS certifications (AI practitioner, Solutions Architect, Big Data Specialty, or Kubernetes certifications like CKA/CKAD).
• Experience with AWS monitoring and logging tools (CloudWatch, CloudTrail) for Kubernetes workloads.
• Financial industry Experience. •