Sr. Data Pipeline Engineer

• Design, develop, and optimize end-to-end data pipelines for efficient data extraction, transformation, and loading (ETL) processes.
• Collaborate with cross-functional teams to understand data requirements and translate them into scalable pipeline solutions.
• Implement best practices for data integration, ensuring high performance, reliability, and scalability. Data Transformation and Quality
• Transform raw data into usable formats, ensuring data quality, consistency, and accuracy.
• Handle data validation, cleansing, and error handling to maintain data integrity.
• Monitor and proactively maintain data pipelines to ensure high service availability. Performance Optimization
• Continuously improve pipeline performance by identifying bottlenecks and implementing optimizations.
• Work with cloud-based technologies (e.g., AWS, GCP, Azure) to enhance scalability and efficiency. Collaboration and Leadership
• Partner with Data Scientists, Analysts, and other stakeholders to understand their data needs.
• Lead discussions on system enhancements, process improvements, and data governance.
• Mentor junior engineers and contribute to the growth of the data engineering team. Requirements
• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 5+ years of experience in data engineering, with a focus on building and maintaining data pipelines.
• 1-2 years experience with building NiFI data flows or similar for Kafka and Hadoop-based No

SQL databases.

• Proficiency in ETL tools, SQL, and scripting languages (Python, Scala, etc.).
• Experience with Data Catalog and Accumulo indexes for information retrieval and discovery.
• Experience with API-led design.
• Experience with ELK stack a plus.
• Experience with cloud-based data platforms (e.g., AWS S3, Redshift, Google Big Query).
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration abilities.
• Security+ Certification
• Active US Government Clearance at Secret level or higher
• Effective written and verbal communications skills for collaboration with both customers and fellow team members.
• Ability to sit for extended periods of time.
• Ability to regularly lift at least 25 pounds.
• Ability to commute to the designated onsite work location as required. Seniority level

Mid-Senior level Employment type

Full-time Job function

Information Technology Industries

IT Services and IT Consulting #J-18808-Ljbffr