DevOps Engineer with Data Engineering Experience

About Scispot

Scispot is building the world's first scientific ETL for unstructured, semi-structured and structured data. Scispot staging lakehouse connects with all the popular R&D sources such as Benchling, Quartzy, Lab Instruments, and others & converts them into machine-readable data to support downstream pipelines.

The Opportunity

We're searching for a seasoned DevOps Engineer with experience in Data Engineering. The right candidate will have an opportunity to become a senior architect as we are moving really fast and scaling our Infrastructure. In this role, you'll be instrumental in ensuring the scalability and stability of our infrastructure as we continue to grow. This position involves a balance of maintaining robust AWS infrastructure, implementing DevOps practices, and handling ETL tasks.

What You'll Do
 Design, manage, and optimize a scalable, secure AWS infrastructure focusing on EC2, ECS, S3, and Lambda services.
 Implement strategies and solutions for scaling and. stabilizing infrastructure.
 Maintain high-availability systems, targeting 99.99% uptime.
 Drive the adoption of Infrastructure as Code (IAC) practices.
 Work with pub-sub architecture and Rabbit MQ.
 Develop and maintain ETL processes
 Contribute to AI infrastructure, including ML pipelines using Next Flow.
 Collaborate with cross-functional teams to translate their needs into technical requirements.

What You'll Need
 Proven experience as a DevOps Engineer focusing on infrastructure management and scalability.
 Experience with Bioinformatics pipelines
 Extensive experience with AWS infrastructure including services like EC2, ECS, API Gateway, S3, and Lambda.
 Strong familiarity with maintaining high-availability systems.
 Proficiency in Infrastructure as Code (IAC) practices.
 Experience in a Data Engineering or similar role involving ETL processes.
 Experience with Rabbit MQ and pub-sub architectures.
 Familiarity with ETL tools.
 Experience with Elastic Search
 Familiarity with Java and Spring Boot is a plus,