Senior Data Engineer
Transportation drives humanity forward. At Stratio we have a purpose: to change the transportation industry. We believe in a future with no disruptions, where vehicles never break down, a zero downtime future. For that we rely on great individuals and great teams.
As a Senior Data Engineer, your main role will be providing technical guidance, promoting data engineering initiatives and building end-to-end analytical solutions. You will work alongside our agile development teams using the latest technologies to deliver solutions in fast-paced and complex environments, ensuring performance, scalability and reliability in our products. You will be responsible for bringing the data to the models and the predictions to our customers.
- Lead the development of the data pipeline environment for our Machine Learning researchers;
- Have a coaching role to other team members in data topics;
- Scale machine learning pipelines, from the base implementation, into a fully productive machine learning model;
- Design solutions and develop machine learning-based applications, services, and APIs to support the machine learning model integration with the Stratio architecture;
- Focus on performance, scalability, resilience, availability and fault tolerance standards;
- Contribute to develop and maintain a highly scalable system that provides advanced insights and predictive capabilities to a real product;
- Be part of a technological evolution in the adoption of new tools, frameworks and processes to automate, scale and improve our end-to-end quality;
- Analyze and identify the right tools and frameworks to deliver an end-to-end data architecture;
- Be committed to Continuous Integration and Continuous Deployment.
- BS/MS degree in Computer Science, with minimum 4 years of work experience;
- Experience developing and maintaining machine learning models in a real production environment;
- Experience creating and deploying large-scale, data-driven systems in a production environment;
- Experience building and integrating ETL pipelines, Data Lakes and/or Analytics tools;
- Knowledge in distributed processing using Apache Spark;
- Strong programming knowledge in Python;
- Experience with SOLID Principles and Design Patterns of SW development;
- Experience in using databases such as MS SQL, Postgres, Elasticsearch, Cassandra or other noSQL databases;
- Experience using message brokers such as Apache Kafka;
- Experience with containerization and orchestration technologies such Docker and Kubernetes;
- Experience in Continuous Integration and Continuous Deployment processes;
- Comfortable with Agile methodologies, such as Scrum and Kanban;
- Creative problem-solving and critical thinking skills, especially around breaking down complex structures into modular solutions;
- Proficiency in English.
- Familiarity with some popular machine learning frameworks, libraries, and platforms like Airflow, MLflow, Kubeflow, Spark ML, Pandas, PyTorch, TensorFlow, Scikit-learn among others;
- Experience with .NET Core;
- Experience with static analysis tooling (e.g.: SonarQube, OWASP Dependency Track, etc.);
- Experience with automation tools like Jenkins, Ansible.
We expect you:
- Be able to work with minimal supervision;
- Help the elements of the team with less experience;
- Be eager to learn new tools for new problems;
- Optimize data delivery and re-design infrastructure for greater scalability;
- Create and maintain optimal data pipeline architecture;
- Keep an eye for possible improvements and identify disruptions in the processes.
What we offer:
- Health Insurance;
- Fringe Policy;
- Flexible Work Hours - adjust your schedule to your needs;
- Work Setup - remote, hybrid, onsite - if your job can be done remotely, and you prefer to, you’re free to choose;
- Hardware and software for a full remote setup;
- Monthly All-Hands;
- Quarterly Events to discuss Strategy;
- Autonomy and Ownership Culture;
- Continuous feedback culture;
- Innovation Mindset;
- Career Acceleration.
- Coimbra / Lisbon / Remote
We want inspiring individuals in our teams, where age, race, gender, sexual orientation, politics and religion do not matter, and seek to create a tolerant and open space for everyone. We thrive to provide an inclusive and trustworthy environment.
You can find our Culture Manifesto and more team information here.
Take the road with us!