Lead Data Engineer (Maritime Industry)

Send CV


Data Engineering





Opinov8 is a technology service provider, with an Opinion in Innovation, we engage with our clients at any stage in their product engineering and innovation goals, with solutions being mainly collaborative and outcome-driven. We are driven by delivering value, for our clients, through technological innovation.
We are a young energetic and dynamically agile company and the team behind Opinov8 are software industry veterans. We are ambitious on our own and on our client’s behalf and we do not look at a client engagement as a project, we are in it to help our clients build sustainable products, platforms, and businesses.
Our core values and what we look for in our Opinov8rs:
You are Collaborative - which is a key component to being successful in distributed environments.
You are Innovative - it is a requirement that we innovate, as individuals and as a company.
Be Adventurous - courageous people are adventurous, we are hiring and nurturing this quality.
Always Respectful - remember if you don’t respect others they will not respect you.
Inherently Intelligent - it goes without saying we are all intelligent in our own way.
Ultimately Responsible - being responsible is a core personality trait that is needed for us and our clients’ success and ultimately a happy work environment.
The client owns an integrated data platform designed to help charterers, ship owners and operators make smarter decisions based on intelligent insights derived from reliable data. The platform brings together all owned and third-party data into a simple to use ecosystem powered by machine-learning that reduces fuel costs, improves vessel performance and helps cut and monitor greenhouse gas emissions.  
The team is committed to re-architecture toward cloud-based data processing solution into a generic data platform along with data pipelines and frontend optimization, which was focused on the determined business and architectural drivers, unification of cloud platform to Azure, and development of shared services. 
The team uses data engineering, data science, big data and full-stack engineering using technologies such as Python (Pandas/PySpark), Azure (mostly Cosmos DB, Airflow, Synapse, Data Factory), Databricks, Spark and Spark Streaming, public APIs, Docker, Terraform, C#/.NET, React.
- Play a key role in architectural decisions as we build toward an efficient innovative analytical solutions using Big Data related technologies;
- Develop and launch new features to adapt to evolving business needs;
- Build and modify Spark jobs (in Python & Pandas/PySpark) to perform various tasks;
- Deploy a range of cloud-based technologies for critical projects;
- Write, test, instrument, and deploy code to our Docker + Terraform environment;
- Help drive the innovation and evolution of the company products;
- Communicate with key technical stakeholders and third parties to understand requirements and help drive decisions, and gather requirements when underspecified;
- Build partnerships with Architects and key individuals within other functional groups;
- Actively participate in code review and test solutions to ensure it meets best practice specifications. Maintain high standards of code quality, and encourage the same by providing constructive code reviews to collaborators;
- Build a high-performance engineering culture, mentor team members of senior and middle+ level and provide them with the tools and motivation; 
- Troubleshoot and resolve issues, problems, and errors encountered across various systems; 
- Collaborate with Data Science, Product, Research, and Engineering teams to iterate on the roadmap; 
- Your participation and contributing to Opinov8 Data Community and Competence Center is a plus.
- 7+ years of experience in software development, including at least 3 years of experience with writing in Python in containerized production environment;
- Ability to write high-performance, well-designed, well-crafted, readable, and tested code in a team environment;
- Good understanding of distributed computing principles, CI/CD principles and best practices; 
- Solid experience with Spark & Spark Streaming (using Python), Databricks, HDFS and Azure (Cosmos DB), that includes streaming processing to joining and aggregating huge data sets, to integrating with third party data sources;
- Experience with debugging and optimizing Cloud-based data pipeline is must;
- Strong knowledge of SQL required;
- Solid understanding of both non-relational and relational database systems (Cosmos DB, DynamoDB, MongoDB, Cassandra) and familiarity with database fundamentals, such as ACID, database schema, normalized/denormalized data;
- Familiarity with the following Infrastructure technologies – Docker/Kubernetes/Yarn, Azure DevOps, Terraform; 
- Computer Science degree or relevant experience; 
- English: Upper-Intermediate (B2) and higher.
Preferred Qualifications 
- Ability to build in multiple languages and the ability to pick up new languages quickly;
- Experience with maritime shipping business domain is a plus; 
- Experience with real-time analytics (MS Power BI) is a plus; 
- Compliance awareness, such as GDPR, SOX, HIPAA is a plus;
- Relevant certifications as Security Specialist are considered an advantage;
- Ability to work in a fast-paced (startup like) Agile development environment is a plus.


Join our global team of innovative technologies and creative thinkers
Send CV

Related Job Opening

Let us innov8 with you

The technology innovation partner with a focus on human to human digital solutions and products
UK, London
Office 9, Weyhouse, Church Street, Weybridge, KT13 8NA
Ukraine, Kyiv
58/10, Simi Prakhovykh Street
Egypt, Cairo
11G/4, Ahmed Kamal Street, 
New Maadi
Prepare for a quick response:
© Opinov8 2021. All rights reserved.       Privacy Policy