Big Data Engineer

Description

Summary: Join the Data Engineering team to design, develop, and operate large-scale data systems, focusing on real-time pipelines and big data infrastructure. Highlights: 1. Design, develop, and operate large-scale data systems 2. Focus on real-time data pipelines and streaming analytics 3. Interact with engineers, product managers, BI developers, and architects As part of the Data Engineering team, you will be responsible for design, development and operations of large\-scale data systems operating at petabytes scale. You will be focusing on real\-time data pipelines, streaming analytics, distributed big data and machine learning infrastructure. You will interact with the engineers, product managers, BI developers and architects to provide scalable robust technical solutions. **Responsibilities and Duties** * Min 6\-8 years of BIG data development experience. * Demonstrates up\-to\-date expertise in Data Engineering, complex data pipeline development. + Experience in agile models + Design, develop, implement and tune large\-scale distributed systems and pipelines that process large volume of data; focusing on scalability, low \-latency, and fault\-tolerance in every system built. * Experience with Java, Python to write data pipelines and data processing layers * Experience in Airflow \& Github. * English conversational * Experience in writing map\-reduce jobs. * Demonstrates expertise in writing complex, highly\-optimized queries across large data sets * Proven, working expertise with Big Data Technologies Hadoop, Hive, Kafka, Presto, Spark, HBase. * Highly Proficient in SQL. * Experience with Cloud Technologies ( GCP, Azure) + Experience with relational model, memory data stores desirable ( Oracle, Cassandra, Druid) * Provides and supports the implementation and operations of the data pipelines and analytical solutions * Performance tuning experience of systems working with large data sets * Experience in REST API data service – Data Consumption + Retail experience is a huge plus. **Requirements and Qualifications** As part of the Data Engineering team, you will be responsible for design, development and operations of large\-scale data systems operating at petabytes scale. You will be focusing on real\-time data pipelines, streaming analytics, distributed big data and machine learning infrastructure. You will interact with the engineers, product managers, BI developers and architects to provide scalable robust technical solutions. Are you looking for a place where your ideas shine? With over 38 years of experience and a global presence, at Stefanini we transform tomorrow together. Here, every action counts and every idea can make a difference. Join a team that values innovation, respect, and commitment. If you are a disruptive individual, committed to continuous learning, and innovation is in your DNA, then we are what you are looking for. Come and build a better future with us!