Senior Data Engineer

Description

Job Summary: You will be responsible for the design, development, and operations of large-scale data systems, focusing on real-time data management, streaming analytics, distributed big data, and ML infrastructure. Key Highlights: 1. Petabyte-scale data systems design and development 2. Focus on real-time data management and distributed big data 3. Collaboration with engineers, product managers, and architects DESCRIPTION As a member of the Data Engineering team, you will be responsible for the design, development, and operations of large-scale data systems operating at petabyte scale. You will focus on real-time data management, streaming analytics, distributed big data, and machine learning infrastructure. You will collaborate with engineers, product managers, business intelligence developers, and architects to deliver robust and scalable technical solutions. REQUIREMENTS * Minimum 6–8 years of experience in big data development. * Current hands-on experience in data engineering and development of complex data pipelines. * Experience with agile methodologies. * Design, develop, implement, and optimize large-scale distributed systems and pipelines that process massive volumes of data, prioritizing scalability, low latency, and fault tolerance in each system. * Proficiency in Java and Python for building data pipelines and processing layers. * Experience with Airflow and GitHub. * Conversational English. * Experience developing map\-reduce jobs. * Proven experience writing complex, highly optimized queries over large datasets. * Demonstrated experience with big data technologies such as Hadoop, Hive, Kafka, Presto, Spark, and HBase. * Advanced SQL proficiency. * Experience with cloud technologies (GCP). * Experience with relational models and in-memory data warehouses (e.g., Oracle, Cassandra, Druid) is a plus. * Provide and support implementation and operation of data pipelines and analytical solutions. * Experience optimizing performance of systems handling large datasets. * Experience with REST API data services: data consumption. * Retail industry experience is a plus.