Data Engineer

Description

Job Summary: We are seeking a Senior Consultant, Data Engineer, to design, develop, and operate large-scale data systems, with a focus on real-time data management, big data, and ML. Key Highlights: 1. Design, develop, and optimize large-scale distributed systems with data. 2. Collaborate with engineers and architects to deliver robust technical solutions. 3. Focus on real-time data management, streaming, and distributed big data. INFORMATION SENIOR CONSULTANT 5 hours ago Description As part of the Data Engineering team, you will be responsible for designing, developing, and operating large-scale data systems operating at petabyte scale. You will focus on real-time data management, streaming analytics, distributed big data, and machine learning infrastructure. You will collaborate with engineers, product managers, business intelligence developers, and architects to deliver robust and scalable technical solutions. Requirements * Minimum 6–8 years of experience in big data development. * Current hands-on experience in data engineering and development of complex data pipelines. * Experience with agile methodologies. * Design, develop, implement, and optimize large-scale distributed systems and pipelines processing high-volume data, prioritizing scalability, low latency, and fault tolerance in each system. * Proficiency in Java and Python for building data pipelines and processing layers. * Experience with Airflow and GitHub. * Conversational English. * Experience building map\-reduce jobs. * Proven ability to write complex, highly optimized queries over large datasets. * Demonstrated experience with big data technologies such as Hadoop, Hive, Kafka, Presto, Spark, and HBase. * Advanced SQL proficiency. * Experience with GCP. * Experience with relational models and in-memory data warehouses (e.g., Oracle, Cassandra, Druid) is a plus. * Provide and support implementation and operation of data pipelines and analytical solutions. * Experience optimizing performance of systems handling large datasets. * Experience with RESTful API data services: data consumption. * Retail industry experience is a plus. Profile **SENIOR CONSULTANT** Location **Huixquilucan, State of Mexico, Mexico (Hybrid)**Experience **6 Years of Experience** o