Data engineer

Description

Job Summary: Axity is seeking a Data Engineer to design, develop, and operate large-scale data systems, driving digital transformation and enabling petabyte-scale data management. Key Highlights: 1. Be part of the digital transformation with petabyte-scale data. 2. Work on the design, development, and operation of large-scale data systems. 3. Collaborate on innovative Big Data and Machine Learning projects. DESCRIPTION **Data Engineer at Axity** ========================== Join our **Axity** team as a **Data Engineer**, and become a key contributor in designing, developing, and operating large-scale data systems! At Axity, we are at the forefront of digital transformation and seek exceptional talent to enhance our capabilities in petabyte-scale data handling. If you're passionate about data engineering, real-time pipelines, streaming analytics, and machine learning infrastructure, this is your place! **Your Role and Responsibilities** ------------------------------ As a Data Engineer, you will be the architect and builder of our data solutions. Your key responsibilities will include: * **Design and Development:** Build and maintain robust, scalable batch and real-time data pipelines. * **Systems Operations:** Ensure reliability and efficiency of our large-scale data systems. * **Analytics Innovation:** Implement and optimize solutions for streaming analytics and distributed big data processing. * **ML Infrastructure:** Collaborate on development and maintenance of infrastructure required for machine learning solutions. * **Cross-functional Collaboration:** Work closely with engineers, product managers, BI developers, and architects to deliver cutting-edge technical solutions. * **Problem Solving:** Tackle complex technical challenges and propose innovative, efficient solutions. **What We’re Looking For? Key Requirements** ----------------------------------- To succeed in this role, you’ll need a solid foundation and hands-on experience in the following areas: * **Experience:** Minimum 6\-8 years of proven experience in **BIG DATA** development. * **Agile Methodologies:** Experience working under agile frameworks (Scrum, Kanban, etc.). * **Language:** Conversational-level **English** proficiency to interact with global teams and access technical documentation. * **Retail Sector:** Experience in the **Retail** sector is considered a significant advantage. Your domain knowledge in this area will be highly valuable. **Essential Technical Skills** --------------------------------------- We seek deep expertise in the following technologies and tools: * **Data Engineering:** Solid experience in designing and developing complex data pipelines. * **Programming Languages:** **Python** and **Java** for building pipelines and data processing layers. * **Orchestration:** Experience with orchestration tools such as **Airflow**. * **Version Control:** Expert-level proficiency with **Github** for code management and collaboration. * **Distributed Processing:** Experience writing map\-reduce jobs and using frameworks like Spark. * **Big Data Technologies:** Proven experience with the **BIG DATA** ecosystem, including but not limited to: Hadoop, Hive, Kafka, Presto, Spark, HBase. * **Databases:** High proficiency in **SQL** for querying and manipulating data. * **Cloud:** Experience with cloud platforms such as **GCP** (Google Cloud Platform). **Why Axity?** ------------------- At Axity, we offer a dynamic, collaborative, and innovative work environment. We foster professional growth, continuous learning, and the opportunity to work on challenging projects that make a difference. You’ll have the chance to: * Work with cutting-edge data technologies. * Be part of a highly skilled and passionate team of professionals. * Contribute to the success of large-scale projects for leading clients. * Access development and training programs. * Enjoy an inclusive and diverse workplace. **Apply Today!** ---------------- If you meet the requirements and are ready to take your career to the next level in the Big Data world, we want to meet you! Apply now and explore the opportunities Axity has for you. **Axity** \- Empowering the future through data. \#DataEngineer \#BigData \#Python \#SQL \#GCP \#English \#Retail \#Airflow \#Github \#Jobs \#Technology \#DataEngineering REQUIREMENTS **Requirements** ================== * Minimum 6\-8 years of experience in BIG Data development. * Experience with agile methodologies. * Conversational English. * Retail experience is a major plus. **Essential Technical Skills** ======================================= * Experience in Data Engineering and development of complex data pipelines. * Experience in Java and Python for writing pipelines and data processing layers. * Experience with Airflow and Github. * Experience writing map\-reduce jobs. * Proven experience with Big Data technologies: Hadoop, Hive, Kafka, Presto, Spark, HBase. * High proficiency in SQL. * Experience with cloud technologies (GCP). * Experience writing complex, highly optimized queries over large volumes of data. * Experience in performance tuning of systems handling large volumes of data. * Experience with REST API data services – Data Consumption. **Desirable Technical Skills** ================================== * Experience with relational models and in-memory data stores (Oracle, Cassandra, Druid). **Key Responsibilities** ============================= * Design, develop, implement, and optimize large-scale distributed systems and pipelines that process large volumes of data, focusing on scalability, low latency, and fault tolerance. * Provide and support implementation and operation of data pipelines and analytical solutions. * Collaborate with engineers, product managers, BI developers, and architects to deliver scalable and robust technical solutions.