Senior Data Engineer

Description

Job Summary: We are seeking a Data Engineer with experience in architecture design, data management, technical leadership, and optimization to ensure data integrity and availability. Key Highlights: 1. Key role in designing and managing data architectures. 2. Opportunity to lead and mentor junior engineers. 3. Focus on data optimization and quality. IDS Comercial is a 100% Mexican IT company with 40 years of experience and presence in Mexico, Latin America, and the U.S. We are accredited at CMMI Level 5 (Capability Maturity Model Integration), a quality model established by the Software Engineering Institute (SEI). Certified by ISO under ISO/IEC 27001:2013\. We hold the Socially Responsible Enterprise (ESR) distinction. WE ARE LOOKING FOR YOU!! **Data Lineage** **Work Mode: Hybrid (3 days at client site, 2 days remote)** **Location: Av. Paseo de la Reforma 347, Cuauhtémoc, 06500 Ciudad de México, CDMX** **Project Duration: 6 Months** **Requirements:** **3+ years of experience as a Data Engineer.** Experience in: * Architecture Design: Building and maintaining robust, efficient data pipelines. * Data Management: Implementing ETL (Extract, Transform, Load) processes to integrate data from diverse sources. * Technical Leadership: Guiding and mentoring junior engineers, and defining coding and development best practices. * Optimization and Quality: Improving query performance and ensuring data integrity, security, and availability. **Mandatory Skills:** * Experience: Proven extensive track record in data engineering or similar roles. * Technical Skills (Hard Skills): Proficiency in programming languages (Python, Scala, Java, SQL), Big Data tools (Spark, Hadoop), and cloud platforms (AWS, Google Cloud, Azure). * Data Modeling: Skills in designing relational and non-relational (NoSQL) databases. * Soft Skills: Leadership, excellent communication, critical thinking, and ability to solve complex problems. **Responsibilities:** Deliver specialized development team services delivering the following: **1\. Data Flow Diagramming:** a. Documentation of services, applications, and interfaces. b. Identification of data structures. c. Identification of elements or fields within these data structures. d. Validation and approval by process owners. **2\. Information Mapping and Lineage:** a. Identification of original data sources. b. Identification of lineage from source table field to destination table field, including transformation steps. c. Validation and approval by process owners. **3\. Data Requirements Document:** a. Explanation of the transformation process from source to destination. b. Validation and approval by process owners