Data Lineage

Description

Job Summary: We are seeking a Data Engineer with experience in Data Lineage to develop and maintain robust data pipelines, implement ETL processes, and ensure data integrity. Key Highlights: 1. Design and maintenance of robust and efficient data pipeline infrastructure 2. Implementation of ETL processes to integrate data from diverse sources 3. Technical leadership, mentoring, and definition of best practices IDS Comercial is a 100% Mexican IT company with 43 years of experience and presence across Mexico, Latin America, and the United States. Over 1,500 consultants work on numerous projects spanning the financial, commercial, manufacturing, telecommunications, services, government, and other sectors. We are accredited at CMMI Level 5 (Capability Maturity Model Integration), the quality model established by the Software Engineering Institute (SEI). We are ISO certified under ISO/IEC 27001:2013. We hold the Socially Responsible Enterprise (ESR) distinction. WE ARE LOOKING FOR YOU! · Data Lineage Key Responsibilities: Provide specialized services through a development cell delivering the following: 1. Data Flow Diagramming:      a. Documentation of services, applications, and interfaces.      b. Identification of data structures.      c. Identification of elements or fields within these data structures.      d. Validation and approval by process owners. 2. Information Mapping and Lineage:      a. Identification of original data sources.      b. Identification of lineage from source table field to destination table field, including transformation logic.      c. Validation and approval by process owners. 3. Data Requirements Document:      a. Explanation of the transformation process from source to destination.      b. Validation and approval by process owners Requirements: Technical Knowledge: Architecture Design: Create and maintain robust and efficient data pipelines. Data Management: Implement ETL (Extract, Transform, Load) processes to integrate data from diverse sources. Technical Leadership: Guide and mentor junior engineers, and define coding and development best practices. Optimization and Quality: Improve query performance and ensure data integrity, security, and availability. Essential Qualifications: Experience: Proven extensive track record in data engineering or similar roles. Technical Skills (Hard Skills): Proficiency in programming languages (Python, Scala, Java, SQL), Big Data tools (Spark, Hadoop), and cloud platforms (AWS, Google Cloud, Azure). Data Modeling: Expertise in designing relational and non-relational (NoSQL) databases. Work Modality: Hybrid (3 days onsite, 2 days remote). Location: CDMX. Project Duration: 6 Months · We Offer: Base monthly salary (100% payroll-based scheme) Statutory benefits Attendance bonus (10% of base monthly salary) 5% housing fund (percentage of payroll) 2% retirement fund (percentage of payroll) Major Medical Expense Insurance Minor Medical Expense Insurance Accident Life Insurance Savings Fund (Optional) Fleet Vehicle Insurance (Optional) Commercial Agreements · We are an inclusive company free from any form of discrimination, where opportunities are based solely on attitude and talent. We do not require medical certificates confirming non-pregnancy or HIV status as prerequisites for joining our large workforce family or accessing equal growth opportunities.-Requirements- Minimum Education: Higher Education – Bachelor’s Degree years of experience Keywords: data, data