Data Lineage

Description

Job Summary: We are seeking a Data Lineage expert to develop and maintain robust data pipelines, manage ETL processes, and ensure data integrity and availability. Key Responsibilities: 1. Develop and maintain robust and efficient data pipelines. 2. Implement ETL processes to integrate data from diverse sources. 3. Provide technical leadership, define best practices, and mentor junior engineers. IDS Comercial is a 100% Mexican IT company with 43 years of experience and presence in Mexico, Latin America, and the USA. Over 1,500 consultants work across numerous projects in sectors including financial services, retail, manufacturing, telecommunications, services, government, and others. We are accredited at CMMI Level 5 (Capability Maturity Model Integration), the quality model established by the Software Engineering Institute (SEI). We are ISO certified under ISO/IEC 27001:2013\. We hold the Socially Responsible Enterprise (ESR) distinction. WE ARE LOOKING FOR YOU! **Data Lineage** * Key Responsibilities: Deliver specialized development cell services delivering the following: 1\. Data Flow Diagramming: a. Document services, applications, and interfaces. b. Identify data structures. c. Identify elements or fields within these data structures. d. Validate and obtain approval from process owners. 2\. Information Mapping and Lineage: a. Identify original data sources. b. Trace lineage from source table field to destination table field, including transformation logic. c. Validate and obtain approval from process owners. 3\. Data Requirements Document: a. Explain the transformation process from source to destination. b. Validate and obtain approval from process owners. * Requirements: Technical Knowledge: Architecture Design: Design and maintain robust and efficient data pipelines. Data Management: Implement ETL (Extract, Transform, Load) processes to integrate data from diverse sources. Technical Leadership: Guide and mentor junior engineers, and define coding and development best practices. Optimization and Quality: Improve query performance and ensure data integrity, security, and availability. Mandatory Qualifications: Experience: Proven extensive track record in data engineering or similar roles. Technical Skills (Hard Skills): Proficiency in programming languages (Python, Scala, Java, SQL), Big Data tools (Spark, Hadoop), and cloud platforms (AWS, Google Cloud, Azure). Data Modeling: Skills in designing relational and non-relational (NoSQL) databases. Work Modality: Hybrid (3 days onsite, 2 days remote). Location: Mexico City (CDMX). Project Duration: 6 Months * We Offer: Monthly base salary (100% payroll-based scheme) Statutory benefits Attendance bonus (10% of monthly base salary) 5% housing fund (percentage of payroll) 2% retirement fund (percentage of payroll) Major Medical Expense Insurance Minor Medical Expense Insurance Accident Life Insurance Savings Fund (Optional) Fleet Auto Insurance (Optional) Commercial Partnership Agreements *We are an inclusive company free of any discrimination, where opportunities are based solely on attitude and talent. We do not require medical certificates confirming non-pregnancy or absence of Human Immunodeficiency Virus (HIV) as prerequisites for joining our large family of professionals and accessing equal growth opportunities.*