Lead Azure Data Engineer

Description

Summary: An Azure Data Engineer designs and optimizes data pipelines and architectures using Azure cloud services, focusing on efficient data ingestion and transformation. Highlights: 1. Design and optimize data pipelines and architectures using Azure cloud services. 2. Utilize strong SQL, Python, and Spark skills for ETL/ELT processes. 3. Focus on data modeling, performance tuning, and security compliance. **Azure Data Engineer (Ingeniero de Datos Azure)** **Location:** Mexico (Mexico City, Guadalajara, Monterrey) – Hybrid as per Infosys Mexico policy (CST/EST time) **Role Overview** As an **Azure** **Data Engineer**, you will be designing and optimizing data pipelines and architectures using Azure cloud services. This role requires strong SQL skills, expertise in Python and Spark, and experience with ETL/ELT processes for efficient data ingestion and transformation. Key capabilities include data modeling, performance tuning, integration with various data sources and BI tools, and ensuring security and governance compliance. **Key Responsibilities** * Review team monitoring reports to identify issues and create the corresponding PBIs/user stories for resolution. * Monitor and respond to alert emails regarding execution failures across all team tools—such as Databricks, Azure Data Factory, or data quality gap reports within our processes. * Proactively respond to user questions on team Zoom channels, capture issues in our processes, and implement necessary fixes. * Work extensively with Azure Cloud Services, Azure Data Factory, Databricks, Python, and Spark, including handling user requested break\-ins such as issue investigation, missing data analysis and remediation of jobs or pipelines that did not run according to their pre\-configured schedules. **Basic Qualifications** * Bachelor’s degree or foreign equivalent in Computer Science or related field. * **7\+ years of IT experience**, including pipeline design, support and development (Batch and Streaming) using Azure cloud services. * Excellent assertive communication with users and stakeholders in English. * Understanding of Agile methodologies. **Mandatory Skills** * Proficiency in Python: Advanced/expert\-level proven experience in Python and Spark. * Strong understanding of ETL/ELT processes: Proven experience designing and maintaining ETL processes in Databricks and Azure Data Factory, including connectivity with third party systems/APIs, Event Hubs, and other potential data sources. * Experience with Azure Cloud Services: Demonstrated knowledge of cloud platforms—specifically Azure—including storage, security, and resource optimization. **Preferred Skills** * Experience working with Git\-based version control (e.g., GitHub, GitLab, Bitbucket) and familiarity with branching strategies, pull requests, and code review workflows. * Focus on system maintenance: Strong emphasis on proactive detection and fast reactive maintenance of running jobs and pipelines. **Other Requirements** * Ability to work under tight timelines and manage complex requirements. * Flexible to work during CST/EST time zones and meet for hand\-over activities to India team if needed. * Self\-motivated, proactive, and a strong team player.