DataOps Engineer (AWS Specialist)

Description

Job Summary: We are seeking a DataOps Engineer passionate about automation and cloud-based data infrastructure, responsible for designing, implementing, and maintaining CI/CD processes for data pipelines on AWS. Key Highlights: 1. Design, implement, and maintain CI/CD processes for data pipelines 2. Ensure data quality, observability, and availability on AWS 3. Infrastructure automation and pipeline orchestration We are looking for a DataOps Engineer passionate about automation and cloud-based data infrastructure. You will be responsible for designing, implementing, and maintaining CI/CD processes for data pipelines, ensuring data quality, observability, and availability on our **AWS** platform. Your goal is to reduce the data development lifecycle and ensure seamless flow from ingestion to analytics. **Key Responsibilities** * **Infrastructure Automation:** Implement and manage infrastructure as code (IaC) using **Terraform** or **AWS CloudFormation**. * **Orchestration and Pipelines:** Configure and optimize workflows using **AWS Step Functions**, **Managed Workflows for Apache Airflow (MWAA)**, or Glue Workflows. * **CI/CD for Data:** Develop automated deployments for data models and ETL scripts using **AWS CodePipeline** or GitHub Actions. * **Monitoring and Observability:** Set up data health alerts and dashboards using **CloudWatch**, DataDog, or data quality tools (e.g., Great Expectations). * **Security and Compliance:** Manage access controls and security policies via **AWS IAM**, Lake Formation, and data encryption (KMS). * **Cost Optimization:** Monitor cloud resource consumption and propose improvements to optimize spending on services such as S3, Redshift, and EMR. **Technical Requirements** * **AWS Experience:** Proficiency in core AWS services (**S3, Glue, Lambda, Athena, Redshift, EMR**). * **Programming Languages:** Advanced **Python** and strong SQL skills. * **Containers:** Experience with **Docker** and orchestrators such as **Amazon ECS** or **EKS**. * **Databases:** Experience with both SQL (PostgreSQL, RDS) and NoSQL (DynamoDB). * **Version Control:** Expert-level Git usage and branching strategies. **Preferred Qualifications** * *AWS Certified Data Engineer* or *AWS Certified DevOps Engineer* certification. * Experience with transformation tools such as **dbt** (data build tool). * Knowledge of data streaming (**AWS Kinesis** or Kafka). * Familiarity with Agile methodologies and DataOps frameworks. **Soft Skills** * Analytical ability to troubleshoot production issues. * Excellent communication skills to collaborate effectively with Data Engineers and Data Scientists. Employment Type: Full-time Salary: $75,000\.00 \- $85,000\.00 per month Benefits: * Salary increases * Maternity leave exceeding statutory requirements * Paternity leave exceeding statutory requirements * Sick leave * Flexible working hours * Option for indefinite-term contract * Remote work * Additional vacation days or paid leave Experience: * AWS: 4 years (Mandatory) Work Location: Hybrid remote in 06600, Juárez, CDMX