We are looking for **AI Evaluation \& Data Engineering Specialists** to design, curate, and operationalize datasets and evaluation frameworks for AI product performance assessment. This role involves working with large language models (LLMs), human raters, and automation tools to measure model accuracy, correctness, and usability. Key Responsibilities * Build and maintain **evaluation datasets** for AI models across programming languages (Python, Golang, JavaScript, Java). * Develop and apply **data labeling and scoring guidelines** based on Google’s evaluation framework. * Implement **LLM\-judge calibration workflows** to align automated and human evaluations. * Perform **error analysis, drift detection**, and regression testing of AI model outputs. * Collaborate with automation engineers to integrate datasets into evaluation pipelines. * Support **rater training**, inter\-rater reliability checks, and dataset validation reviews. * Manage **data quality assurance** and documentation for contributions to Google\-maintained repositories. Required Skills \& Experience * plus 4 years of experience in **AI/ML data operations**, **evaluation**, or **data engineering**. * Proficiency in **Python** (mandatory) for dataset manipulation, analysis, and scripting. * Experience with **LLM evaluation**, **prompt engineering**, or **text generation quality assessment**. * Familiarity with **Gemini CLI, Vertex AI, or LangChain evaluation tools**. * Strong understanding of **data curation, annotation workflows**, and **labeling quality metrics**. * Hands\-on with **Git\-based repositories** and CI/CD data workflows. * Excellent analytical and problem\-solving skills with attention to detail. Preferred Qualifications * Experience evaluating code\-generation or NLP\-based AI products. * Exposure to **data governance and privacy compliance frameworks**. * Background in computer science, data science, or linguistics preferred. Tipo de puesto: Tiempo completo, Por tiempo indeterminado Sueldo: $60,000\.00 \- $70,000\.00 al mes Pregunta(s) de postulación: * Familiarity with Gemini CLI, Vertex AI, or LangChain evaluation tools * Proficiency in Python (mandatory) for dataset manipulation, analysis, and scripting. * Hands\-on with Git\-based repositories and CI/CD data workflows. Lugar de trabajo: Empleo presencial