AI Engineer

Description

Summary: This role involves evaluating and integrating AI capabilities, specifically Veeva AI and generative AI, within regulated environments, focusing on quality, consistency, and compliance. Highlights: 1. Evaluate and integrate Veeva AI and Generative AI systems 2. Ensure AI quality, consistency, and compliance in regulated settings 3. Collaborate across teams for robust AI solutions and governance **Work Schedule** Standard (Mon\-Fri)**Environmental Conditions** Office**Job Description** Qualifications: **Education and Experience:** ----------------------------- Bachelor's degree or equivalent and relevant formal academic / vocational qualification *Master's degree preferred* Previous experience that provides the knowledge, skills, and abilities to perform the job (minimum 2\+ years). *In some cases, an equivalency, consisting of a combination of appropriate education, training.* *and/or directly related experience, will be considered sufficient for an individual to meet the* *requirements of the role.* **Knowledge, Skills, and Abilities:** ------------------------------------- * **Veeva AI \& SaaS AI Evaluation** + Ability to evaluate vendor\-provided AI capabilities (specifically Veeva AI) where model internals are not directly accessible. + Experience assessing AI behavior for quality, consistency, failure modes, latency, and operational constraints in enterprise workflows. + Ability to distinguish when a SaaS AI capability should be adopted as\-is, adapted with platform guardrails, or rejected in favor of a custom internal solution. * **Generative AI Systems Understanding** + Strong understanding of generative AI behavior, including non\-determinism, hallucination risk, prompt sensitivity, and configuration\-driven outcomes. + Experience applying lightweight evaluation techniques for generative AI (e.g., task\-specific test cases, accuracy checks, hallucination tracking, refusal behavior). + Ability to reason about AI suitability in regulated or high\-impact decision contexts. * **AI Evaluation, Validation, and Documentation** + Proficiency in documenting AI evaluations in clear, audit\-ready formats, including data flows, decision points, risks, and mitigations. + Ability to partner with AI governance functions to support model cards, validation summaries, and AI registry or approval artifacts. + Experience supporting audit and compliance reviews through evidence collection (logs, configurations, access controls). * **Platform Integration \& AI Guardrails** + Ability to integrate SaaS AI capabilities into enterprise platforms using approved architectural patterns. + Experience designing and implementing integration “shims” such as additional logging, input/output filtering, workflow controls, or human\-in\-the\-loop steps. + Understanding of AI telemetry requirements, including latency, error handling, usage tracking, and cost observability. * **Technical Proficiency** + Strong proficiency in Python and modern SQL for integration, analysis, and validation tasks. + Experience working with REST APIs, SaaS integrations, and enterprise authentication and authorization patterns. + Familiarity with Databricks concepts (jobs, notebooks, MLflow) and cloud data storage patterns (e.g., Azure Blob or S3\-equivalent). * **Operating in Regulated Environments** + Experience working within regulated, validated, or compliance\-driven environments (e.g., clinical research, life sciences, healthcare). + Ability to operate within defined separation of duties across build, review, and operate phases. + Comfort working in documentation\-first and control\-oriented delivery models. * **Collaboration and Ways of Working** + Proven ability to work effectively in a multi\-functional, matrixed environment across Product, Platform Engineering, AI Governance, and external vendors. + Strong written and verbal communication skills, with the ability to explain technical AI behavior and risks to non\-technical stakeholders. + Ability to manage multiple concurrent evaluations and priorities in the context of evolving vendor roadmaps and business needs. **Management Role:** ==================== No management responsibility **Working Conditions and Environment:** ======================================= * Work is performed in an office environment with exposure to electrical office equipment. * Occasional drives to site locations with occasional travel both domestic and international. **Physical Requirements:** ========================== * Frequently stationary for 6\-8 hours per day. * Repetitive hand movement of both hands with the ability to make fast, simple, repeated movements * of the fingers, hands, and wrists. * Frequent mobility required. * Occasional crouching, stooping, bending and twisting of upper body and neck. * Light to moderate lifting and carrying (or otherwise moves) objects including luggage and laptop * computer with a maximum lift of 15\-20 lbs. * Ability to access and use a variety of computer software developed both in\-house and off\-the\-shelf. * Ability to communicate information and ideas so others will understand; with the ability to listen to * and understand information and ideas presented through spoken words and sentences. * Frequently interacts with others to obtain or relate information to diverse groups. * Works independently with little guidance or reliance on oral or written instructions and plans work * schedules to meet goals. Requires multiple periods of intense concentration. * Performs a wide range of variable tasks as dictated by variable demands and changing conditions * with little predictability as to the occurrence. Ability to perform under stress. Ability to multitask. * Regular and consistent attendance. **Percent Billable:** ===================== 0% \- 20%