AI Evaluation Engineer (QA)

Appnovation
San Jose, Medellín, Bogotá, Mexico City, Buenos Aires, São Paulo, Toronto
Workplace: HybridContractFunction: Data Science & Machine LearningExperience: 8-12 yearsEducation: bachelorsSkills: ["Systems thinking","Problem discovery","Precise communication","Outcome ownership","Curiosity"]

Design and build scalable data pipelines and cloud-native data infrastructure that enable analytics and AI capabilities. Define and implement data validation and AI/ML evaluation frameworks to support deployment and governance, while also delivering reliable systems with clear SLOs/SLIs. Lead end-to-end solution delivery (ETL/ELT, storage, and infrastructure), prototype full-stack data applications, and drive resilient, well-tested implementations across complex enterprise environments.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Appnovation
Appnovation
1 day ago

AI Evaluation Engineer (QA)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 12 hours agoStatus: Live

Job Summary

Design and build scalable data pipelines and cloud-native data infrastructure that enable analytics and AI capabilities. Define and implement data validation and AI/ML evaluation frameworks to support deployment and governance, while also delivering reliable systems with clear SLOs/SLIs. Lead end-to-end solution delivery (ETL/ELT, storage, and infrastructure), prototype full-stack data applications, and drive resilient, well-tested implementations across complex enterprise environments.
Location: San Jose, Medellín, Bogotá, Mexico City, Buenos Aires, São Paulo, Toronto
Workplace: Hybrid
Employment Type: Contract · Permanent
Job Function: Data Science & Machine Learning

Key Responsibilities

  • •Define and build scalable data pipelines and cloud-native data infrastructure to enable analytics and AI capabilities.
  • •Design data validation and AI/ML evaluation frameworks that support model deployment, governance, and reliability.
  • •Prototype and deliver full end-to-end data applications across diverse technology stacks, prioritizing speed and business value while managing technical debt.
  • •Deploy and operate data systems with reliability and resilience by defining and monitoring SLOs/SLIs and improving after incidents.
  • •Perform advanced data modeling and manage production schema migrations, making architectural trade-offs and producing well-tested code.

Key Requirements

  • •Bachelor’s degree in Computer Science or Engineering with 8–12 years of relevant experience delivering complex data engineering projects with multi-team or business impact.
  • •Proven expertise designing and maintaining scalable ETL/ELT data pipelines across complex enterprise data landscapes, including handling undocumented schemas.
  • •Deep experience deploying and managing data infrastructure on cloud platforms (AWS, Azure, or GCP), including Infrastructure as Code (Terraform/CloudFormation) and CI/CD pipelines for data solutions.
  • •Strong data modeling skills (conceptual/logical/physical) with the ability to balance normalization, query performance, and production schema migrations.
  • •Mastery of incorporating AI tools into data engineering workflows, including designing data validation and evaluation frameworks for AI/ML model deployment and governance.
Experience:8-12 years
Education:Bachelor's in Computer Science or Engineering
Skills:Systems thinkingProblem discoveryPrecise communicationOutcome ownershipCuriosity
Languages:English
Tech Stack:ETLELTAWSAzureGCPTerraformCloudFormationCI/CDSLOsSLIsAI/MLInfrastructure as Code

Company Brief

Appnovation
Provides digital transformation, design, and engineering services, helping enterprises build customer experiences, digital platforms, and integrations using open-source and cloud technologies across industries.
Industry: Consulting
Company Size: Enterprise (1,001+ employees)
Growth: Established Company
Funding: Bootstrapped
Headquarters: Vancouver, Canada
Founded: 2007
WebsiteLinkedIn