AI Evaluation Engineer (QA)

Appnovation
São Paulo, Toronto
Workplace: HybridFull timeFunction: Data Science & Machine LearningExperience: 8-12 yearsEducation: bachelorsSkills: ["Autonomy","Architectural thinking","Systems thinking","Communication","Problem solving"]

Design and build an end-to-end “discovery-to-scale” data and AI evaluation capability, creating robust pipelines (ETL/ELT) that integrate, validate, and support governance for AI/ML model deployment. Define SLOs/SLIs and monitor resilient, production-grade data systems with graceful degradation. Prototype full data applications, establish CI/CD and infrastructure-as-code on cloud platforms (AWS, Azure, or GCP), and drive architectural trade-offs across teams.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Appnovation
Appnovation
1 day ago

AI Evaluation Engineer (QA)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 12 hours agoStatus: Live

Job Summary

Design and build an end-to-end “discovery-to-scale” data and AI evaluation capability, creating robust pipelines (ETL/ELT) that integrate, validate, and support governance for AI/ML model deployment. Define SLOs/SLIs and monitor resilient, production-grade data systems with graceful degradation. Prototype full data applications, establish CI/CD and infrastructure-as-code on cloud platforms (AWS, Azure, or GCP), and drive architectural trade-offs across teams.
Location: São Paulo, Toronto
Workplace: Hybrid
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Build data integration, storage, and infrastructure to enable analytics and AI capabilities from discovery through scalable deployment.
  • •Develop resilient enterprise data pipelines and integration solutions for high-volume data, including undocumented schemas.
  • •Establish cloud-native data ecosystems with Infrastructure as Code and end-to-end CI/CD for data solutions.
  • •Define and monitor SLOs/SLIs, design highly resilient data systems, and drive post-incident improvements.
  • •Prototype and deliver end-to-end data applications while making architectural trade-offs and managing technical debt.

Key Requirements

  • •Bachelor’s degree in Computer Science or Engineering with 8–12 years of relevant experience delivering complex data engineering projects with multi-team or business impact.
  • •Design, build, and maintain scalable data pipelines (ETL/ELT) in complex enterprise data landscapes, including handling undocumented schemas.
  • •Deploy and manage cloud-native data infrastructure on AWS, Azure, or GCP; use Infrastructure as Code (Terraform/CloudFormation) and set up end-to-end CI/CD for data solutions.
  • •Create conceptual, logical, and physical data models, balancing normalization, query performance, and production schema migrations.
  • •Incorporate AI tools to optimize data engineering workflows by designing data validation and evaluation frameworks for AI/ML model deployment and governance.
Experience:8-12 yearsData engineeringAI/MLEnterprise data
Education:Bachelor's in Computer Science or Engineering
Skills:AutonomyArchitectural thinkingSystems thinkingCommunicationProblem solving
Tech Stack:ETLELTAWSAzureGCPTerraformCloudFormationCI/CDSLOsSLIsData modelingData validationAI/ML

Company Brief

Appnovation
Provides digital transformation, design, and engineering services, helping enterprises build customer experiences, digital platforms, and integrations using open-source and cloud technologies across industries.
Industry: Consulting
Company Size: Enterprise (1,001+ employees)
Growth: Established Company
Funding: Bootstrapped
Headquarters: Vancouver, Canada
Founded: 2007
WebsiteLinkedIn