Technical Lead-AI QA

Birlasoft
Bengaluru
Workplace: OnsiteFull timeFunction: Data Science & Machine LearningExperience: 7+ yearsSkills: ["Attention to detail","Problem-solving","Debugging","Test automation","Quality mindset"]

Own validation and automation of testing for agentic AI and GenAI systems, using strong Python expertise. Build Python-based automation frameworks (e.g., Pytest) to test LLM outputs for accuracy, hallucination, and relevance, and validate multi-agent workflows and pipelines. Create evaluation metrics such as precision/recall and F1, perform API and integration testing, and ensure high coverage with seamless CI/CD integration.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Birlasoft
Birlasoft
1 day ago

Technical Lead-AI QA

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 3 hours agoStatus: Live

Job Summary

Own validation and automation of testing for agentic AI and GenAI systems, using strong Python expertise. Build Python-based automation frameworks (e.g., Pytest) to test LLM outputs for accuracy, hallucination, and relevance, and validate multi-agent workflows and pipelines. Create evaluation metrics such as precision/recall and F1, perform API and integration testing, and ensure high coverage with seamless CI/CD integration.
Location: Bengaluru
Workplace: Onsite
Employment Type: Full time
Job Function: Data Science & Machine Learning

Key Responsibilities

  • •Develop Python-based automation frameworks (Pytest) to validate agentic AI and GenAI systems.
  • •Test LLM outputs for accuracy, hallucination, and relevance.
  • •Validate multi-agent workflows and pipelines.
  • •Build evaluation metrics including F1, precision, and recall.
  • •Perform API/integration testing and integrate testing into CI/CD pipelines with high coverage.

Key Requirements

  • •7+ years of QA experience with exposure to GenAI/ML systems.
  • •Strong Python programming skills.
  • •Experience building automation frameworks using Pytest or unittest.
  • •Understanding of GenAI/LLMs and how to assess output quality.
  • •Hands-on API testing and debugging experience.
Experience:7+ yearsGenAIMLLLMs
Skills:Attention to detailProblem-solvingDebuggingTest automationQuality mindset
Tech Stack:PythonPytestUnittestGenAILLMsAPI testingCI/CDVersion controlF1PrecisionRecall

Company Brief

Birlasoft
Birlasoft is a global IT services and digital solutions provider delivering consulting, enterprise applications, cloud, and infrastructure services to enterprises across industries, helping accelerate digital transformation and business modernization.
Industry: Professional Services
Company Size: Enterprise (1,001+ employees)
Revenue: USD 500M to 1B
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Noida, India
WebsiteLinkedIn