Subcontractor- Gen AI QA (INDIA - BENGALURU - HP, IN)

Birlasoft
Bengaluru
Workplace: OnsiteContractFunction: Data Science & Machine LearningExperience: 7+ yearsSkills: []

Validate and automate testing for agentic AI and GenAI systems, using strong Python skills to build test automation frameworks and ensure robust test coverage. Test LLM outputs for accuracy, hallucination, and relevance, and validate multi-agent workflows and pipelines. Create evaluation metrics (F1, precision, recall) and execute API/integration testing with tight CI/CD and version-control integration.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Birlasoft
Birlasoft
3 days ago

Subcontractor- Gen AI QA (INDIA - BENGALURU - HP, IN)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 8 hours agoStatus: Live

Job Summary

Validate and automate testing for agentic AI and GenAI systems, using strong Python skills to build test automation frameworks and ensure robust test coverage. Test LLM outputs for accuracy, hallucination, and relevance, and validate multi-agent workflows and pipelines. Create evaluation metrics (F1, precision, recall) and execute API/integration testing with tight CI/CD and version-control integration.
Location: Bengaluru
Workplace: Onsite
Employment Type: Contract
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Develop Python-based automation frameworks using Pytest.
  • •Test LLM outputs for accuracy, hallucination, and relevance.
  • •Validate multi-agent workflows and pipelines end to end.
  • •Build evaluation metrics such as F1, precision, and recall.
  • •Perform API and integration testing and integrate test coverage into CI/CD.

Key Requirements

  • •7+ years of QA experience with exposure to GenAI/ML systems.
  • •Strong Python programming for building QA automation.
  • •Experience with automation frameworks such as Pytest and/or unittest.
  • •Understanding of GenAI/LLMs and how to test LLM outputs for quality signals.
  • •Ability to perform API and integration testing, plus experience with CI/CD and version control.
Experience:7+ yearsGenAILLMsMachine learningQA
Tech Stack:PythonPytestUnittestGenAILLMsAPI testingCI/CDF1PrecisionRecallMulti-agent workflowsVersion control

Company Brief

Birlasoft
Birlasoft is a global IT services and digital solutions provider delivering consulting, enterprise applications, cloud, and infrastructure services to enterprises across industries, helping accelerate digital transformation and business modernization.
Industry: Professional Services
Company Size: Enterprise (1,001+ employees)
Revenue: USD 500M to 1B
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Noida, India
WebsiteLinkedIn