Sr Architect (INDIA - BENGALURU - HP, IN)

Birlasoft
Bengaluru
Workplace: OnsiteFull timeFunction: Software EngineeringSkills: []

Build Python-based automation frameworks using Pytest/unittest to ensure reliable testing of GenAI/LLM outputs, focusing on accuracy, hallucination, and relevance. Validate multi-agent workflows and pipelines, and create evaluation metrics such as F1, precision, and recall. Perform API and integration testing, maintain strong test coverage, and integrate test runs into CI/CD and version control workflows to improve release quality.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Birlasoft
Birlasoft
3 days ago

Sr Architect (INDIA - BENGALURU - HP, IN)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 8 hours agoStatus: Live

Job Summary

Build Python-based automation frameworks using Pytest/unittest to ensure reliable testing of GenAI/LLM outputs, focusing on accuracy, hallucination, and relevance. Validate multi-agent workflows and pipelines, and create evaluation metrics such as F1, precision, and recall. Perform API and integration testing, maintain strong test coverage, and integrate test runs into CI/CD and version control workflows to improve release quality.
Location: Bengaluru
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Develop Python-based automation frameworks using Pytest.
  • •Test LLM outputs for accuracy, hallucination, and relevance.
  • •Validate multi-agent workflows and pipelines end to end.
  • •Build evaluation metrics such as F1, precision, and recall for model assessment.
  • •Perform API and integration testing and ensure strong test coverage with CI/CD integration.

Key Requirements

  • •Strong Python programming to build and maintain automation assets.
  • •Hands-on experience with automation frameworks such as Pytest and/or unittest.
  • •Understanding of GenAI/LLMs and how to evaluate model behavior (e.g., accuracy, hallucination, relevance).
  • •Experience with API testing and debugging/integration verification.
  • •Experience integrating tests into CI/CD and working with version control practices.
Tech Stack:PythonPytestUnittestGenAILLMsAPI testingCI/CDVersion controlMulti-agent workflowsAccuracyHallucinationRelevanceF1PrecisionRecall

Company Brief

Birlasoft
Birlasoft is a global IT services and digital solutions provider delivering consulting, enterprise applications, cloud, and infrastructure services to enterprises across industries, helping accelerate digital transformation and business modernization.
Industry: Professional Services
Company Size: Enterprise (1,001+ employees)
Revenue: USD 500M to 1B
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Noida, India
WebsiteLinkedIn