Glean

2 weeks ago

Software Engineer, Evals

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 8 hours agoStatus: Live

Job Summary

Build large-scale evaluation and observability systems that measure and improve assistant and agent quality for real and synthetic workflows. Own backend platforms for running evaluations, processing traces, powering dashboards and debugging, and producing signals for regressions and launch readiness across frontier and OSS model releases. Partner with product, ML, and infrastructure teams to make evals a first-class part of shipping AI features while balancing reliability, security, performance, and cost.

Location: Bengaluru

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Design and build large-scale evaluation pipelines measuring assistant and agent quality across real user and synthetic workflows.
•Evaluate frontier model releases and open-source model drops, creating infrastructure and quality signals for regressions, tradeoffs, and launch readiness.
•Build agent observability infrastructure including trace enrichment, telemetry pipelines, dashboards, and debugging workflows to make AI behavior inspectable.
•Own backend systems end-to-end from architecture and design docs through production rollout, reliability, monitoring, and iteration.
•Partner with product, ML, and infrastructure engineers to make evals a first-class part of how the company ships AI features and connect eval results to product improvements.

Key Requirements

•4+ years of software engineering experience building backend systems, infrastructure, distributed systems, or data platforms.
•Strong coding skills in Go, Python, Java, C++, or similar languages with an emphasis on reliability, scale, and well-tested components.
•Experience with distributed data pipelines, production services, observability systems, or cloud-native infrastructure.
•Analytical rigor and a focus on metrics that reflect real user experience.
•Experience with LLM applications, evals, tracing, data warehouses, workflow orchestration, or ML infrastructure is a strong plus.

Experience:4+ yearsLLM applicationsDistributed systemsData infrastructureObservabilityML infrastructure

Skills:Analytical rigorQuality focusCustomer-focused collaborationCross-functional teamworkMentorship

Languages:English

Tech Stack:GoPythonJavaC++LLM applicationsEvalsTracingObservabilityData pipelinesData warehousesWorkflow orchestrationDistributed systemsCloud-native infrastructureTelemetryDashboardsDebugging workflowsAgent observabilityFrontier model releasesOpen-source modelsMicroservices

Company Brief

Glean

Provides an enterprise search and knowledge discovery platform that helps organizations find information across apps, drives, and internal tools, improving employee productivity and onboarding through AI-powered search and relevance ranking.

Industry: Enterprise Software

Company Size: Large (251 to 1,000 employees)

Growth: Scaleup

Headquarters: San Francisco, United States

Founded: 2017

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

Glean

2 weeks ago

Software Engineer, Evals

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 8 hours agoStatus: Live

Job Summary

Location: Bengaluru

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Design and build large-scale evaluation pipelines measuring assistant and agent quality across real user and synthetic workflows.
•Evaluate frontier model releases and open-source model drops, creating infrastructure and quality signals for regressions, tradeoffs, and launch readiness.
•Build agent observability infrastructure including trace enrichment, telemetry pipelines, dashboards, and debugging workflows to make AI behavior inspectable.
•Own backend systems end-to-end from architecture and design docs through production rollout, reliability, monitoring, and iteration.
•Partner with product, ML, and infrastructure engineers to make evals a first-class part of how the company ships AI features and connect eval results to product improvements.

Key Requirements

•4+ years of software engineering experience building backend systems, infrastructure, distributed systems, or data platforms.
•Strong coding skills in Go, Python, Java, C++, or similar languages with an emphasis on reliability, scale, and well-tested components.
•Experience with distributed data pipelines, production services, observability systems, or cloud-native infrastructure.
•Analytical rigor and a focus on metrics that reflect real user experience.
•Experience with LLM applications, evals, tracing, data warehouses, workflow orchestration, or ML infrastructure is a strong plus.

Experience:4+ yearsLLM applicationsDistributed systemsData infrastructureObservabilityML infrastructure

Skills:Analytical rigorQuality focusCustomer-focused collaborationCross-functional teamworkMentorship

Languages:English

Company Brief

Glean

Industry: Enterprise Software

Company Size: Large (251 to 1,000 employees)

Growth: Scaleup

Headquarters: San Francisco, United States

Founded: 2017

Website LinkedIn