DevOps Engineer - AI Runtime Services

Demandbase
Hyderabad
Workplace: HybridFull timeFunction: DevOps, Cloud & InfrastructureSkills: ["On-call operations","Incident response","Observability","System instrumentation","Accountability"]

Own and operate the shared AI runtime powering Demandbase’s LLM features, including the LLM gateway, reliability (SLOs, on-call, incident response), cost controls, and observability. Build and improve evals/experimentation infrastructure, caching, and safety/guardrails so product and internal teams can ship faster with confidence. Work across gateway and ingest pipelines, taking production ownership end to end.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Demandbase
Demandbase
1 month ago

DevOps Engineer - AI Runtime Services

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Own and operate the shared AI runtime powering Demandbase’s LLM features, including the LLM gateway, reliability (SLOs, on-call, incident response), cost controls, and observability. Build and improve evals/experimentation infrastructure, caching, and safety/guardrails so product and internal teams can ship faster with confidence. Work across gateway and ingest pipelines, taking production ownership end to end.
Location: Hyderabad
Workplace: Hybrid
Employment Type: Full time
Job Function: DevOps, Cloud & Infrastructure

Key Responsibilities

  • •Own the LLM gateway, including provider routing, rate limits, failover, and quotas.
  • •Provide SRE-style reliability: SLOs, on-call, incident response, and runtime postmortems.
  • •Implement cost controls with per-team/per-model attribution, budgets, and quotas.
  • •Deliver LLM observability with traces, spans, prompt/response capture, and ingest/billing visibility.
  • •Build evals and experimentation infrastructure, plus caching and guardrails (including PII handling and policy enforcement).

Pay and Benefits

Perks:Health InsuranceDentalVisionLife InsuranceGym Membership

Key Requirements

  • •Strong production infrastructure background with Kubernetes, AWS, Terraform, and GitOps.
  • •Experience owning services and tooling in Python (not just scripting).
  • •Real on-call and incident-response experience, including lessons learned and follow-up changes.
  • •Observability experience: instrument systems, manage metrics/logging cost/cardinality, and build reliable alerts.
  • •Hands-on experience building LLM systems (e.g., agents, RAG pipelines, internal tools) and using evals/experiments to evaluate quality.
Experience:LLMSREObservability
Skills:On-call operationsIncident responseObservabilitySystem instrumentationAccountability
Tech Stack:KubernetesAWSTerraformGitOpsPythonLLM gatewayLiteLLMDatadogPrometheusLokiClickHouseGitLab CIEKSFluxKarpenterGCPAPM

Company Brief

Demandbase
Demandbase provides an account‑based go‑to‑market (GTM) platform combining account intelligence, B2B intent data, advertising, and AI to help enterprises identify, engage, and convert high‑value accounts and accelerate pipeline growth.
Industry: Enterprise Software
Company Size: Large (251 to 1,000 employees)
Growth: Scaleup
Funding: Private Equity Backed
Headquarters: San Francisco, United States
Founded: 2007
Glassdoor
Glassdoor: 3.8
WebsiteLinkedInGlassdoor