Infrastructure Engineer

Arena
San Francisco
Workplace: HybridFull timeFunction: DevOps, Cloud & InfrastructureExperience: 4+ yearsSkills: ["Product-oriented mindset","Comfort with ambiguity"]

Build core infrastructure for online AI evaluations, including low-latency APIs, streaming gateways (SSE), and enterprise-grade serving layers. You’ll handle bursty traffic, partial failures, and consistent response normalization across multiple model providers. Own observability with distributed tracing and real-time dashboards, integrate with the evaluation platform and benchmarks, and contribute to backend systems that power leaderboards and evals at scale.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Arena
Arena
1 month ago

Infrastructure Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 7 hours agoStatus: Live

Job Summary

Build core infrastructure for online AI evaluations, including low-latency APIs, streaming gateways (SSE), and enterprise-grade serving layers. You’ll handle bursty traffic, partial failures, and consistent response normalization across multiple model providers. Own observability with distributed tracing and real-time dashboards, integrate with the evaluation platform and benchmarks, and contribute to backend systems that power leaderboards and evals at scale.
Location: San Francisco
Workplace: Hybrid
Employment Type: Full time
Job Function: DevOps, Cloud & Infrastructure
Seniority: Manager level

Key Responsibilities

  • •Design and implement low-latency, high-reliability APIs for leaderboards, models, and arenas.
  • •Build streaming/gateway functionality for SSE responses across heterogeneous providers, including partial failure recovery, mid-stream fallback, and consistent normalization.
  • •Develop enterprise-grade infrastructure features such as rate limiting, authentication, usage metering, cost attribution, audit logging, and SOC 2 compliance support.
  • •Implement deep observability using distributed tracing, latency breakdowns, token-level usage tracking, and real-time dashboards.
  • •Integrate with the evaluation platform, Arena data, and customer benchmarks, and contribute to backend systems for leaderboards and evals as needed.

Pay and Benefits

Perks:Health InsuranceDentalVision

Key Requirements

  • •4+ years of backend engineering experience focused on distributed systems, infrastructure, or developer-facing platforms.
  • •Strong proficiency in Go and/or Rust, building high-throughput APIs and/or proxy/gateway systems.
  • •Experience integrating LLM provider APIs (e.g., OpenAI, Anthropic, Google) with knowledge of streaming, token management, rate limits, and model quirks.
  • •Comfort with cloud infrastructure (AWS or GCP), Kubernetes, Terraform, and databases such as Postgres and Redis.
  • •A product-oriented mindset for developer experience and comfort operating with ambiguity in a fast-changing startup.
Experience:4+ yearsAILLMDistributed systemsCloud infrastructureEnterprise infrastructure
Skills:Product-oriented mindsetComfort with ambiguity
Tech Stack:GoRustSSEOpenAIAnthropicGoogleAWSGCPKubernetesTerraformPostgresRedisSOC 2Distributed tracingToken-level usage trackingAPI gatewaysProxies

Company Brief

Arena
LMArena operates a community-driven platform for evaluating and benchmarking large language models via crowdsourced pairwise comparisons and leaderboards, used by researchers and AI labs to measure real-world model performance. ([linkedin.com](https://www.linkedin.com/company/lmarena?utm_source=openai))
Industry: Developer Tools
Company Size: Small (11 to 50 employees)
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series A
Headquarters: San Francisco, United States
Founded: 2025
WebsiteLinkedIn