Software Engineer, Model Routing & Inference

Cursor
New York, San Francisco
Workplace: OnsiteFull timeFunction: Software EngineeringSkills: ["Communication","Problem-solving","Team collaboration"]

Software Engineer on Cursor's Model Routing & Inference team builds the inference platform powering AI interactions. You’ll own high-throughput, low-latency distributed systems, focusing on inference serving, traffic routing, and real-time data pipelines to make Cursor’s AI faster, more reliable, and cost-efficient at scale. You’ll tackle gateway abstractions, cross-provider failover, and backpressure/admission control for millions of requests.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Cursor
Cursor
5 months ago

Software Engineer, Model Routing & Inference

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 3 hours agoStatus: Live

Job Summary

Software Engineer on Cursor's Model Routing & Inference team builds the inference platform powering AI interactions. You’ll own high-throughput, low-latency distributed systems, focusing on inference serving, traffic routing, and real-time data pipelines to make Cursor’s AI faster, more reliable, and cost-efficient at scale. You’ll tackle gateway abstractions, cross-provider failover, and backpressure/admission control for millions of requests.
Location: New York, San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering

Key Responsibilities

  • •Build and maintain Cursor's inference platform used by all AI interactions, ensuring high availability and efficient resource usage.
  • •Develop and evolve the inference gateway to abstract provider API semantics and enable model onboarding via configuration.
  • •Design cross-provider failover so outages of a single provider do not degrade user experience.
  • •Implement routing backpressure and admission control to prevent traffic spikes from cascading to providers.
  • •Ship production systems that handle millions of requests, balancing reliability, cost, latency, and user experience.

Key Requirements

  • •Deep experience building high-throughput, low-latency distributed systems, especially in inference serving, traffic routing, or real-time data pipelines.
  • •Ability to reason about cost/performance tradeoffs at scale, including GPU utilization, provider economics, and capacity planning.
  • •Strong software engineering fundamentals and a track record of shipping production systems that handle millions of requests.
  • •Ability to make good decisions balancing reliability, cost, latency, and user experience when there isn't a single right answer.
  • •Experience building scalable platforms and abstractions over multiple providers' APIs (e.g., an inference gateway).
Experience:InfrastructureDistributed systemsAI servingReal-time data pipelines
Skills:CommunicationProblem-solvingTeam collaboration
Tech Stack:Inference servingRoutingHigh throughputLow latencyBackpressureAdmission control

Company Brief

Cursor
Builds Cursor, an AI-powered development platform and code editor that helps developers write, review, and maintain code using large language models and developer-focused automation tools for enterprise engineering teams.
Industry: Developer Tools
Company Size: Large (251 to 1,000 employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series D
Headquarters: San Francisco, United States
Founded: 2022
Glassdoor
Glassdoor: 3.0
WebsiteLinkedInGlassdoor