Staff Software Engineer, Foundational Model Serving

Databricks
San Francisco, Mountain View
Workplace: OnsiteFull timeUSD 192,000 - 260,000 annuallyFunction: Software EngineeringExperience: 10+ yearsSkills: ["Communication","Mentoring","Leadership","Team collaboration","Problem-solving"]

Lead the design and implementation of scalable Foundation Model Serving infrastructure, building high-throughput, low-latency GPU inference systems for frontier AI models. Collaborate across platform, product, infrastructure, and research teams to shape architecture, mentor engineers, and drive operational excellence while delivering a world-class API product for hosting open source and proprietary models.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Databricks
Databricks
10 months ago

Staff Software Engineer, Foundational Model Serving

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 3 hours agoStatus: Live

Job Summary

Lead the design and implementation of scalable Foundation Model Serving infrastructure, building high-throughput, low-latency GPU inference systems for frontier AI models. Collaborate across platform, product, infrastructure, and research teams to shape architecture, mentor engineers, and drive operational excellence while delivering a world-class API product for hosting open source and proprietary models.
Location: San Francisco, Mountain View
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering

Key Responsibilities

  • •Design and implement core systems and APIs powering the Foundation Model Serving platform with scalability, reliability, and operational excellence.
  • •Partner with product and engineering leadership to define the technical roadmap and long-term architecture for serving workloads.
  • •Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for GPU serving workloads.
  • •Contribute to key components across the serving infrastructure, including systems like vLLM and SGLang, and implement token-based rate limiters and optimizers.
  • •Collaborate cross-functionally with product, platform, and research teams to translate customer needs into reliable and performant systems.

Pay and Benefits

Salary: USD 192,000 - 260,000 annually

Key Requirements

  • •10+ years of experience building and operating large-scale distributed systems.
  • •Experience leading high-scale operationally sensitive backend systems.
  • •A track record of up-leveling teams engineering excellence.
  • •Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems.
  • •Proven ability to deliver technically complex, high-impact initiatives that create measurable customer or business value.
Experience:10+ yearsAIEnterprise softwareDistributed systems
Skills:CommunicationMentoringLeadershipTeam collaborationProblem-solving
Languages:English
Tech Stack:GPUAPIVLLMSGLangLLMInference

Company Brief

Databricks
Provides a unified data analytics platform powered by Apache Spark to simplify building, deploying, and scaling data engineering, data science, and machine learning workloads for enterprises.
Industry: Data Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series E+
Headquarters: San Francisco, United States
Founded: 2013
WebsiteLinkedIn