Staff Software Engineer, Foundation Model Inference

Databricks
Mountain View, San Francisco
Workplace: OnsiteFull timeUSD 190,000 - 265,000 annuallyFunction: Software EngineeringExperience: 8+ yearsSkills: ["Ownership","Customer empathy","Product mindset","Collaboration","System reliability focus"]

Build FMAPI, the unified serving layer for large language models powering real-time and batch inference at enterprise scale. You’ll design LLM infrastructure for partner and self-hosted models, improve reliability, latency, and efficiency of distributed AI workloads, and shape the product from roadmap to execution through direct engagement with users and model providers. Work closely with platform, infra, and ML teams to deliver seamless end-to-end AI developer experiences on Databricks.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Databricks
Databricks
1 month ago

Staff Software Engineer, Foundation Model Inference

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 15 hours agoStatus: Live

Job Summary

Build FMAPI, the unified serving layer for large language models powering real-time and batch inference at enterprise scale. You’ll design LLM infrastructure for partner and self-hosted models, improve reliability, latency, and efficiency of distributed AI workloads, and shape the product from roadmap to execution through direct engagement with users and model providers. Work closely with platform, infra, and ML teams to deliver seamless end-to-end AI developer experiences on Databricks.
Location: Mountain View, San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Build LLM infrastructure for large-scale inference workloads using partner and self-hosted models.
  • •Shape FMAPI product direction from roadmap to execution with customer and model-provider engagement.
  • •Improve reliability, latency, and efficiency of distributed AI workloads.
  • •Collaborate with platform, infra, and ML teams to deliver end-to-end AI experiences.
  • •Help shape how developers and data scientists build and interact with AI on Databricks.

Pay and Benefits

Salary: USD 190,000 - 265,000 annually
Equity and Bonus:Equity

Key Requirements

  • •8+ years of backend or infrastructure engineering experience.
  • •Experience with distributed systems, scalable APIs, and/or cloud-native infrastructure.
  • •Strong product and ownership mindset focused on shipping user-facing value.
  • •Experience with real-time serving, ML infrastructure, or GPU orchestration.
  • •Strong programming skills in Scala, Go, or Python.
Experience:8+ yearsAI infrastructureLLM infrastructureCloud-nativeDistributed systemsMachine learningGPU orchestration
Skills:OwnershipCustomer empathyProduct mindsetCollaborationSystem reliability focus
Languages:English
Tech Stack:ScalaGoPythonLLMFoundation Model APIsFMAPIDistributed systemsScalable APIsCloud-native infrastructureReal-time servingML infrastructureGPU orchestrationService-oriented architectureDeployment pipelinesSystem observabilityVector SearchOpenAIAnthropicGeminiQwen

Company Brief

Databricks
Provides a unified data analytics platform powered by Apache Spark to simplify building, deploying, and scaling data engineering, data science, and machine learning workloads for enterprises.
Industry: Data Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series E+
Headquarters: San Francisco, United States
Founded: 2013
WebsiteLinkedIn