Staff Software Engineer- Foundation Model Inference

Databricks
Mountain View
Workplace: OnsiteFull timeUSD 190,000 - 265,000Function: Software EngineeringExperience: 8+ yearsSkills: ["Collaboration","Systems thinking","Problem-solving","Reliability focus"]

Build the Foundation Model Inference infrastructure that powers enterprise-scale generative AI. You’ll deliver LLM serving and scalable inference for partner and self-hosted models, improving reliability, latency, and efficiency of distributed workloads. Collaborate across platform, infrastructure, and ML teams to create end-to-end developer experiences on Databricks, helping developers and data scientists deploy and interact with AI via unified Foundation Model APIs.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Databricks
Databricks
1 month ago

Staff Software Engineer- Foundation Model Inference

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 14 hours agoStatus: Live

Job Summary

Build the Foundation Model Inference infrastructure that powers enterprise-scale generative AI. You’ll deliver LLM serving and scalable inference for partner and self-hosted models, improving reliability, latency, and efficiency of distributed workloads. Collaborate across platform, infrastructure, and ML teams to create end-to-end developer experiences on Databricks, helping developers and data scientists deploy and interact with AI via unified Foundation Model APIs.
Location: Mountain View
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Build LLM infrastructure for large-scale inference workloads using partner and self-hosted models.
  • •Improve reliability, latency, and efficiency of distributed AI workloads.
  • •Collaborate with platform, infra, and ML teams to deliver seamless end-to-end experiences.
  • •Shape how developers and data scientists build and interact with AI on Databricks.

Pay and Benefits

Salary: USD 190,000 - 265,000
Equity and Bonus:Equity

Key Requirements

  • •8+ years of experience in backend or infrastructure engineering.
  • •Experience with distributed systems, scalable APIs, or cloud-native infrastructure.
  • •Experience with real-time serving, ML infrastructure, or GPU orchestration.
  • •Familiarity with service-oriented architecture, deployment pipelines, and system observability.
Experience:8+ yearsAI infrastructureLLM servingDistributed systems
Skills:CollaborationSystems thinkingProblem-solvingReliability focus
Languages:English
Tech Stack:LLMFoundation Model APIsOpenAIAnthropicGeminiQwenGPT-OSSLlamaDistributed systemsScalable APIsCloud-native infrastructureReal-time servingML infrastructureGPU orchestrationService-oriented architectureDeployment pipelinesSystem observabilitySageMakerVertex AIAzure ML

Company Brief

Databricks
Provides a unified data analytics platform powered by Apache Spark to simplify building, deploying, and scaling data engineering, data science, and machine learning workloads for enterprises.
Industry: Data Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series E+
Headquarters: San Francisco, United States
Founded: 2013
WebsiteLinkedIn