Senior Engineer II, Inference Engine - Serving Engine

Digital Ocean
Seattle
Workplace: RemoteFull timeUSD 167,200 - 209,000Function: Solutions Engineering & Sales EngineeringSkills: ["Go","Python","GRPC","Tensor/data parallelism","KV cache optimization","Routing","Observability","SLOs"]

Senior Engineer 2 leads the design, development, and delivery of high-scale data plane components powering DigitalOcean's AI Inference Data Plane. You’ll work at the intersection of distributed systems and AI hardware to ensure scalable, reliable model deployment, with hands-on software development and collaboration across product, customers, and engineering teams.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Digital Ocean
Digital Ocean
4 months ago

Senior Engineer II, Inference Engine - Serving Engine

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 6 hours agoStatus: Live

Job Summary

Senior Engineer 2 leads the design, development, and delivery of high-scale data plane components powering DigitalOcean's AI Inference Data Plane. You’ll work at the intersection of distributed systems and AI hardware to ensure scalable, reliable model deployment, with hands-on software development and collaboration across product, customers, and engineering teams.
Location: Seattle
Workplace: Remote
Employment Type: Full time
Job Function: Solutions Engineering & Sales Engineering
Seniority: Sr. Manager level

Key Responsibilities

  • •Act as a technical leader on the team, driving the end-to-end design, development, and delivery of critical data plane components hosting large generative AI models.
  • •Architect and refine system design proposals for a high-scale, multi-tenant AI inference cloud ecosystem, ensuring they meet availability and resiliency standards.
  • •Implement and optimize distributed inference hosting using techniques like tensor/data parallelism, KV cache optimizations, and smart routing.
  • •Collaborate cross-functionally with Product Managers, customer-facing teams, and other engineering teams to align technical roadmaps with customer needs.
  • •Coach and mentor junior engineers, fostering a culture of technical excellence and continuous improvement.

Pay and Benefits

Salary: USD 167,200 - 209,000

Key Requirements

  • •Experience designing and delivering high-scale data plane services within a distributed systems environment.
  • •Strong distributed systems expertise: microservices, messaging systems, databases, and infrastructure as code.
Experience:Ai/mlDistributed systemsCloud computing
Skills:GoPythonGRPCTensor/data parallelismKV cache optimizationRoutingObservabilitySLOs
Languages:English
Tech Stack:GoPythonGRPCVLLMSGLangModularLlm-dNVIDIA DynamoRay ServeNVLinkXGMIRoCEQuantization

Company Brief

Digital Ocean
Provides cloud infrastructure and developer-focused cloud services including scalable droplets, managed databases, Kubernetes, object storage, and networking to simplify deploying and managing applications for developers and small-to-medium businesses.
Industry: Cloud Computing
Company Size: Enterprise (1,001+ employees)
Revenue: USD 250M to 500M
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: New York, United States
Founded: 2011
Glassdoor
Glassdoor: 3.8
WebsiteLinkedIn