Tech Lead Manager, Inference

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 2 hours agoStatus: Live

Job Summary

Lead Luma’s inference platform team to scale a production serving stack across thousands of GPUs and heterogeneous fleets. You’ll set the technical roadmap for routing, scheduling, autoscaling, caching, observability, and deployment pipelines—while staying hands-on with architecture, debugging, and platform incident response. Partner with research to ship new model architectures quickly and integrate serving into online reinforcement learning. Own SLOs, performance, and cost per generation.

Location: Redwood City

Workplace: Hybrid

Employment Type: Full time

Job Function: Administration & Executive Assistance

Seniority: Manager level

Key Responsibilities

•Spend at least half your time hands-on designing, building, and debugging core serving stack components and production incidents.
•Lead, grow, and develop the inference engineering team, including hiring, coaching, on-call ownership, incident response, capacity planning, and postmortems.
•Set the technical roadmap for model serving engines and the end-to-end serving platform, including routing/scheduling, autoscaling, caching, observability, and deployment pipelines.
•Own platform SLOs and economics across latency/availability targets, GPU utilization, and cost per generation for models in production.
•Partner with research to ship new model architectures into production rapidly and integrate serving into online reinforcement learning and evaluation loops.

Key Requirements

•8+ years engineering experience in large-scale distributed systems or ML infrastructure, including building/operating model-serving or inference platforms in production.
•Experience operating inference platforms at scale, including fleets of thousands of GPUs across multiple clusters or clouds.
•Technical leadership experience managing or leading engineers through rapid growth while keeping significant hands-on time.
•Deep, practical expertise in LLM and foundation-model serving engines such as vLLM, SGLang, or TensorRT-LLM.
•Strong command of serving performance practices including continuous batching, KV-cache management, quantization, speculative decoding, and parallelism (TP/EP/pipeline), plus Python/PyTorch and Kubernetes operations at scale.

Experience:8+ yearsML infrastructureModel servingLLMDistributed systemsKubernetes

Skills:Technical leadershipHands-on problem-solvingIncident responseAbility to set technical directionCross-functional collaboration

Tech Stack:VLLMSGLangTensorRT-LLMPythonPyTorchKubernetesContinuous batchingKV-cache managementQuantizationSpeculative decodingTPEPPipelineAutoscalingCachingObservabilityDeployment pipelinesRequest routingSchedulingQueueing

Company Brief

Luma

Develops AI-powered tools for capturing, editing, and rendering high-quality 3D scenes from photos and videos, enabling creators to generate photorealistic 3D assets and spatial experiences.

Industry: AR/VR & Spatial Computing

Website

Loading job details...

Preparing the role view and application actions.

Luma

2 weeks ago

Tech Lead Manager, Inference

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 2 hours agoStatus: Live

Job Summary

Location: Redwood City

Workplace: Hybrid

Employment Type: Full time

Job Function: Administration & Executive Assistance

Seniority: Manager level

Key Responsibilities

•Spend at least half your time hands-on designing, building, and debugging core serving stack components and production incidents.
•Lead, grow, and develop the inference engineering team, including hiring, coaching, on-call ownership, incident response, capacity planning, and postmortems.
•Set the technical roadmap for model serving engines and the end-to-end serving platform, including routing/scheduling, autoscaling, caching, observability, and deployment pipelines.
•Own platform SLOs and economics across latency/availability targets, GPU utilization, and cost per generation for models in production.
•Partner with research to ship new model architectures into production rapidly and integrate serving into online reinforcement learning and evaluation loops.

Key Requirements

•8+ years engineering experience in large-scale distributed systems or ML infrastructure, including building/operating model-serving or inference platforms in production.
•Experience operating inference platforms at scale, including fleets of thousands of GPUs across multiple clusters or clouds.
•Technical leadership experience managing or leading engineers through rapid growth while keeping significant hands-on time.
•Deep, practical expertise in LLM and foundation-model serving engines such as vLLM, SGLang, or TensorRT-LLM.
•Strong command of serving performance practices including continuous batching, KV-cache management, quantization, speculative decoding, and parallelism (TP/EP/pipeline), plus Python/PyTorch and Kubernetes operations at scale.

Experience:8+ yearsML infrastructureModel servingLLMDistributed systemsKubernetes

Skills:Technical leadershipHands-on problem-solvingIncident responseAbility to set technical directionCross-functional collaboration

Company Brief

Luma

Develops AI-powered tools for capturing, editing, and rendering high-quality 3D scenes from photos and videos, enabling creators to generate photorealistic 3D assets and spatial experiences.

Industry: AR/VR & Spatial Computing

Website