Member of Technical Staff - Research, Inference

Modal
New York, San Francisco
Workplace: OnsiteFull timeUSD 150,000 - 350,000 annuallyFunction: Research & Scientific (R&D)Skills: ["Independent ownership","Collaboration","Research execution"]

Build hands-on inference research to improve cost per token and tail latency for real customer LLM workloads. Own end-to-end research bets spanning speculative decoding, disaggregated prefill/decode, quantization, KV-cache and memory management, and autoscaling for spiky serverless traffic. Train and iterate speculators from production traffic, partner with customers and Forward Deployed Engineers to deploy and tune models, and collaborate with external labs to turn frontier serving techniques into products.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Modal
Modal
2 months ago

Member of Technical Staff - Research, Inference

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 9 hours agoStatus: Live

Job Summary

Build hands-on inference research to improve cost per token and tail latency for real customer LLM workloads. Own end-to-end research bets spanning speculative decoding, disaggregated prefill/decode, quantization, KV-cache and memory management, and autoscaling for spiky serverless traffic. Train and iterate speculators from production traffic, partner with customers and Forward Deployed Engineers to deploy and tune models, and collaborate with external labs to turn frontier serving techniques into products.
Location: New York, San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Research & Scientific (R&D)
Seniority: Mid level

Key Responsibilities

  • •Own end-to-end inference research bets, including speculative decoding, disaggregated prefill/decode, quantization, KV-cache and memory management, and autoscaling.
  • •Train custom speculators against real production traffic and feed results back into target models using acceptance length as the metric.
  • •Work directly with customers (with Forward Deployed Engineers) to deploy and tune models, then bring learnings back into research.
  • •Carry and expand collaborations with outside research labs, including ZLab and SGLang workstreams.
  • •Partner with engineering to turn frontier serving techniques into products, including primitives, observability, and next-generation inference engine ideas.

Pay and Benefits

Salary: USD 150,000 - 350,000 annually
Equity and Bonus:Equity
Perks:Equity

Key Requirements

  • •Research-leaning or systems background in LLM inference, with work you can point to.
  • •Fluency in the LLM serving stack, from kernels and quantization to schedulers and autoscaling.
  • •A record of shipping research or systems that others build on, in a lab or in industry.
  • •Ability to independently take a research bet from idea to result, working in the open with the team.
  • •Ability to work in-person in the NYC or San Francisco office.
Experience:LLM inferenceResearch systems
Skills:Independent ownershipCollaborationResearch execution
Tech Stack:FP8INT4KV-cacheSpeculative decodingSGLangFlash Attention 4Python

Company Brief

Modal
Provides a serverless, high-performance cloud platform for AI, ML, and data workloads — offering instant autoscaling, elastic GPU access, and developer-first tooling to run inference, training, and batch jobs at scale.
Industry: Cloud Computing
Company Size: Small (11 to 50 employees)
Growth: Growth Stage Startup
Valuation: Unicorn (USD 1B+)
Funding: Series B
Headquarters: New York City, United States
Founded: 2021
WebsiteLinkedIn