Member of Technical Staff, Evals & Post-Training Product

Fireworks AI
San Mateo
Workplace: OnsiteFull timeUSD 175,000 - 220,000 annuallyFunction: Education & TrainingExperience: 1-7 yearsSkills: ["Cross-functional collaboration","User focus","Problem solving","Product thinking"]

Build internal evaluation workflows and user-facing fine-tuning experiences that connect model evaluation and post-training into a continuous improvement loop. Work across APIs, SDKs, backend systems, and the web app to help users author evals, interpret results, and iterate quickly. Partner with customers and internal teams to triage issues and turn recurring pain points into productized capabilities across SFT, RFT, and related improvements.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Fireworks AI
Fireworks AI
9 months ago

Member of Technical Staff, Evals & Post-Training Product

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 4 hours agoStatus: Live
Reposted: similar role first listed 9 months ago

Job Summary

Build internal evaluation workflows and user-facing fine-tuning experiences that connect model evaluation and post-training into a continuous improvement loop. Work across APIs, SDKs, backend systems, and the web app to help users author evals, interpret results, and iterate quickly. Partner with customers and internal teams to triage issues and turn recurring pain points into productized capabilities across SFT, RFT, and related improvements.
Location: San Mateo
Workplace: Onsite
Employment Type: Full time
Job Function: Education & Training
Seniority: Mid level

Key Responsibilities

  • •Design and scale internal evaluation tooling to measure model quality and compare model changes to inform post-training decisions.
  • •Build and improve user-facing post-training workflows, including fine-tuning experiences across SFT, RFT, and related model-improvement capabilities.
  • •Partner with customers and internal stakeholders to understand evaluation and fine-tuning needs, support high-priority engagements, and triage issues.
  • •Convert bespoke evaluation and fine-tuning workflows into reusable, productized solutions.
  • •Collaborate across APIs, SDKs, backend systems, and web app surfaces to help users author evals, interpret results, and iterate quickly.

Pay and Benefits

Salary: USD 175,000 - 220,000 annually
Equity and Bonus:Equity

Key Requirements

  • •1 - 7 years of software engineering experience (multiple levels).
  • •Hands-on experience with LLM evaluations and/or post-training methods, including using eval results to guide model improvement.
  • •Ability to work across backend systems and developer-facing product surfaces; comfortable shipping full-stack features when needed.
  • •Understanding of the GenAI lifecycle from prompting and dataset curation through fine-tuning and productionizing agents.
  • •User-centric mindset: talk to users, triage GitHub issues for open-source projects, and build products from scratch.
Experience:1-7 yearsAI infrastructureDeveloper toolsOpen sourceGenAIEnterprise AI
Skills:Cross-functional collaborationUser focusProblem solvingProduct thinking
Tech Stack:LLM evaluationsPost-trainingEval Protocol SDKAPIsSDKsWeb appFull-stackSFTRFTGitHubGPUInference optimizationPyTorch

Company Brief

Fireworks AI
Develops AI-driven tools to generate and optimize visual marketing content for brands and creators, automating production of short-form videos and multimedia assets for social platforms to improve engagement and scale creative workflows.
Industry: SaaS
Website