9 months ago

Member of Technical Staff, Evals & Post-Training Product

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 7 hours agoStatus: Live

Job Summary

Build internal evaluation workflows and user-facing fine-tuning experiences that connect model evaluation and post-training into a continuous improvement loop. Work across APIs, SDKs, backend systems, and the web app to help users author evals, interpret results, and iterate quickly. Partner with customers and internal teams to triage issues and turn recurring pain points into productized capabilities across SFT, RFT, and related improvements.

Location: San Mateo

Workplace: Onsite

Employment Type: Full time

Job Function: Education & Training

Seniority: Mid level

Key Responsibilities

•Design and scale internal evaluation tooling to measure model quality and compare model changes to inform post-training decisions.
•Build and improve user-facing post-training workflows, including fine-tuning experiences across SFT, RFT, and related model-improvement capabilities.
•Partner with customers and internal stakeholders to understand evaluation and fine-tuning needs, support high-priority engagements, and triage issues.
•Convert bespoke evaluation and fine-tuning workflows into reusable, productized solutions.
•Collaborate across APIs, SDKs, backend systems, and web app surfaces to help users author evals, interpret results, and iterate quickly.

Pay and Benefits

Salary: USD 175,000 - 220,000 annually

Equity and Bonus:Equity

Key Requirements

•1 - 7 years of software engineering experience (multiple levels).
•Hands-on experience with LLM evaluations and/or post-training methods, including using eval results to guide model improvement.
•Ability to work across backend systems and developer-facing product surfaces; comfortable shipping full-stack features when needed.
•Understanding of the GenAI lifecycle from prompting and dataset curation through fine-tuning and productionizing agents.
•User-centric mindset: talk to users, triage GitHub issues for open-source projects, and build products from scratch.

Experience:1-7 yearsAI infrastructureDeveloper toolsOpen sourceGenAIEnterprise AI

Skills:Cross-functional collaborationUser focusProblem solvingProduct thinking

Tech Stack:LLM evaluationsPost-trainingEval Protocol SDKAPIsSDKsWeb appFull-stackSFTRFTGitHubGPUInference optimizationPyTorch

Company Brief

Fireworks AI

Develops AI-driven tools to generate and optimize visual marketing content for brands and creators, automating production of short-form videos and multimedia assets for social platforms to improve engagement and scale creative workflows.

Industry: SaaS

Website

Loading job details...

Preparing the role view and application actions.

Fireworks AI

9 months ago

Member of Technical Staff, Evals & Post-Training Product

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 7 hours agoStatus: Live

Job Summary

Location: San Mateo

Workplace: Onsite

Employment Type: Full time

Job Function: Education & Training

Seniority: Mid level

Key Responsibilities

•Design and scale internal evaluation tooling to measure model quality and compare model changes to inform post-training decisions.
•Build and improve user-facing post-training workflows, including fine-tuning experiences across SFT, RFT, and related model-improvement capabilities.
•Partner with customers and internal stakeholders to understand evaluation and fine-tuning needs, support high-priority engagements, and triage issues.
•Convert bespoke evaluation and fine-tuning workflows into reusable, productized solutions.
•Collaborate across APIs, SDKs, backend systems, and web app surfaces to help users author evals, interpret results, and iterate quickly.

Pay and Benefits

Salary: USD 175,000 - 220,000 annually

Equity and Bonus:Equity

Key Requirements

•1 - 7 years of software engineering experience (multiple levels).
•Hands-on experience with LLM evaluations and/or post-training methods, including using eval results to guide model improvement.
•Ability to work across backend systems and developer-facing product surfaces; comfortable shipping full-stack features when needed.
•Understanding of the GenAI lifecycle from prompting and dataset curation through fine-tuning and productionizing agents.
•User-centric mindset: talk to users, triage GitHub issues for open-source projects, and build products from scratch.

Experience:1-7 yearsAI infrastructureDeveloper toolsOpen sourceGenAIEnterprise AI

Skills:Cross-functional collaborationUser focusProblem solvingProduct thinking

Tech Stack:LLM evaluationsPost-trainingEval Protocol SDKAPIsSDKsWeb appFull-stackSFTRFTGitHubGPUInference optimizationPyTorch

Company Brief

Fireworks AI

Industry: SaaS

Website