Member of Technical Staff (Evals)

Nitrode
San Francisco
Workplace: OnsiteFull timeUSD 120,000 - 250,000 annuallyFunction: Data Science & Machine LearningSkills: []

Build evaluation frameworks and benchmarks to measure AI performance in game development. Work on Nitrode’s upcoming open-source suite of benchmarks, including GameEngineBench, which evaluates agentic coding performance on C++ game engine tasks in real runtime environments. Collaborate with the small team of founders while sharing your own work and research to help advance robust ways of judging models in complex game settings.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Nitrode
Nitrode
2 months ago

Member of Technical Staff (Evals)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 19 hours agoStatus: Live

Job Summary

Build evaluation frameworks and benchmarks to measure AI performance in game development. Work on Nitrode’s upcoming open-source suite of benchmarks, including GameEngineBench, which evaluates agentic coding performance on C++ game engine tasks in real runtime environments. Collaborate with the small team of founders while sharing your own work and research to help advance robust ways of judging models in complex game settings.
Location: San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Sr. Manager level

Key Responsibilities

  • •Help develop robust evaluation frameworks for AI models used in game development.
  • •Contribute to an open-source suite of game-development benchmarks over the coming months.
  • •Work on GameEngineBench for agentic coding performance in C++ game engine tasks.
  • •Evaluate model performance in real runtime environments.
  • •Share your built projects and research with the team.

Pay and Benefits

Salary: USD 120,000 - 250,000 annually
Equity and Bonus:Equity

Key Requirements

  • •Send us what you’ve built and any cool research you’ve done.
  • •Machine learning and reinforcement learning experience.
  • •Experience with Python and Torch/PyTorch.
  • •Experience with Amazon Web Services (AWS).
  • •C++ and/or game development experience is a plus.
Tech Stack:Amazon Web Services (AWS)C++JavaPythonTorch/PyTorchMachine LearningReinforcement learning (RL)

Company Brief

Nitrode
Nitrode provides developer-focused tools and services to help teams build, test, and deploy web applications with emphasis on performance, developer experience, and cloud integrations.
Industry: Developer Tools
Website