Agent Evaluation & Evolution Researcher Graduate (AML-Ark-US) - 2027 Start (PhD)

ByteDance
Seattle
Workplace: OnsiteFull timeFunction: Research & Scientific (R&D)Education: phdSkills: ["Research ability","Problem-solving","Analysis"]

Join the Applied Machine Learning Ark team to design evaluation systems for LLM-based agents, create benchmarks and automated judging pipelines, and analyze agent execution traces and user feedback to pinpoint failure modes. You’ll help build a closed loop from experience to improved capabilities, collaborating with research, platform, and product teams to bring agent evaluation and evolution methods into production at scale.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
ByteDance
ByteDance
2 days ago

Agent Evaluation & Evolution Researcher Graduate (AML-Ark-US) - 2027 Start (PhD)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 14 hours agoStatus: Live

Job Summary

Join the Applied Machine Learning Ark team to design evaluation systems for LLM-based agents, create benchmarks and automated judging pipelines, and analyze agent execution traces and user feedback to pinpoint failure modes. You’ll help build a closed loop from experience to improved capabilities, collaborating with research, platform, and product teams to bring agent evaluation and evolution methods into production at scale.
Location: Seattle
Workplace: Onsite
Employment Type: Full time
Job Function: Research & Scientific (R&D)
Seniority: Graduate level

Key Responsibilities

  • •Design evaluation systems for LLM-based agents, including task success, tool use, reasoning quality, and reliability.
  • •Build benchmarks and automated judging pipelines combining rule-based checks, model-based judging, and human review.
  • •Analyze agent execution traces and user feedback to identify failure patterns and drive concrete system improvements.
  • •Support the closed loop from experience to capability and collaborate with research, platform, and product teams to bring methods into production.

Key Requirements

  • •Completing or recently completed a PhD in Computer Science, Artificial Intelligence, Machine Learning, Data Science, or a related field.
  • •Solid foundation in machine learning and deep learning.
  • •Hands-on experience with LLM-based systems (e.g., agents, tool calling, retrieval, multi-agent systems) via research, internships, or projects.
  • •Strong Python skills and experience with a mainstream ML or agent evaluation framework.
  • •Demonstrated research or engineering ability through publications, substantial projects, internships, or open-source work.
Experience:Open sourceLLM-based systemsAI agents
Education:PhD / Doctorate
Skills:Research abilityProblem-solvingAnalysis
Tech Stack:PythonLarge Language Models (LLMs)Machine learningDeep learning

Company Brief

ByteDance
Develops consumer internet and content platforms, including TikTok and other apps for short-form video, news, and entertainment. It also builds advertising, commerce, and creator tools that connect audiences, brands, and publishers across global markets.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Established Company
Headquarters: Beijing, China
Founded: 2012
WebsiteLinkedIn