Agent Evaluation & Evolution Machine Learning Engineer Graduate (AML-Ark-US) - 2027 Start

ByteDance
San Jose
Workplace: OnsiteFull timeFunction: Data Science & Machine LearningEducation: bachelorsSkills: []

Build evaluation and evolution systems for LLM-based agents, including task success measurement, tool-use analysis, reasoning-quality checks, and reliability testing. Create benchmarks and automated judging pipelines that combine rule-based checks, model-based judging, and human review. Analyze agent execution traces and user feedback to uncover failure patterns and translate them into production system improvements, collaborating across research, platform, and product teams.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
ByteDance
ByteDance
1 day ago

Agent Evaluation & Evolution Machine Learning Engineer Graduate (AML-Ark-US) - 2027 Start

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 7 hours agoStatus: Live

Job Summary

Build evaluation and evolution systems for LLM-based agents, including task success measurement, tool-use analysis, reasoning-quality checks, and reliability testing. Create benchmarks and automated judging pipelines that combine rule-based checks, model-based judging, and human review. Analyze agent execution traces and user feedback to uncover failure patterns and translate them into production system improvements, collaborating across research, platform, and product teams.
Location: San Jose
Workplace: Onsite
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Graduate level

Key Responsibilities

  • •Design evaluation systems for LLM-based agents across task success, tool use, reasoning quality, and reliability.
  • •Build benchmarks and automated judging pipelines combining rule-based checks, model-based judging, and human review.
  • •Analyze agent execution traces and user feedback to identify failure patterns and translate them into concrete system improvements.
  • •Support closed-loop learning from experience to capability in collaboration with research, platform, and product teams.
  • •Operate and improve Large Language Model service platform systems and related log analytics pipelines to drive continuous, data-driven feedback loops.

Key Requirements

  • •Currently completing or recently completed a Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, or a related field.
  • •Strong foundation in machine learning and deep learning.
  • •Hands-on experience with LLM-based systems (agents, tool calling, retrieval, multi-agent systems) via research, internships, or projects.
  • •Strong Python skills and experience with mainstream ML or agent evaluation frameworks.
  • •Evidence of research or engineering ability through publications, substantial projects, internships, or open-source work.
Experience:Open sourceResearchLLMAI agents
Education:Bachelor's in Computer Science, Artificial Intelligence, Machine Learning, Data Science (or related field)
Tech Stack:PythonLLMMachine learningDeep learning

Company Brief

ByteDance
Develops consumer internet and content platforms, including TikTok and other apps for short-form video, news, and entertainment. It also builds advertising, commerce, and creator tools that connect audiences, brands, and publishers across global markets.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Established Company
Headquarters: Beijing, China
Founded: 2012
WebsiteLinkedIn