Research Engineer, LangSmith Engine

LangChain
San Francisco, New York
Workplace: OnsiteFull timeFunction: Research & Scientific (R&D)Experience: 4+ yearsEducation: mastersSkills: ["Technical leadership","Research judgment","Communication"]

Build and maintain benchmarks and evaluations for LangSmith Engine agents, then design experiments to improve real-world performance across models, prompting, context, tools, orchestration, and strategies. Study production agent failures, implement post-training and fine-tuning approaches when they improve capability, quality, or cost, and turn successful research into production improvements with measurable impact on reliability, latency, scalability, and regressions.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
LangChain
LangChain
4 days ago

Research Engineer, LangSmith Engine

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 1 hour agoStatus: Live

Job Summary

Build and maintain benchmarks and evaluations for LangSmith Engine agents, then design experiments to improve real-world performance across models, prompting, context, tools, orchestration, and strategies. Study production agent failures, implement post-training and fine-tuning approaches when they improve capability, quality, or cost, and turn successful research into production improvements with measurable impact on reliability, latency, scalability, and regressions.
Location: San Francisco, New York
Workplace: Onsite
Employment Type: Full time
Job Function: Research & Scientific (R&D)
Seniority: Mid level

Key Responsibilities

  • •Build and maintain benchmarks and evaluations that measure Engine agent quality and efficiency on real-world tasks.
  • •Design and run experiments to improve agent performance across models, prompting, context, tools, orchestration, and agent strategies.
  • •Explore and implement post-training and fine-tuning techniques when they improve capabilities, quality, or cost.
  • •Turn successful experiments into production improvements in partnership with engineers and researchers to prevent regressions.
  • •Help define the ML roadmap and technical direction for improving Engine agents and mentor other engineers through technical leadership.

Pay and Benefits

Equity and Bonus:Equity
Perks:Health InsuranceDentalVision401kLife InsurancePaid LeaveMeal Allowance

Key Requirements

  • •4+ years of experience in ML/AI research (or closely related field).
  • •Master’s or Ph.D. in a relevant scientific field.
  • •Hands-on experience working with LLMs and AI agents, including analyzing model behavior and improving real-world performance.
  • •Strong experience designing benchmarks, evaluations, and experiments for AI/ML systems to measure whether changes improve agents.
  • •Strong software engineering skills with a track record moving research prototypes into measurable production impact.
Experience:4+ yearsML/AI researchLLMsAI agents
Education:Master's
Skills:Technical leadershipResearch judgmentCommunication
Tech Stack:LLMsAI agentsFine-tuningPost-trainingReinforcement learningRLHFRLAIF

Company Brief

LangChain
Provides an open-source framework and commercial platform (LangSmith/LangGraph) for building, testing, monitoring, and deploying LLM-powered applications and agentic AI for developers and enterprises.
Industry: Developer Tools
Company Size: Medium (51 to 250 employees)
Revenue: USD 10M to 25M
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series B
Headquarters: San Francisco, United States
Founded: 2022
WebsiteLinkedIn