Research Scientist

OpenRouter
United States
Workplace: RemoteFull timeFunction: Data Science & Machine LearningEducation: mastersSkills: ["Curiosity","Self-directed","Scientific rigor","Communication","Collaboration"]

Conduct deep, original research to advance how large language models are evaluated, compared, and routed across providers. Own a research agenda covering LLM evaluation, model quality, routing optimization, and AI usage patterns, designing novel benchmarks from real-world generation data. Run large-scale empirical studies, develop statistical and mathematical foundations for routing systems, and translate findings into practical improvements to the platform while collaborating with product and engineering.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
OpenRouter
OpenRouter
2 months ago

Research Scientist

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 11 hours agoStatus: Live

Job Summary

Conduct deep, original research to advance how large language models are evaluated, compared, and routed across providers. Own a research agenda covering LLM evaluation, model quality, routing optimization, and AI usage patterns, designing novel benchmarks from real-world generation data. Run large-scale empirical studies, develop statistical and mathematical foundations for routing systems, and translate findings into practical improvements to the platform while collaborating with product and engineering.
Location: United States
Workplace: Remote
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Own and pursue a research agenda focused on LLM evaluation, model quality, routing optimization, and AI usage patterns.
  • •Design novel evaluation frameworks and benchmarks using real-world generation data to capture performance across tasks and contexts.
  • •Conduct large-scale empirical studies on LLM behavior across providers and over time.
  • •Develop the statistical and mathematical foundations behind routing systems, including models and heuristics for provider and model selection.
  • •Translate research findings into platform improvements while collaborating with product and engineering teams, and work with external researchers and model providers.

Key Requirements

  • •MS or PhD in a quantitative field (machine learning, statistics, computer science, mathematics, or computational linguistics).
  • •Track record of original research via first-author publications, significant open-source contributions, or equivalent industry impact.
  • •Deep expertise in statistics, experimental design, and causal inference to ensure validity, bias control, and generalizability.
  • •Strong programming skills in Python for building data pipelines and running large-scale experiments.
  • •Proficiency in SQL and hands-on experience with modern ML/NLP techniques including LLM evaluation, fine-tuning, embeddings, classification, or RLHF.
Experience:AIMachine learningNLP
Education:Master's
Skills:CuriositySelf-directedScientific rigorCommunicationCollaboration
Tech Stack:PythonSQLClickHouseBigQueryMachine learningMLNLPLLM evaluationFine-tuningEmbeddingsClassificationReinforcement learning from human feedbackRLHFCausal inference

Company Brief

OpenRouter
Provides a unified, standards-compatible API and marketplace to route requests to 400+ large language models and cloud hosts, optimizing for cost, performance, and reliability for developers and enterprises.
Industry: API Platforms
Company Size: Micro (1 to 10 employees)
Revenue: USD 1M to 5M
Growth: Scaleup
Valuation: USD 250M to 500M
Funding: Series A
Headquarters: New York, United States
Founded: 2023
WebsiteLinkedIn