Applied Research - Evals & Data

Prime Intellect
New York
Workplace: RemoteFull timeFunction: Research & Scientific (R&D)Skills: ["Communication","Collaboration","Problem-solving","Cross-functional"]

Customer-facing ML engineering role focused on evaluating and aligning large models through post-training methods, RL workflows, and applied data. You’ll design agent capabilities, build scalable evaluation pipelines, and translate customer insights into research directions, working across RL, evaluation, and deployment in a frontier open superintelligence infrastructure.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Prime Intellect
Prime Intellect
10 months ago

Applied Research - Evals & Data

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Customer-facing ML engineering role focused on evaluating and aligning large models through post-training methods, RL workflows, and applied data. You’ll design agent capabilities, build scalable evaluation pipelines, and translate customer insights into research directions, working across RL, evaluation, and deployment in a frontier open superintelligence infrastructure.
Location: New York
Workplace: Remote
Employment Type: Full time
Job Function: Research & Scientific (R&D)

Key Responsibilities

  • •Design and iterate on next-generation AI agents for automation, workflow orchestration, and decision-making.
  • •Extend and integrate with agent frameworks to support evolving feature requests and performance requirements.
  • •Architect and maintain distributed training and inference pipelines, ensuring scalability and cost efficiency.
  • •Develop observability and monitoring (Prometheus, Grafana, tracing) to ensure reliability and performance in production deployments.
  • •Collaborate with customers and research teams to translate insights into roadmap and research direction.

Pay and Benefits

Perks:Remote WorkVisa SponsorshipRelocation Support

Key Requirements

  • •Strong background in machine learning engineering, with experience in post-training, RL, or large-scale model alignment.
  • •Experience with applied data workflows and evaluation frameworks for large models or agents (e.g., SWE-Bench, HELM, EvalFlow, internal eval pipelines).
  • •Deep expertise in distributed training/inference frameworks (e.g., vLLM, sglang, Ray, Accelerate).
  • •Experience deploying containerized systems at scale (Docker, Kubernetes, Terraform).
  • •Track record of research contributions (publications, open-source contributions, benchmarks) in ML/RL.
Experience:Open sourceMachine learningReinforcement learningAIAgentsDistributed systems
Skills:CommunicationCollaborationProblem-solvingCross-functional
Tech Stack:DockerKubernetesTerraformPrometheusGrafanaRayAccelerateVLLMSglang

Company Brief

Prime Intellect
Builds a decentralized, open compute and training platform that enables distributed training and collective ownership of AI models, aggregating global GPU resources and offering tools for agentic RL and model evaluation.
Industry: AI & Machine Learning
Company Size: Small (11 to 50 employees)
Growth: Early Stage Startup
Funding: Seed
Headquarters: San Francisco, United States
Founded: 2024
WebsiteLinkedIn