AI Engineer, Evaluation
Distyl
San Francisco
Workplace: HybridFull time150,000 - 250,000 annuallyFunction: Data Science & Machine LearningExperience: 2+ yearsSkills: ["Python","Testing","Regression","Pipelines","LLMs","Evaluation pipelines","Git","Code quality"]Design and implement evaluation frameworks for AI systems in production, building pipelines and graders that translate human judgments into measurable test cases. Collaborate across engineers and domain experts to ensure evaluation drives system iteration, prompts, model selection, and release readiness in a hybrid, production-focused environment.
