Principal AI Engineer - Context - Agents and Context

Elastic
United Kingdom
Workplace: HybridFull timeFunction: Data Science & Machine LearningExperience: 10+ yearsSkills: ["Eval-driven product improvement","Technical leadership","Mentoring","Communication trade-off clarity","Pragmatic collaboration"]

Build and run the end-to-end improvement loop for an agent knowledge layer in Elasticsearch. Own production behavior for context extraction, retrieval tools, and memory—using offline evaluations and real customer telemetry to identify failure modes, ship safe prompt/skill updates, and prove fixes. Design evaluation and telemetry pipelines, partner with data scientists on quality/latency/cost gating, and mentor engineers on eval-driven development.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Elastic
Elastic
23 hours ago

Principal AI Engineer - Context - Agents and Context

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 8 hours agoStatus: Live

Job Summary

Build and run the end-to-end improvement loop for an agent knowledge layer in Elasticsearch. Own production behavior for context extraction, retrieval tools, and memory—using offline evaluations and real customer telemetry to identify failure modes, ship safe prompt/skill updates, and prove fixes. Design evaluation and telemetry pipelines, partner with data scientists on quality/latency/cost gating, and mentor engineers on eval-driven development.
Location: United Kingdom
Workplace: Hybrid
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Sr. Manager level

Key Responsibilities

  • •Own the production improvement loop for the Context Engine, analyzing how extraction automations, retrieval tools, and memory behave using evaluations and telemetry.
  • •Define safe iteration practices for agents and skills, including versioning/rollouts, regression coverage, staged/shadow evaluation, and guardrails.
  • •Design telemetry for agent traces, tool calls, and knowledge retrieval, and ensure it feeds evaluation dashboards and the feedback loop.
  • •Partner with data science to create evaluation strategies, including golden datasets and quality/latency/cost evaluators.
  • •Raise engineering quality by reviewing designs/PRs, mentoring eval-driven development, and proposing technical roadmap shaping changes.

Pay and Benefits

Perks:Health InsuranceParental LeavePaid Leave

Key Requirements

  • •10+ years of software engineering experience shipping and operating AI-driven products on real production traffic, ideally with public APIs and evolving data models.
  • •Track record of eval-driven improvement: diagnose issues from traces/user feedback, design evaluations, ship fixes, and measure outcomes.
  • •Experience building agents with state and memory, iterating prompts/skills/tool behavior safely in production.
  • •Familiarity with MCP, including exposing public MCP servers and tools.
  • •Strong backend skills in either Python or TypeScript, including APIs, stateful workflows, data pipelines, and production services.
Experience:10+ yearsAILLMElasticsearch
Skills:Eval-driven product improvementTechnical leadershipMentoringCommunication trade-off clarityPragmatic collaboration
Languages:English
Tech Stack:TypeScriptPythonElasticsearchMCPLangChainClaude CodeLangGraphCrewAIClaude Agent SDK

Company Brief

Elastic
Builds the Elastic Stack (Elasticsearch, Kibana, Beats, Logstash) and provides search, observability, and security solutions that enable organizations to search, analyze, and protect data in real time across applications, infrastructure, and enterprises.
Industry: Data Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Amsterdam, Netherlands
Founded: 2012
WebsiteLinkedIn