Senior Machine Learning Engineer, AI-Native Evaluation & Search

Atlassian
San Francisco, Mountain View, Seattle
Workplace: RemoteFull timeUSD 171,063 - 269,075 annuallyFunction: Data Science & Machine LearningExperience: 4+ yearsEducation: mastersSkills: ["Communication","Collaboration","Problem-solving","Mentoring","Continuous improvement"]

Develop and productionize machine learning systems that power search, retrieval, ranking, conversational experiences, and AI agents. Own projects across the full ML lifecycle—from problem definition and data development through experimentation, evaluation, deployment, and continuous improvement. Make evaluation a first-class practice by defining quality, building evaluation datasets and benchmarks, analyzing model quality, and using results to guide product and engineering decisions. Collaborate across product, engineering, data science, and platform teams.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Atlassian
Atlassian
1 day ago

Senior Machine Learning Engineer, AI-Native Evaluation & Search

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 8 hours agoStatus: Live

Job Summary

Develop and productionize machine learning systems that power search, retrieval, ranking, conversational experiences, and AI agents. Own projects across the full ML lifecycle—from problem definition and data development through experimentation, evaluation, deployment, and continuous improvement. Make evaluation a first-class practice by defining quality, building evaluation datasets and benchmarks, analyzing model quality, and using results to guide product and engineering decisions. Collaborate across product, engineering, data science, and platform teams.
Location: San Francisco, Mountain View, Seattle
Workplace: Remote
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Build and productionize machine learning models and AI systems for Search & Intelligence products.
  • •Own ML lifecycle projects from concept through deployment, monitoring, evaluation, and iteration.
  • •Develop AI-native experiences across search, retrieval, ranking, recommendations, conversational systems, and agentic workflows.
  • •Create evaluation-driven development processes, including quality metrics, test cases, benchmarks, and acceptance criteria.
  • •Conduct offline and online evaluations (human and model-based), perform error analysis, and communicate results to technical and non-technical stakeholders.

Pay and Benefits

Salary: USD 171,063 - 269,075 annually
Equity and Bonus:Equity

Key Requirements

  • •Bachelor’s or Master’s degree in Computer Science, Machine Learning, Data Science, or a related field (or equivalent experience).
  • •At least 4 years of experience developing and deploying machine learning or AI systems in production.
  • •Strong Python skills and experience with Java, Kotlin, TypeScript, or another production language.
  • •Experience with SQL and large-scale data processing technologies such as Spark.
  • •Experience building, evaluating, deploying, and scaling models with large datasets, including evaluation strategy design and model-quality analysis.
Experience:4+ yearsAI-native productsRAGConversational assistantsLLMsAgentic workflowsSearch relevanceRecommendationsPersonalizationModel servingResponsible AI
Education:Master's in Computer Science, Machine Learning, Data Science, or related field
Skills:CommunicationCollaborationProblem-solvingMentoringContinuous improvement
Tech Stack:PythonJavaKotlinTypeScriptSQLSparkAWSDatabricksEmbeddingsNLPDeep learningLarge language modelsRAGInformation retrievalRankingRecommendationsLLM-as-a-judge

Company Brief

Atlassian
Atlassian develops collaboration and workflow software—Jira, Confluence, Trello, Loom and more—used by teams for project management, software development, and IT service management across enterprises worldwide.
Industry: Developer Tools
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Sydney, Australia
Founded: 2002
Glassdoor
Glassdoor: 3.1
WebsiteLinkedInGlassdoor