Senior Machine Learning Engineer, Applied AI Quality

Block
San Francisco
Workplace: RemoteFull timeUSD 194,500 - 343,100 annuallyFunction: Data Science & Machine LearningExperience: 5+ yearsSkills: ["Systems thinking","Architectural judgment","Cross-functional leadership","Communication","Mentorship"]

Build and lead next-generation AI-driven product quality evaluation systems. You’ll define technical direction and architecture for scalable frameworks that use LLMs, AI agents, and behavioral signals to detect regressions, surface emerging issues, and generate actionable insights. Working across engineering, product, platform, and leadership, you’ll operationalize quality measurement across complex product surfaces while mentoring engineers and influencing standards for trustworthy AI systems.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Block
Block
1 month ago

Senior Machine Learning Engineer, Applied AI Quality

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 1 hour agoStatus: Live

Job Summary

Build and lead next-generation AI-driven product quality evaluation systems. You’ll define technical direction and architecture for scalable frameworks that use LLMs, AI agents, and behavioral signals to detect regressions, surface emerging issues, and generate actionable insights. Working across engineering, product, platform, and leadership, you’ll operationalize quality measurement across complex product surfaces while mentoring engineers and influencing standards for trustworthy AI systems.
Location: San Francisco
Workplace: Remote
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Lead the technical strategy and architecture for AI-driven quality and evaluation systems used across products and teams.
  • •Develop scalable systems using LLMs, agents, and behavioral signals to evaluate quality, detect regressions, and generate product insights.
  • •Define long-term approaches for evaluation, measurement, and quality intelligence across complex product surfaces.
  • •Translate ambiguous organizational needs into technical direction, roadmap priorities, and platform capabilities.
  • •Mentor engineers through technical leadership, design reviews, and systems thinking while influencing engineering standards and best practices.

Pay and Benefits

Salary: USD 194,500 - 343,100 annually
Perks:Remote WorkHealth InsurancePaid Leave

Key Requirements

  • •5+ years of experience in software engineering, machine learning engineering, or applied AI.
  • •Deep experience designing and shipping large-scale AI/ML systems in production environments.
  • •Strong expertise with LLMs, agents, evaluation systems, retrieval architectures, and modern AI infrastructure.
  • •Proven ability to lead ambiguous, high-impact technical initiatives from concept through adoption across multiple teams.
  • •Excellent communication and cross-functional leadership skills to align engineering, product, and organizational priorities.
Experience:5+ years
Skills:Systems thinkingArchitectural judgmentCross-functional leadershipCommunicationMentorship
Tech Stack:LLMsAI agentsEvaluation systemsRetrieval architecturesAI infrastructure

Company Brief

Block
Builds financial services and commerce tools to empower businesses and individuals, operating products such as Square, Cash App, Spiral (Bitcoin initiatives), TIDAL, and developer-focused payment and banking solutions.
Industry: Fintech Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: San Francisco, United States
Founded: 2009
WebsiteLinkedIn