Sr. Manager, Test Automation & AI Evaluation

WebMD
Newark
Workplace: OnsiteFull timeUSD 98,000 - 135,000 annuallyFunction: QA, Test & Release EngineeringSkills: ["Leadership","Mentorship","Communication","Quality risk translation","Governance"]

Lead an automation-first quality strategy for the Ignite Growth Platform, removing manual testing bottlenecks as AI-assisted development accelerates. Own modern test automation frameworks and end-to-end coverage with deep expertise in Playwright. Build LLM/agentic evaluation frameworks (correctness, hallucinations, prompt drift, behavioral regression) and apply AI to QA via self-healing automation and intelligent defect triage. Provide direct leadership to the quality assurance team and set organization-wide test strategy and release-readiness governance.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
WebMD
WebMD
2 months ago

Sr. Manager, Test Automation & AI Evaluation

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 13 hours agoStatus: Live

Job Summary

Lead an automation-first quality strategy for the Ignite Growth Platform, removing manual testing bottlenecks as AI-assisted development accelerates. Own modern test automation frameworks and end-to-end coverage with deep expertise in Playwright. Build LLM/agentic evaluation frameworks (correctness, hallucinations, prompt drift, behavioral regression) and apply AI to QA via self-healing automation and intelligent defect triage. Provide direct leadership to the quality assurance team and set organization-wide test strategy and release-readiness governance.
Location: Newark
Workplace: Onsite
Employment Type: Full time
Job Function: QA, Test & Release Engineering
Seniority: Sr. Manager level

Key Responsibilities

  • •Lead an automation-first quality strategy, eliminating manual testing waste and making automated testing the default across teams.
  • •Own modern test automation frameworks and end-to-end automated coverage, with expertise in Playwright and comparable frameworks.
  • •Build and own evaluation frameworks for LLM-powered features and agentic systems, including correctness, hallucination, prompt drift, and behavioral regression.
  • •Apply AI to QA itself, including self-healing automation, AI-assisted test generation, and intelligent defect triage.
  • •Provide direct management and set organization-wide quality standards, test strategy, and release-readiness governance.

Pay and Benefits

Salary: USD 98,000 - 135,000 annually
Perks:Health InsuranceDentalVisionPaid Leave401kLife InsuranceDisability InsuranceEapCommuter BenefitsTransit Benefits

Key Requirements

  • •7+ years of experience in quality engineering or test automation, including team leadership or management.
  • •Hands-on expertise in modern test automation frameworks such as Playwright, Cypress, or Selenium.
  • •Experience building automated test suites and integrating them into CI/CD pipelines.
  • •Experience designing evaluation/testing approaches for AI/ML or LLM-powered systems, with a strong data-and-metrics orientation.
  • •Strong understanding of test strategy, defect management, and release-readiness governance across multiple teams.
Skills:LeadershipMentorshipCommunicationQuality risk translationGovernance
Tech Stack:PlaywrightCypressSeleniumCI/CDLLM-powered featuresAgentic systemsSelf-healing automationAI-assisted test generation

Company Brief

WebMD
Operates a leading online health information and health services platform offering medical news, symptom checkers, drug information, and physician directory services to consumers and healthcare professionals.
Industry: HealthTech
Company Size: Enterprise (1,001+ employees)
Growth: Established Company
Funding: Private Equity Backed
Headquarters: New York, United States
Founded: 1996
WebsiteLinkedIn