Sr. Software Development Engineer in Test (Agentic)

Dialpad
Vancouver
Workplace: OnsiteFull timeCAD 150,500 - 175,250 annuallyFunction: Product ManagementExperience: 6+ yearsEducation: bachelorsSkills: ["Collaboration","Mentorship","Problem-solving","Quality mindset","Communication"]

Own end-to-end quality for agentic AI features and workflows, building test automation frameworks and tooling for AI/LLM-driven systems. Develop evaluation frameworks to measure response quality, accuracy, and hallucination rates, and integrate AI quality checks into CI/CD with fast feedback. Drive high coverage for critical AI workflows through deterministic and probabilistic validation, while creating observability and debugging tools and executing performance/load tests for AI services.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Dialpad
Dialpad
2 days ago

Sr. Software Development Engineer in Test (Agentic)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 17 hours agoStatus: Live

Job Summary

Own end-to-end quality for agentic AI features and workflows, building test automation frameworks and tooling for AI/LLM-driven systems. Develop evaluation frameworks to measure response quality, accuracy, and hallucination rates, and integrate AI quality checks into CI/CD with fast feedback. Drive high coverage for critical AI workflows through deterministic and probabilistic validation, while creating observability and debugging tools and executing performance/load tests for AI services.
Location: Vancouver
Workplace: Onsite
Employment Type: Full time
Job Function: Product Management
Seniority: Mid level

Key Responsibilities

  • •Own end-to-end quality for agentic features and workflows, covering strategy, development, execution, and release qualification.
  • •Design and build automation tooling and frameworks for prompt flows, agent orchestration, and tool integrations.
  • •Develop and maintain evaluation frameworks (evals) to measure response quality, accuracy, and hallucination rates.
  • •Drive automation coverage (80%+ for critical AI workflows) using deterministic + probabilistic validation approaches.
  • •Integrate AI quality checks into CI/CD pipelines and build observability/debugging tooling such as prompt tracing and response analysis.

Pay and Benefits

Salary: CAD 150,500 - 175,250 annually

Key Requirements

  • •6+ years of software engineering or SDET experience with an emphasis on software development.
  • •Strong programming skills in Python (preferred), Java, or JavaScript.
  • •Experience testing distributed, cloud-native SaaS systems and APIs.
  • •Hands-on exposure to LLMs or AI/ML systems and understanding of non-deterministic, probabilistic testing approaches.
  • •Bachelor’s degree in Computer Science or equivalent practical experience.
Experience:6+ yearsAI agentsLLMCloud-native SaaSDistributed systems
Education:Bachelor's in Computer Science
Skills:CollaborationMentorshipProblem-solvingQuality mindsetCommunication
Languages:English
Tech Stack:PythonJavaJavaScriptGoGoogle Cloud PlatformCloud RunApp EngineKubernetesDatastoreRedisElasticsearchVue3ReactLLM APIsLiveKitPrompt orchestration frameworksEvaluation toolingJenkinsGitHub ActionsCI/CD

Company Brief

Dialpad
Provides cloud-based business communications platform offering voice, video, messaging, contact center, and AI-driven call intelligence to help teams communicate and collaborate across devices and locations.
Industry: SaaS
Company Size: Enterprise (1,001+ employees)
Growth: Scaleup
Headquarters: San Francisco, United States
Founded: 2011
WebsiteLinkedIn