Sr. Software Development Engineer in Test (Agentic)

Dialpad
Kitchener, Vancouver
Workplace: OnsiteFull timeCAD 151,000 - 175,000 annuallyFunction: Product ManagementEducation: bachelorsSkills: ["Collaboration","Mentoring","Risk mitigation","Technical problem-solving","Attention to quality"]

Own end-to-end quality for Dialpad’s agentic AI voice services, building test automation and quality frameworks that make AI/LLM workflows reliable, performant, and user-friendly. Develop automated E2E tests across frontend UI, backend services, APIs, and audio/text interactions, including orchestration and safety safeguards. Create evaluation frameworks to measure response quality and hallucination rates, integrate AI quality checks into CI/CD, and drive high coverage using deterministic and probabilistic validation.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Dialpad
Dialpad
21 hours ago

Sr. Software Development Engineer in Test (Agentic)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 9 hours agoStatus: Live

Job Summary

Own end-to-end quality for Dialpad’s agentic AI voice services, building test automation and quality frameworks that make AI/LLM workflows reliable, performant, and user-friendly. Develop automated E2E tests across frontend UI, backend services, APIs, and audio/text interactions, including orchestration and safety safeguards. Create evaluation frameworks to measure response quality and hallucination rates, integrate AI quality checks into CI/CD, and drive high coverage using deterministic and probabilistic validation.
Location: Kitchener, Vancouver
Workplace: Onsite
Employment Type: Full time
Job Function: Product Management
Seniority: Mid level

Key Responsibilities

  • •Own end-to-end quality for agentic features and workflows, covering strategy, execution, and release qualification.
  • •Design and build automation tooling/frameworks for AI/LLM systems, including prompt flows, agent orchestration, and tool integrations.
  • •Develop evaluation frameworks (evals) to measure response quality, accuracy, and hallucination rates.
  • •Drive AI workflow automation coverage (80%+ for critical workflows) using deterministic and probabilistic validation approaches.
  • •Integrate AI quality checks into CI/CD pipelines and build LLM observability/debugging tooling (e.g., prompt tracing and response analysis).

Pay and Benefits

Salary: CAD 151,000 - 175,000 annually

Key Requirements

  • •6+ years of software engineering or SDET experience with software development emphasis.
  • •Strong programming skills in Python (preferred), Java, or JavaScript.
  • •Experience testing distributed, cloud-native SaaS systems and APIs.
  • •Hands-on exposure to LLMs/AI-ML systems (e.g., OpenAI, Claude, Gemini) and probabilistic testing for non-deterministic systems.
  • •Build test frameworks and scalable automation; familiarity with AI evaluation techniques; bachelor’s degree in Computer Science or equivalent experience.
Experience:SaaSCloud-nativeAI/ML
Education:Bachelor's in Computer Science
Skills:CollaborationMentoringRisk mitigationTechnical problem-solvingAttention to quality
Languages:English
Tech Stack:PythonJavaJavaScriptGoGoogle Cloud PlatformCloud RunApp EngineKubernetesDatastoreRedisElasticSearchVue3ReactLLM APIsLiveKitPrompt orchestration frameworksEvaluation toolingJenkinsGitHub ActionsCI/CD

Company Brief

Dialpad
Provides cloud-based business communications platform offering voice, video, messaging, contact center, and AI-driven call intelligence to help teams communicate and collaborate across devices and locations.
Industry: SaaS
Company Size: Enterprise (1,001+ employees)
Growth: Scaleup
Headquarters: San Francisco, United States
Founded: 2011
WebsiteLinkedIn