AI Automation Engineer

Fortinet
Sunnyvale
Workplace: HybridFull timeUSD 101,600 - 124,200 annuallyFunction: QA, Test & Release EngineeringEducation: bachelorsSkills: ["Attention to detail","Technical writing","Independence","Adaptability"]

Build and maintain a technical interview automation pipeline, including creating and validating question repositories and designing end-to-end workflows for question selection, test delivery, answer collection, and AI-assisted scoring. Calibrate and benchmark LLM scoring prompts/models against human-graded samples, handle model failure modes with safeguards, and flag low-confidence cases for review. Partner with engineering teams to expand automation scope and improve pipeline quality, consistency, and auditability.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Fortinet
Fortinet
22 hours ago

AI Automation Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 12 hours agoStatus: Live

Job Summary

Build and maintain a technical interview automation pipeline, including creating and validating question repositories and designing end-to-end workflows for question selection, test delivery, answer collection, and AI-assisted scoring. Calibrate and benchmark LLM scoring prompts/models against human-graded samples, handle model failure modes with safeguards, and flag low-confidence cases for review. Partner with engineering teams to expand automation scope and improve pipeline quality, consistency, and auditability.
Location: Sunnyvale
Workplace: Hybrid
Employment Type: Full time
Job Function: QA, Test & Release Engineering
Seniority: Entry level

Key Responsibilities

  • •Manage the technical interview assessment repository by adding, revising, and retiring questions and maintaining question metadata.
  • •Architect scoring workflows that evaluate candidate answers against reference rubrics, calibrating and benchmarking against human-graded samples.
  • •Design safeguards for LLM failure modes and route edge cases or low-confidence scores to human review.
  • •Automate the end-to-end pipeline from question selection through scoring and reporting, and track results to improve benchmarks.
  • •Transition code and repositories to an internal Git platform and expand automation to other internal workflows as scope grows.

Pay and Benefits

Salary: USD 101,600 - 124,200 annually
Equity and Bonus:Equity
Perks:Health InsuranceDentalVisionLife InsuranceDisability Insurance401k

Key Requirements

  • •Bachelor's degree in Computer Science or related field, or equivalent practical experience.
  • •Solid programming fundamentals (Python, JavaScript/TypeScript, or similar).
  • •Comfort working with APIs, including LLM/AI APIs (OpenAI, Anthropic, etc.).
  • •Demonstrated experience designing and evaluating prompts/pipelines for LLM-based tools.
  • •Proficient with Git and a hosted platform (GitHub or GitLab), plus basic database understanding (SQL or similar).
Education:Bachelor's in Computer Science
Skills:Attention to detailTechnical writingIndependenceAdaptability
Languages:US
Tech Stack:PythonJavaScriptTypeScriptLLM/AI APIsOpenAIAnthropicGitGitHubGitLabSQL

Eligibility

Work Authorization:Authorization required. Sponsorship not provided.

Company Brief

Fortinet
Provides enterprise cybersecurity solutions including data protection, cloud security, network security, and user and entity behavior analytics to help organizations prevent insider threats, secure cloud and on-premises environments, and enforce data loss prevention policies.
Industry: Cybersecurity
Company Size: Enterprise (1,001+ employees)
Growth: Established Company
Funding: Private Equity Backed
Headquarters: Austin, United States
Founded: 1994
WebsiteLinkedIn