Engineering Manager, AI

PermitFlow
New York
Workplace: HybridFull timeUSD 180,000 - 250,000 annuallyFunction: Data Science & Machine LearningSkills: ["Leadership","Communication","System design","Architecture review","Stakeholder management"]

Lead and grow the engineering team building PermitFlow’s core agentic infrastructure and evaluation systems. Own technical architecture for workflow agents—orchestration, model serving, latency, cost per task, and failure handling—while partnering with product to define release quality gates. Scale evaluation infrastructure (golden datasets, LLM-as-judge, annotation, monitoring for drift/regression) and drive internal tooling for testing and reliability as coverage expands.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
PermitFlow
PermitFlow
2 days ago

Engineering Manager, AI

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 21 hours agoStatus: Live

Job Summary

Lead and grow the engineering team building PermitFlow’s core agentic infrastructure and evaluation systems. Own technical architecture for workflow agents—orchestration, model serving, latency, cost per task, and failure handling—while partnering with product to define release quality gates. Scale evaluation infrastructure (golden datasets, LLM-as-judge, annotation, monitoring for drift/regression) and drive internal tooling for testing and reliability as coverage expands.
Location: New York
Workplace: Hybrid
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Manager level

Key Responsibilities

  • •Lead and grow an engineering team building and hardening PermitFlow’s agentic infrastructure, raising ownership, quality, and execution speed.
  • •Own the technical architecture behind workflow agents, including orchestration, model serving, latency, cost per task, and failure handling at scale.
  • •Partner with the Product Manager, AI to define what “reliable enough to submit” means and build engineering release gates to that bar.
  • •Build and scale evaluation infrastructure using golden datasets, LLM-as-judge pipelines, human annotation tooling, and online monitoring for drift and regression.
  • •Drive internal tooling to test, error-analyze, and improve agentic work; improve reliability, observability, and engineering efficiency; support hiring and development.

Pay and Benefits

Salary: USD 180,000 - 250,000 annually
Equity and Bonus:Equity
Perks:Health InsuranceDentalVision401kPaid LeaveEquipment StipendCommuter Benefits

Key Requirements

  • •5+ years in software engineering with 1+ year in engineering leadership.
  • •Experience building or shipping production AI/agentic systems and owning the surrounding infrastructure.
  • •Hands-on familiarity with evaluation infrastructure such as golden datasets, LLM-as-judge, human annotation, or online monitoring.
  • •Strong system design and architecture fundamentals, including judgment on when to use an agent vs. simpler systems.
  • •Ability to lead through ambiguity and translate unclear problems into structured execution.
Experience:AIAgentic systemsMachine learningInfrastructure productsEnterprise SaaS
Skills:LeadershipCommunicationSystem designArchitecture reviewStakeholder management
Tech Stack:AILLM-as-judgeGolden data setsHuman annotationOnline monitoringModel servingOrchestrationObservabilityDrift regression

Company Brief

PermitFlow
AI-powered platform that automates and manages construction permitting end-to-end for builders, developers, and contractors — streamlining research, application preparation, submission, and monitoring across U.S. jurisdictions.
Industry: Construction Tech
Company Size: Medium (51 to 250 employees)
Growth: Scaleup
Funding: Series B
Headquarters: New York, United States
Founded: 2021
Glassdoor
Glassdoor: 4.0
WebsiteLinkedInGlassdoor