DroneDeploy

2 weeks ago

Senior Software Engineer, Data Science (AI Accuracy) (NZ)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 9 hours agoStatus: Live

Job Summary

Build systems that make AI trustworthy by measuring, understanding, and improving the accuracy of LLM-powered features across Progress AI and related products. Own evaluation datasets and pipelines, create internal tooling for error analysis, audits, and experimentation, and refine prompts and underlying models through production rollout and post-launch monitoring. Connect ontology and schema design to real-world accuracy outcomes while collaborating closely with engineering, product, and Inspection/Progress AI stakeholders.

Location: Auckland

Workplace: Hybrid

Employment Type: Contract · Permanent

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Design and maintain evaluation datasets, pipelines, and ground-truth quality controls.
•Diagnose and fix measurement-path issues between offline evaluations and production accuracy.
•Iterate on and scale labeling frameworks and knowledge capture systems.
•Own prompt optimization across the lifecycle from candidate selection through production rollout and post-launch validation.
•Automate evaluation pipelines and build tooling for error analysis, audits, and experimentation at scale.

Travel: Low travel

Key Requirements

•Strong professional experience with Python for data manipulation, analysis, and tooling (e.g., pandas, NumPy).
•Solid SQL and database experience, including querying, data modeling, and working with large datasets.
•Background in data science with knowledge of LLM accuracy, including visual reasoning.
•Experience building and maintaining evaluation datasets; running, debugging, and iterating on evals (e.g., with Braintrust or Langfuse) and conducting error analysis.
•Experience designing ground-truth and labeling systems, including measuring label agreement and data quality.

Skills:CollaborationIterative developmentEngineering judgmentOwnership

Tech Stack:PythonPandasNumPySQLBraintrustLangfuse

Company Brief

DroneDeploy

Provides cloud-based drone mapping and analytics software that enables teams in construction, agriculture, mining, and inspection to capture, process, and analyze aerial data for operational insights and decision-making.

Industry: Enterprise Software

Company Size: Large (251 to 1,000 employees)

Growth: Scaleup

Funding: Series D

Headquarters: San Francisco, United States

Founded: 2013

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

DroneDeploy

2 weeks ago

Senior Software Engineer, Data Science (AI Accuracy) (NZ)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 9 hours agoStatus: Live

Job Summary

Location: Auckland

Workplace: Hybrid

Employment Type: Contract · Permanent

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Design and maintain evaluation datasets, pipelines, and ground-truth quality controls.
•Diagnose and fix measurement-path issues between offline evaluations and production accuracy.
•Iterate on and scale labeling frameworks and knowledge capture systems.
•Own prompt optimization across the lifecycle from candidate selection through production rollout and post-launch validation.
•Automate evaluation pipelines and build tooling for error analysis, audits, and experimentation at scale.

Travel: Low travel

Key Requirements

•Strong professional experience with Python for data manipulation, analysis, and tooling (e.g., pandas, NumPy).
•Solid SQL and database experience, including querying, data modeling, and working with large datasets.
•Background in data science with knowledge of LLM accuracy, including visual reasoning.
•Experience building and maintaining evaluation datasets; running, debugging, and iterating on evals (e.g., with Braintrust or Langfuse) and conducting error analysis.
•Experience designing ground-truth and labeling systems, including measuring label agreement and data quality.

Skills:CollaborationIterative developmentEngineering judgmentOwnership

Tech Stack:PythonPandasNumPySQLBraintrustLangfuse

Company Brief

DroneDeploy

Industry: Enterprise Software

Company Size: Large (251 to 1,000 employees)

Growth: Scaleup

Funding: Series D

Headquarters: San Francisco, United States

Founded: 2013

Website LinkedIn