Applied AI Data Analyst

Confido
New York
Workplace: OnsiteFull timeUSD 180,000 - 230,000 annuallyFunction: Data Analytics & Business IntelligenceEducation: mastersSkills: ["Analytical mindset","Root cause analysis","Cross-functional collaboration"]

Bridge unstructured financial and operational data with the Applied AI Research team to evaluate and improve agentic AI workflows. Own the evaluation platform for forecasting pipelines, build golden datasets, and create dashboards and monitoring harnesses to track model drift, hallucination rates, and extraction accuracy. Collaborate with applied AI engineers, product managers, and business operations to turn real-world problem definitions into quantitative insights for mission-critical decisions.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Confido
Confido
1 day ago

Applied AI Data Analyst

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 2 hours agoStatus: Live

Job Summary

Bridge unstructured financial and operational data with the Applied AI Research team to evaluate and improve agentic AI workflows. Own the evaluation platform for forecasting pipelines, build golden datasets, and create dashboards and monitoring harnesses to track model drift, hallucination rates, and extraction accuracy. Collaborate with applied AI engineers, product managers, and business operations to turn real-world problem definitions into quantitative insights for mission-critical decisions.
Location: New York
Workplace: Onsite
Employment Type: Full time
Job Function: Data Analytics & Business Intelligence
Seniority: Mid level

Key Responsibilities

  • •Evaluate LLM-driven document processing and agentic workflows by analyzing edge cases, prediction quality, hallucination rates, and extraction accuracy.
  • •Perform root cause analysis on inconsistent financial documents and fragmented enterprise systems to determine why models succeed or fail.
  • •Define, curate, and maintain golden evaluation datasets for automated backtesting of AI agents and demand forecasting models.
  • •Build dashboards, data transformations, and evaluation harnesses to monitor system health, model drift, and AI performance across release cycles.
  • •Partner with applied AI engineers, product managers, and business operations managers to convert qualitative requirements into quantitative product-defining insights.

Pay and Benefits

Salary: USD 180,000 - 230,000 annually
Equity and Bonus:Equity
Perks:Health InsuranceDentalVision401kPaid LeaveParental LeaveEquity

Key Requirements

  • •Graduate degree (M.S. or Ph.D.) in Computer Science, Data Science, Statistics, Mathematics, Physics, or a related STEM field.
  • •2–4 years of experience as a quantitative data analyst or in AI evaluation work alongside AI/ML R&D teams.
  • •Proficiency in Python and hands-on experience with AI/ML models.
  • •Strong SQL skills and enterprise data infrastructure experience (PostgreSQL, Snowflake, dbt, AWS).
  • •Ability to translate noisy, unstructured document data into rigorous evaluation insights for engineers.
Education:Master's in Computer Science, Data Science, Statistics, Mathematics, Physics (or related STEM)
Skills:Analytical mindsetRoot cause analysisCross-functional collaboration
Tech Stack:PythonAI/MLSQLPostgreSQLSnowflakeDbtAWSLLMLangSmithArize AITruLensAgentic workflowsMulti-agent orchestrationDashboards

Eligibility

Nationality:US National
Work Authorization:Authorization required. Sponsorship not provided.

Company Brief

Confido
Provides custom software development, cloud solutions, and IT consulting services to help businesses build scalable web and mobile applications, implement cloud infrastructure, and improve security and operational efficiency through tailored technology solutions.
Industry: Professional Services
Website