Senior Python Engineer - LLM Code Evaluation

G2i
Anywhere
Workplace: RemoteContractUSD 100 - 200 hourlyFunction: Solutions Engineering & Sales EngineeringSkills: ["Critical thinking","Opinionated feedback","Rigorous judgment"]

Evaluate the quality of AI coding agent interactions in real-world scenarios, focusing on whether responses make sense, whether preambles and reasoning are useful, and whether outputs reflect strong engineering judgment. You’ll judge explanations, distinguish response quality levels, and provide clear, opinionated feedback. This role prioritizes engineering “taste” over syntax correctness and includes tools like Codex, Claude Code, and Cursor.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
G2i
G2i
2 days ago

Senior Python Engineer - LLM Code Evaluation

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 9 hours agoStatus: Live

Job Summary

Evaluate the quality of AI coding agent interactions in real-world scenarios, focusing on whether responses make sense, whether preambles and reasoning are useful, and whether outputs reflect strong engineering judgment. You’ll judge explanations, distinguish response quality levels, and provide clear, opinionated feedback. This role prioritizes engineering “taste” over syntax correctness and includes tools like Codex, Claude Code, and Cursor.
Location: Anywhere
Workplace: Remote
Employment Type: Contract
Job Function: Solutions Engineering & Sales Engineering
Seniority: Mid level

Key Responsibilities

  • •Assess AI-generated coding interactions end-to-end in realistic scenarios.
  • •Judge whether responses are useful, correct at a high level, and aligned with strong engineering thinking.
  • •Evaluate the quality of explanations and reasoning, not just the code.
  • •Provide clear, opinionated feedback on what worked, what didn’t, and what felt misleading.
  • •Help define what “great” looks like for interactions with tools like Cursor.

Pay and Benefits

Salary: USD 100 - 200 hourly

Key Requirements

  • •Staff/Principal-level engineering experience (or equivalent).
  • •Strong background in TypeScript/JavaScript or Python.
  • •Hands-on experience using OpenAI Codex, Claude Code, and Cursor.
  • •Comfort evaluating code without fully executing or deeply reviewing every line.
  • •Ability to make direct, opinionated but rigorous judgments and provide feedback.
Experience:AI-assisted development
Skills:Critical thinkingOpinionated feedbackRigorous judgment
Tech Stack:PythonTypeScriptJavaScriptOpenAI CodexClaude CodeCursor

Company Brief

G2i
G2i is a video-first hiring platform that connects companies with pre-vetted remote engineers (React, React Native, GraphQL, etc.) using technical screening, live interviews, and AI-assisted hiring infrastructure to speed hiring.
Industry: HR Tech
Company Size: Medium (51 to 250 employees)
Growth: Growth Stage Startup
Headquarters: Miami, United States
Founded: 2016
Glassdoor
Glassdoor: 3.3
WebsiteLinkedInGlassdoor