Senior Software Engineer - AI Benchmark & Training (SWE-bench)

Job Type: Contractor (~15 hrs a week)

Location: Remote

Schedule: Flexible, you pick the hours and days (including weekends if desired)

Job Summary: In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

You'll create realistic, interesting and challenging coding tasks for AI to solve involving feature implementation and bug fixes, and implement deterministic verifiers that accept reasonable valid solutions and reject everything else.

What we're looking for

1. Strong backend experience in Java, Node.js, Go, Rust, Typescript, Python, C++ or similar
2. A track record building scalable services and APIs
3. Comfort with debugging, performance work and automated testing
4. Solid code review instincts
5. A strong testing background is a big plus — experience designing test strategy, building and maintaining large test suites, diagnosing flaky tests, or raising the testing bar on a team.

Preferred Qualifications

1. Proven experience in backend development, with a track record of delivering high-quality and performant projects.
2. Demonstrated success in identifying and resolving software bugs and system bottlenecks.
3. Experience with codebase refactoring and implementing complex features in large-scale applications.
4. Strong understanding of algorithms, data structures, and foundational computer science concepts.
5. Excellent analytical, problem-solving, and technical communication skills.

A note on the process: our interview and assessment stages are designed to see how you think, so we ask that you complete them without outside assistance. The same applies once you're on the project — the value of the work comes from your own judgment.

Process:

1. Apply to the role, filling out the screening questions
2. Complete AI Interview (approx. 30 minutes)
3. Technical Assessment (Tentative)
4. Hiring Manager review

Compensation Structure

Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the expert’s experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.

Start Timeline & Availability

We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected, we expect you to start your first tasks within 24–48 hours of completing onboarding.