Python Insfrastructure Engineer - Model Evaluation

Python Infrastructure Engineer — Model Evaluation (AI Training)

About the Role

What if your Python expertise could directly shape how the world's most advanced AI models are built, tested, and improved? We're looking for a senior Python engineer to design and build the data pipelines, evaluation harnesses, and annotation tooling that sit at the heart of cutting-edge AI development.

This is a fully remote, flexible contract role working alongside leading AI research labs on real production systems. If you're a strong Python engineer who wants to do meaningful, high-impact work at the frontier of AI — this is the role for you.

• Organization: Alignerr
• Type: Hourly Contract
• Location: Remote
• Commitment: 20–40 hours/week

What You'll Do

• Design, build, and optimize high-performance Python systems supporting AI data pipelines and model evaluation workflows
• Develop full-stack tooling and backend services for large-scale data annotation, validation, and quality control
• Build and maintain evaluation harnesses that integrate with ML inference frameworks
• Improve reliability, performance, and safety across existing Python codebases
• Instrument systems with observability and metrics collection to monitor reliability and model performance
• Identify bottlenecks and edge cases in data and system behavior, and implement scalable fixes
• Collaborate with data, research, and engineering teams to support model training and evaluation workflows
• Participate in synchronous design reviews to iterate on architecture and implementation decisions

Who You Are

• Native or fluent English speaker with clear written and verbal communication skills
• Full-stack developer with a strong systems programming background
• 3–5+ years of professional experience writing production-grade Python
• Experienced building evaluation harnesses for ML models and integrating with inference frameworks
• Solid background in observability, metrics collection, and monitoring for production systems
• Self-motivated and reliable — able to commit 20–40 hours per week

Nice to Have

• Prior experience with data annotation, data quality, or evaluation systems
• Familiarity with AI/ML workflows, model training, or benchmarking pipelines
• Experience with distributed systems or developer tooling
• Background in MLOps or AI infrastructure

Why Join Us

• Work directly on cutting-edge AI projects alongside leading research labs
• Fully remote and flexible — structure your work week around your life
• Freelance autonomy with the depth and consistency of meaningful, long-term technical work
• Make a tangible impact on how next-generation AI models are evaluated and improved
• Potential for ongoing work and contract extension as new projects launch