Join to apply for the Senior Code Reviewer for LLM Data Training (Python) role at AgilityFeat 2 days ago Be among the first 25 applicants Join to apply for the Senior Code Reviewer for LLM Data Training (Python) role at AgilityFeat Get AI-powered advice on this job and more exclusive features. Direct message the job poster from AgilityFeat COO | E-Commerce | Startup | Operations & Product Management | Staff Augmentation | Market Expansion Strategist | International Operations | Business… Location: Remote in Latam Job Type: Hourly Contractor Client Location : India For companies building cutting-edge AI systems, high-quality data is everything. Latin America is home to exceptional engineering talent that can contribute to improving AI models through expert-level reviews and technical evaluation. At AgilityFeat, we connect top Latin American tech professionals with remote teams at leading companies worldwide. We’ve helped dozens of organizations refine their product experiences while opening doors for creative engineers to take on impactful, career-growing work. This role is ideal for seasoned Python developers who want to contribute to training and aligning large language models (LLMs) through expert code evaluation. If you're a highly analytical developer with excellent attention to detail and a deep understanding of Python, we’ve got an exciting opportunity for you! Please note: There is NO FLEXIBILITY in the fluency requirement: B2+ or above is required . Do not apply if you have only Intermediate English. About the Role We’re looking for a Senior Code Reviewer to evaluate the quality of assessments made by annotators reviewing AI-generated Python code. You’ll ensure that annotation guidelines are followed, and validate both the correctness of the code and the quality of the reviewer feedback. Key Responsibilities Review and audit annotator evaluations of AI-generated Python code responses. Determine whether the code meets prompt instructions, is functionally correct, and secure. Use proof-of-work techniques to verify the code’s validity and performance. Flag inaccuracies in annotator scoring or reasoning. Provide clear and constructive feedback to support high QA standards. Adhere to Project Atlas protocols for evaluation consistency and accuracy. Technical Requirements Must Have: 5–7+ years of experience in Python development , QA , or code review . Strong command of Python syntax , testing strategies, debugging, and edge cases. Proficiency with code execution environments and evaluation tools. Solid written communication skills for technical feedback. Familiarity with structured QA/annotation workflows . English proficiency at B2 level or higher . Nice to Have: Knowledge of annotation platforms and tooling. Exposure to Reinforcement Learning from Human Feedback (RLHF) pipelines. Fluent English is mandatory. All information must be submitted in English. Seniority level Seniority level Mid-Senior level Employment type Employment type Full-time Job function Job function Administrative Industries IT Services and IT Consulting Referrals increase your chances of interviewing at AgilityFeat by 2x Sign in to set job alerts for “Reviewer” roles. Freelance English Content Writer - AI Trainer Renewals Declaration Writer for an Immigration Law Firm Bogota, D.C., Capital District, Colombia 2 weeks ago Environmental & Safety (EHS) Proposal Writer English to Japanese AI Translation Reviewer - Computer Science and Mathematics We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr