AgileEngine is one of the Inc. 5000 fastest-growingcompanies in the US and a top-3 ranked dev shop according toClutch. We create award-winning custom software solutions that helpcompanies across 15+ industries change the lives of millions. Ifyou like a challenging environment where you’re working with thebest and are encouraged to learn and experiment every day, there’sno better place - guaranteed! :) WHAT YOU WILL DO 1. Architect andoversee end-to-end test strategies across different layers in thetesting pyramid; 2. Define, evaluate, and monitor model validationcriteria, including accuracy threshold, confidence scoring,fallback behaviors, etc. 3. Drive the automation of QA processes,ensuring robust, scalable processes for data ingestion, extraction,and classification; 4. Lead the validation integration pointsbetween AI components and business logic, identifying risks andensuring seamless operation; 5. Own, maintain, and evolve the testinfrastructure, specifically the CI/CD quality gates, datageneration, and automated validation scripts; 6. Establish andmonitor key quality metrics aligned with the business goals,including turnaround time, confidence scores, and user corrections;7. Act as a strategic partner to Product and Engineering, andadvocate for quality from design through delivery; 8. Keep up withthe latest AI tech and how to best apply the latest advancements tomake the previously impossible possible for our customers; 9. Senseof ownership, take ownership of code quality through exploratoryand automated tests. MUST HAVES 1. Bachelor’s degree in ComputerScience, Information Systems, Engineering, or a related fieldexperience; 2. 5+ years in AQA, Software Test Engineering, orrelated roles, including leadership responsibilities; 3. Experiencein AQA Javascript; 4. Solid knowledge of software development lifecycle (SDLC), Agile methodologies, and Quality Assurance or QualityEngineering methodologies; 5. Proven experience building andscaling test automation frameworks (e.g., Pytest, Cypress,Selenium, Postman, etc.). Strong technical skills with REST APIs,data, pipelines, and backend services; 6. Strong technicalbackground with APIs, CI/CD pipelines, backend services, and cloudenvironments (AWS, Azure); 7. Exceptional attention to detail, witha mindset for building robust systems that catch issues early andprotect the customer experience; 8. Demonstrated fast, proactivelearning skills, especially crucial in the rapidly changing fieldof AI; 9. Upper-Intermediate English level. NICE TO HAVES 1. Experience testing AI/ML in production environments; 2. Experiencewith GenAI model evaluation metrics; 3. Experience setting upproactive monitoring and alerting systems to catch issues likemodel drift, performance degradation, or workflow failures. THEBENEFITS OF JOINING US 1. Professional growth: Accelerate yourprofessional journey with mentorship, TechTalks, and personalizedgrowth roadmaps. 2. Competitive compensation: We match yourever-growing skills, talent, and contributions with competitiveUSD-based compensation and budgets for education, fitness, and teamactivities. 3. A selection of exciting projects: Join projects withmodern solutions development and top-tier clients that includeFortune 500 enterprises and leading product brands. 4. Flextime:Tailor your schedule for an optimal work-life balance, with optionsof working from home and going to the office—whatever makes you thehappiest and most productive. Your application doesn't end here! Tounlock the next steps, check your email and complete yourregistration on our Applicant Site. Incomplete registration willresult in the termination of your process. #J-18808-Ljbffr Engineering