Evaluation Scenario Writer - QA

Overview At Mindrift, we connect domain experts with cutting-edge AI projects. We are seeking an Evaluation Scenario Writer - QA for a project focused on ensuring the quality and correctness of evaluation scenarios created for LLM agents. This is a flexible, project-based opportunity that blends manual scenario validation, automated test thinking, and collaboration with writers and engineers. Responsibilities Reviewing and validating test scenarios from Evaluation Writers Spotting logical inconsistencies, ambiguities, or missing checks Suggesting improvements to structure, edge cases, or scoring logic Collaborating with infrastructure and tool developers to automate parts of the review Creating clean and testable examples for others to follow Qualifications Strong QA background (manual or automation), preferably in complex testing environments Understanding of test design, regression testing, and edge case detection Ability to evaluate logic and structure of test scenarios (even if written by others) Experience reviewing and debugging structured test case formats (JSON, YAML) Familiarity with Python and JS scripting for test automation or validation Clear communication and documentation skills Willingness to occasionally write or refactor test scenarios Experience testing AI-based systems or NLP applications (valued) Familiarity with scoring systems and behavioral evaluation Git/GitHub workflow familiarity (PR review, versioning of test cases) Experience using test management systems or tracking tools Benefits Get paid for your expertise, with rates that can go up to $55/hour depending on your skills, experience, and project needs Take part in a flexible, remote, freelance project that fits around your commitments Participate in an advanced AI project and gain valuable experience for your portfolio Influence how future AI models understand and communicate in your field of expertise How To Get Started Apply to this post, qualify, and get the chance to contribute to a project aligned with your skills, on your own schedule. Employment details Seniority level: Internship Employment type: Part-time #J-18808-Ljbffr