DeWinter Group is hiring: Content Evaluator in San Jose
DeWinter Group, San Jose, CA, United States, 95199
Overview
Job Title: Content Evaluator — Duration: Through EOY — Start Date: ASAP — Location: Remote — W2/C2C: W2 Only
Job Description
As a relevance evaluator you will be responsible for evaluating how relevant a given document (or image, video) is for a particular user query. You will rate the document relevance 1-5 (from not relevant at all to exactly what the user was looking for). Rating relevance requires first understanding user intent, for example, whether they are researching a topic, or looking for their own document, and forming an opinion what would be a good document (or image, etc.) for this particular query for this particular user.
Responsibilities
- Review the query and document pairs, including context provided by the system (such as when the query was issued and who issued it), researching the problem to understand user intent by using the product (Dash assistant and search, including formulating queries for elastic search index in KQL), and then rating the document or other required artifact (e.g. assistant answer) according to the provided relevance rubric.
Requirements
- Understanding of English
- Ability to formulate search or ChatGPT-like queries
Preferred Qualifications
- No engineering experience is required, but being able to script certain tasks in Python can be beneficial.
- Previous experience with relevance labeling or rating would be preferable.
Equal Opportunity
DeWinter Group and Maris Consulting is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. We post pay scales which are based on our client pay ranges. DeWinter, Maris, and our clients have the right to modify the requirements of the role which can impact the pay ranges posted.