Research Scientist, Gemini, UI Control Engineering Mountain View, California, US

Artificial Intelligence could be one of humanitys most useful inventions. At Google DeepMind, were a team of scientists, engineers, machine learning experts and more, working together to advance the state of the art in artificial intelligence. We use our technologies for widespread public benefit and scientific discovery, and collaborate with others on critical challenges, ensuring safety and ethics are the highest priority.

About Us

There is great interest in having AI agents execute complex and long-running tasks. For example, instead of merely creating a travel itinerary, an agent could read reviews, compare options, and finds flights, accommodations, and activities. To be able to perform all actions a human can do, it is likely our agents will need to be able perform actions via a user interface. The Gemini UI Control group is dedicated to enabling computers to understand and interact with user interfaces in a human-like manner, thereby unlocking a wide range of agentic capabilities.

We develop models that power UI Control applications across Google, including Project Mariner and the AI Studio Computer Use API, with more on the horizon.. Our primary focus is on post-training generative models to equip them with the ability to comprehend and interact with user interfaces to achieve specific goals

The Role

We concentrate on the following three key areas related to UI understanding and control:

• Data Enhancement: Adding new data to post-training mixtures in Gemini
• Self-Improvement: Exploring methods such as online exploration and reinforcement learning
• Evaluation Contributions: Developing and contributing new evaluation methodologies

The objective is to continuously improve quality while tackling tasks of increasing complexity and ambiguity across a wider range of surfaces.

Key responsibilities:

This role involves making core contributions to post-training in Gemini that advances the frontier of what digital agents can do and is deployable into production. The employee will identify model areas of improvement and own work related to training and evaluating UI Control agents across various digital environments. Multi-step reinforcement learning in multimodal, digital environments is a focus. Research that improves the ability of the agent to handle more complex tasks or incorporate more user interaction is a plus.

About You

The ideal candidate is someone who can push the boundaries of AI research related to computer use models and is comfortable working across the research stack to see their work through.

In order to set you up for success as a Research Engineer at Google DeepMind, we look for the following skills and experience:

• PhD or relevant experience in computer science related field.
• Experience with multi-step reinforcement learning
• Experience with post-training generative models

In addition, the following would be an advantage:

• Understanding of model training, evaluation metrics
• Experience with designing and analyzing machine learning experiments
• Ability to design and build evaluations
• Adaptability: The field is evolving rapidly, requiring a willingness to learn new technologies and approaches.

The US base salary range for this full-time position is between $166,000 - $244,000 + bonus + equity + benefits. Your recruiter can share more about the specific salary range for your targeted location during the hiring process.

At Google DeepMind, we value diversity of experience, knowledge, backgrounds and perspectives and harness these qualities to create extraordinary impact. We are committed to equal employment opportunity regardless of sex, race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, gender identity, pregnancy, or related condition (including breastfeeding) or any other basis as protected by applicable law. If you have a disability or additional need that requires accommodation, please do not hesitate to let us know.

Create a Job Alert

Interested in building your career at DeepMind? Get future opportunities sent straight to your email.

Apply for this job

*

First Name *

Last Name *

Email *

Phone

Resume/CV *

Enter manually

Accepted file types: pdf, doc, docx, txt, rtf

Enter manually

Accepted file types: pdf, doc, docx, txt, rtf

LinkedIn Profile

Link to external profile e.g. LinkedIn, GitHub etc.

Where did you hear about this role? * Select...

U.S. Standard Demographic Questions

Google DeepMind is subject to certain governmental recordkeeping and reporting requirements for the administration of civil rights laws and regulations. In order to comply with these laws and achieve our goal of a diverse and inclusive workforce, Google DeepMind invites employees to voluntarily self-identify their race or ethnicity. Submission of this information is voluntary and refusal to provide it will not subject you to any adverse treatment. The information obtained will be kept confidential and may only be used in accordance with the provisions of applicable laws, executive orders, and regulations, including those that require the information to be summarized and reported to the federal government for civil rights enforcement. When reported, data will not identify any specific individual. If you'd like more information about your EEO rights as an applicant under the law, please click herehttps://www.eeoc.gov/ employers/eeo-law-poster.

How would you describe your gender identity? (mark all that apply) Select...

How would you describe your racial/ethnic background? (mark all that apply) Select...

How would you describe your sexual orientation? (mark all that apply) Select...

Do you identify as transgender? Select...

Do you have a disability or chronic condition (physical, visual, auditory, cognitive, mental, emotional, or other) that substantially limits one or more of your major life activities, including mobility, communication (seeing, hearing, speaking), and learning? Select...

Are you a veteran or active member of the United States Armed Forces? Select...