AI User Experience Reliability Lead
Lenovo, Raleigh, NC, United States
We are Lenovo. We do what we say. We own what we do. We WOW our customers.
Lenovo is a US$69 billion revenue global technology powerhouse, ranked #196 in the Fortune Global 500, and serving millions of customers every day in 180 markets. Focused on a bold vision to deliver Smarter Technology for All, Lenovo has built on its success as the world’s largest PC company with a full-stack portfolio of AI-enabled, AI-ready, and AI-optimized devices (PCs, workstations, smartphones, tablets), infrastructure (server, storage, edge, high performance computing and software defined infrastructure), software, solutions, and services. Lenovo’s continued investment in world‑changing innovation is building a more equitable, trustworthy, and smarter future for everyone, everywhere. Lenovo is listed on the Hong Kong stock exchange under Lenovo Group Limited (HKSE: 992) (ADR: LNVGY).
This transformation together with Lenovo’s world‑changing innovation is building a more inclusive, trustworthy, and smarter future for everyone, everywhere. To find out more visit www.lenovo.com, and read about the latest news via our StoryHub.
About Our Team Lenovo is building
Quantum , a next‑generation hybrid AI platform spanning Windows, Android, and cloud. As part of this vision, we are expanding the engineering organization behind
Qira , Lenovo’s cross‑device Personal AI that delivers intelligent, safe, and reliable experiences across Lenovo and Motorola products.
We are hiring an
AI User Experience Reliability Lead
to define and drive the technical strategy for reliability across Qira’s AI behaviors, outputs, and runtime safety systems. This role leads the engineering direction for how Qira’s intelligence is evaluated, monitored, and improved — ensuring users experience correct, consistent, stable, and trustworthy AI behavior at global scale.
This is a critical leadership position shaping one of the most important engineering domains within Qira and Lenovo’s broader AI strategy.
Location Open to remote work in the US. The preferred work location is Chicago, IL.
What You’ll Do AI Quality Behavioral Reliability
Define and execute the strategy for measuring and improving the accuracy, stability, and task success of Qira’s AI actions.
Build and evolve evaluation frameworks, behavioral scorecards, and quality validation for models, prompts, retrievers, and task orchestration.
Develop systems to detect hallucinations, regressions, safety deviations, and other behavioral anomalies in real‑time.
Safety Guardrail Reliability
Ensure the reliability of runtime safety systems, including content moderation, jailbreak/misuse detection, safety classifiers, and policy enforcement.
Partner with Safety, Legal, Ethics, and Product teams to convert requirements into robust technical safety solutions.
Validate safety updates through testing, evaluation, and monitored deployment.
AI Observability Telemetry
Define the telemetry, metrics, traces, and data needed to understand AI behavior end‑to‑end across device, edge, and cloud.
Collaborate with observability and platform teams to integrate AI‑specific signals (quality, drift, safety events) into a unified reliability platform.
Lead the creation of dashboards and analytics that provide deep insight into AI behavior and experience reliability.
Reliability Engineering Architecture Influence
Partner with engineering, AI/ML, and product teams to embed reliability into the design of prompts, models, policies, and workflow orchestration.
Influence architecture to ensure AI behavior is predictable, testable, explainable, and resilient.
Establish standards for AI evaluation, rollout safety, service readiness, and runtime validation.
Cross‑Functional Leadership
Represent AI experience reliability in architectural reviews, product decisions, roadmap development, and launch readiness.
Drive cross‑team alignment on reliability metrics, evaluation methods, and monitoring strategies.
Collaborate with ML researchers, applied AI teams, data scientists, and UX to ensure user‑centric reliability goals.
Execution Delivery
Lead major engineering initiatives across AI quality, evaluation, safety assurance, and behavioral monitoring.
Set priorities, ensure accountability, and drive timely delivery of reliability systems and tooling.
Foster a culture of engineering excellence, learning, and continuous improvement.
Basic Qualifications
8+ years in AI/ML engineering, evaluation engineering, applied ML, reliability engineering, or large-scale distributed systems, with depth in AI behavior, evaluation, or safety.
Bachelor’s Degree in Computer Science, Engineering, Machine Learning, or a related field.
Strong programming skills in Python (Go, Java, or C++ and more).
Experience instrumenting, evaluating, or operating AI systems in production.
Deep understanding of LLMs, model behavior, evaluation methods, retrieval‑augmented systems, or content moderation logic.
Strong ability to lead technical initiatives and influence cross‑functional engineering teams.
Preferred Qualifications
Experience with OpenTelemetry, Grafana, Prometheus, Loki, Tempo, or similar observability systems.
Hands‑on experience with hallucination detection, behavioral anomaly detection, or evaluation frameworks at scale.
Experience in AI safety engineering, runtime validation, or policy enforcement systems.
Understanding of hybrid architectures (device + edge + cloud).
Background guiding teams or owning cross‑functional architectural decisions.
A passion for building AI systems that are correct, safe, reliable, and deeply aligned with user expectations.
Why This Role Matters Qira’s value is defined by its
experience reliability
— the confidence that users can trust its responses, its behavior, and its safety.
In This Role, You Will
Define the standards for AI behavioral quality and correctness.
Build the systems that measure, validate, and enforce those standards.
Influence architecture, evaluation, safety, and production behavior.
Shape the AI experience for millions of Lenovo and Motorola users.
This is a rare opportunity to lead one of the most strategically important engineering functions in Lenovo’s AI roadmap.
The base salary budgeted range for this position is $190K - $230K. Individuals may also be considered for bonus and/or commission.
Lenovo’s various benefits can be found on www.lenovobenefits.com.
We are an Equal Opportunity Employer and do not discriminate against any employee or applicant for employment because of race, color, sex, age, religion, sexual orientation, gender identity, national origin, status as a veteran, and basis of disability or any federal, state, or local protected class.
#J-18808-Ljbffr
Lenovo is a US$69 billion revenue global technology powerhouse, ranked #196 in the Fortune Global 500, and serving millions of customers every day in 180 markets. Focused on a bold vision to deliver Smarter Technology for All, Lenovo has built on its success as the world’s largest PC company with a full-stack portfolio of AI-enabled, AI-ready, and AI-optimized devices (PCs, workstations, smartphones, tablets), infrastructure (server, storage, edge, high performance computing and software defined infrastructure), software, solutions, and services. Lenovo’s continued investment in world‑changing innovation is building a more equitable, trustworthy, and smarter future for everyone, everywhere. Lenovo is listed on the Hong Kong stock exchange under Lenovo Group Limited (HKSE: 992) (ADR: LNVGY).
This transformation together with Lenovo’s world‑changing innovation is building a more inclusive, trustworthy, and smarter future for everyone, everywhere. To find out more visit www.lenovo.com, and read about the latest news via our StoryHub.
About Our Team Lenovo is building
Quantum , a next‑generation hybrid AI platform spanning Windows, Android, and cloud. As part of this vision, we are expanding the engineering organization behind
Qira , Lenovo’s cross‑device Personal AI that delivers intelligent, safe, and reliable experiences across Lenovo and Motorola products.
We are hiring an
AI User Experience Reliability Lead
to define and drive the technical strategy for reliability across Qira’s AI behaviors, outputs, and runtime safety systems. This role leads the engineering direction for how Qira’s intelligence is evaluated, monitored, and improved — ensuring users experience correct, consistent, stable, and trustworthy AI behavior at global scale.
This is a critical leadership position shaping one of the most important engineering domains within Qira and Lenovo’s broader AI strategy.
Location Open to remote work in the US. The preferred work location is Chicago, IL.
What You’ll Do AI Quality Behavioral Reliability
Define and execute the strategy for measuring and improving the accuracy, stability, and task success of Qira’s AI actions.
Build and evolve evaluation frameworks, behavioral scorecards, and quality validation for models, prompts, retrievers, and task orchestration.
Develop systems to detect hallucinations, regressions, safety deviations, and other behavioral anomalies in real‑time.
Safety Guardrail Reliability
Ensure the reliability of runtime safety systems, including content moderation, jailbreak/misuse detection, safety classifiers, and policy enforcement.
Partner with Safety, Legal, Ethics, and Product teams to convert requirements into robust technical safety solutions.
Validate safety updates through testing, evaluation, and monitored deployment.
AI Observability Telemetry
Define the telemetry, metrics, traces, and data needed to understand AI behavior end‑to‑end across device, edge, and cloud.
Collaborate with observability and platform teams to integrate AI‑specific signals (quality, drift, safety events) into a unified reliability platform.
Lead the creation of dashboards and analytics that provide deep insight into AI behavior and experience reliability.
Reliability Engineering Architecture Influence
Partner with engineering, AI/ML, and product teams to embed reliability into the design of prompts, models, policies, and workflow orchestration.
Influence architecture to ensure AI behavior is predictable, testable, explainable, and resilient.
Establish standards for AI evaluation, rollout safety, service readiness, and runtime validation.
Cross‑Functional Leadership
Represent AI experience reliability in architectural reviews, product decisions, roadmap development, and launch readiness.
Drive cross‑team alignment on reliability metrics, evaluation methods, and monitoring strategies.
Collaborate with ML researchers, applied AI teams, data scientists, and UX to ensure user‑centric reliability goals.
Execution Delivery
Lead major engineering initiatives across AI quality, evaluation, safety assurance, and behavioral monitoring.
Set priorities, ensure accountability, and drive timely delivery of reliability systems and tooling.
Foster a culture of engineering excellence, learning, and continuous improvement.
Basic Qualifications
8+ years in AI/ML engineering, evaluation engineering, applied ML, reliability engineering, or large-scale distributed systems, with depth in AI behavior, evaluation, or safety.
Bachelor’s Degree in Computer Science, Engineering, Machine Learning, or a related field.
Strong programming skills in Python (Go, Java, or C++ and more).
Experience instrumenting, evaluating, or operating AI systems in production.
Deep understanding of LLMs, model behavior, evaluation methods, retrieval‑augmented systems, or content moderation logic.
Strong ability to lead technical initiatives and influence cross‑functional engineering teams.
Preferred Qualifications
Experience with OpenTelemetry, Grafana, Prometheus, Loki, Tempo, or similar observability systems.
Hands‑on experience with hallucination detection, behavioral anomaly detection, or evaluation frameworks at scale.
Experience in AI safety engineering, runtime validation, or policy enforcement systems.
Understanding of hybrid architectures (device + edge + cloud).
Background guiding teams or owning cross‑functional architectural decisions.
A passion for building AI systems that are correct, safe, reliable, and deeply aligned with user expectations.
Why This Role Matters Qira’s value is defined by its
experience reliability
— the confidence that users can trust its responses, its behavior, and its safety.
In This Role, You Will
Define the standards for AI behavioral quality and correctness.
Build the systems that measure, validate, and enforce those standards.
Influence architecture, evaluation, safety, and production behavior.
Shape the AI experience for millions of Lenovo and Motorola users.
This is a rare opportunity to lead one of the most strategically important engineering functions in Lenovo’s AI roadmap.
The base salary budgeted range for this position is $190K - $230K. Individuals may also be considered for bonus and/or commission.
Lenovo’s various benefits can be found on www.lenovobenefits.com.
We are an Equal Opportunity Employer and do not discriminate against any employee or applicant for employment because of race, color, sex, age, religion, sexual orientation, gender identity, national origin, status as a veteran, and basis of disability or any federal, state, or local protected class.
#J-18808-Ljbffr