AI User Experience Reliability Lead

About Our Team

Lenovo is building Quantum, a nextgeneration hybrid AI platform spanning Windows, Android, and cloud. As part of this vision, we are expanding the engineering organization behind Qira, Lenovo's crossdevice Personal AI that delivers intelligent, safe, and reliable experiences across Lenovo and Motorola products.

We are hiring an AI User Experience Reliability Lead to define and drive the technical strategy for reliability across Qira's AI behaviors, outputs, and runtime safety systems. This role leads the engineering direction for how Qira's intelligence is evaluated, monitored, and improved - ensuring users experience correct, consistent, stable, and trustworthy AI behavior at global scale.

This is a critical leadership position shaping one of the most important engineering domains within Qira and Lenovo's broader AI strategy.

Location: Open to remote work in the US. The preferred work location is Chicago, IL.

What You'll Do

AI Quality & Behavioral Reliability

• Define and execute the strategy for measuring and improving the accuracy, stability, and task success of Qira's AI actions.

• Build and evolve evaluation frameworks, behavioral scorecards, and quality validation for models, prompts, retrievers, and task orchestration.

• Develop systems to detect hallucinations, regressions, safety deviations, and other behavioral anomalies in real time.

Safety & Guardrail Reliability

• Ensure the reliability of runtime safety systems, including content moderation, jailbreak/misuse detection, safety classifiers, and policy enforcement.

• Partner with Safety, Legal, Ethics, and Product teams to convert requirements into robust technical safety solutions.

• Validate safety updates through testing, evaluation, and monitored deployment.

AI Observability & Telemetry

• Define the telemetry, metrics, traces, and data needed to understand AI behavior endtoend across device, edge, and cloud.

• Collaborate with observability and platform teams to integrate AIspecific signals (quality, drift, safety events) into a unified reliability platform.

• Lead the creation of dashboards and analytics that provide deep insight into AI behavior and experience reliability.

Reliability Engineering & Architecture Influence

• Partner with engineering, AI/ML, and product teams to embed reliability into the design of prompts, models, policies, and workflow orchestration.

• Influence architecture to ensure AI behavior is predictable, testable, explainable, and resilient.

• Establish standards for AI evaluation, rollout safety, service readiness, and runtime validation.

CrossFunctional Leadership

• Represent AI experience reliability in architectural reviews, product decisions, roadmap development, and launch readiness.

• Drive cross-team alignment on reliability metrics, evaluation methods, and monitoring strategies.

• Collaborate with ML researchers, applied AI teams, data scientists, and UX to ensure usercentric reliability goals.

Execution & Delivery

• Lead major engineering initiatives across AI quality, evaluation, safety assurance, and behavioral monitoring.

• Set priorities, ensure accountability, and drive timely delivery of reliability systems and tooling.

• Foster a culture of engineering excellence, learning, and continuous improvement.

Basic Qualifications

• 8+ years in AI/ML engineering, evaluation engineering, applied ML, reliability engineering, or large-scale distributed systems, with depth in AI behavior, evaluation, or safety.

• Bachelor's Degree in Computer Science, Engineering, Machine Learning, or a related field.

• Strong programming skills in Python (Go, Java, or C++ a plus).

• Experience instrumenting, evaluating, or operating AI systems in production.

• Deep understanding of LLMs, model behavior, evaluation methods, retrievalaugmented systems, or content moderation logic.

• Strong ability to lead technical initiatives and influence crossfunctional engineering teams.

Preferred Qualifications

• Experience with OpenTelemetry, Grafana, Prometheus, Loki, Tempo, or similar observability systems.

• Hands-on experience with hallucination detection, behavioral anomaly detection, or evaluation frameworks at scale.

• Experience in AI safety engineering, runtime validation, or policy enforcement systems.

• Understanding of hybrid architectures (device + edge + cloud).

• Background guiding teams or owning cross-functional architectural decisions.

• A passion for building AI systems that are correct, safe, reliable, and deeply aligned with user expectations.

Why This Role Matters

Qira's value is defined by its experience reliability - the confidence that users can trust its responses, its behavior, and its safety.
In this role, you will:

• Define the standards for AI behavioral quality and correctness

• Build the systems that measure, validate, and enforce those standards

• Influence architecture, evaluation, safety, and production behavior

• Shape the AI experience for millions of Lenovo and Motorola users

This is a rare opportunity to lead one of the most strategically important engineering functions in Lenovo's AI roadmap.