Senior AI Hardware Architect

Do you want to be at the forefront of innovating the latest hardware designs to propel Microsoft's cloud growth? Are you seeking a unique career opportunity that combines technical capabilities, cross-team collaboration, with business insight and strategy?

Microsoft's mission is to empower every person and every organization on the planet to achieve more. As employees, we come together with a growth mindset, innovate to empower others, and collaborate to achieve our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. In alignment with our Microsoft values, we are committed to cultivating an inclusive work environment for all employees to positively impact our culture every day.

Join the Systems Planning and Architecture (SPARC) team within Microsoft's Azure Hardware Systems and Infrastructure (AHSI) organization, the team behind Microsoft's expanding Cloud Infrastructure and for powering Microsoft's "Intelligent Cloud" mission. Microsoft delivers more than 200 online services to more than one billion individuals worldwide, and AHSI is the team behind our expanding cloud infrastructure. We deliver the core infrastructure and foundational technologies for Microsoft's cloud businesses including Microsoft Azure, Bing, MSN, Office 365, OneDrive, Skype, Teams and Xbox Live.

We are seeking a Senior AI Hardware Architect to join the AI Systems Architecture (ASA) group, where we define, analyze, and optimize next-generation AI accelerator platforms and large-scale inference and training systems. In this role, you will lead performance analysis, profiling, kernel-level optimization, and end-to-end performance characterization across GPU and accelerator architectures, working across hardware, software, and system boundaries.

You will analyze real-world AI workloads across modern GPU platforms and in-house AI accelerators, identifying performance bottlenecks and architectural trade-offs through rigorous measurement and benchmarking. A key aspect of this role is correlating on-silicon measurements, software traces, and kernel execution behavior with architectural models and simulators, enabling deep insight into performance behavior and guiding data-driven architectural decisions

You will collaborate closely with architecture, microarchitecture, compiler, runtime, and systems teams, and contribute to the development of data correlation, analysis, and visualization tools that improve performance insight and optimization velocity. Through quantitative analysis and cross-platform understanding, you will play a critical role in shaping future accelerator and system architectures across the AI hardware and software stack.

• Lead performance analysis, profiling, and benchmarking across GPU and in-house AI accelerator architectures, applying rigorous data and statistical analysis to identify complex performance bottlenecks, root causes, and optimization opportunities across hardware, software, and system layers.
• Run and analyze end-to-end AI models on production-like serving infrastructure, performing deep dives into modern AI serving stacks (e.g., optimized LLM serving frameworks, schedulers, runtimes, and memory management systems) to understand performance behavior, scalability limits, and system-level trade-offs.
• Provide data-driven recommendations and architectural trade-offs to senior technical leadership, balancing performance, complexity, cost, quality, reliability, and development timelines to inform accelerator and system architecture decisions.
• Develop and implement technical solutions to complex performance, quality, and design challenges, including kernel-level optimization, architectural tuning, and system-level performance improvements across multiple products or feature areas.
• Correlate on-silicon measurements, software traces, and kernel execution behavior with architectural models and simulators, ensuring alignment between measured performance and architectural intent, and identifying gaps that drive future design enhancements.
• Design, build, and evolve data correlation, analysis, and visualization tools and workflows that scale performance insight, accelerate debugging, and improve clarity and communication of optimization opportunities across teams.
• Lead and contribute to design and performance documentation, including architecture reviews, performance reports, functional specifications, and customized analyses; communicate progress, risks, and recommendations within and across teams, and help identify and mitigate significant project risks.

Required Qualifications:

• Master's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 3+ years technical engineering experience OR Bachelor's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 5+ years technical engineering experience OR equivalent experience.

Other Requirements

Ability to meet Microsoft, customer, and/or government security screening requirements for this role. These requirements include, but are not limited to, the following specialized security screenings:

Microsoft Cloud Background Check: This position requires passing the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred Qualifications

• Doctorate in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 3+ years technical engineering experience OR Master's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 6+ years technical engineering experience OR Bachelor's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 8+ years technical engineering experience OR equivalent experience.
• MS or PhD in Machine Learning, Computer Architecture/Systems, Electrical Engineering, High-Performance Computing, or related areas.
• 4+ years of experience in Computer Architecture, AI Systems, or closely related technical domains.
• MS or PhD in Machine Learning, Computer Architecture/Systems, Electrical Engineering, High-Performance Computing, or related areas.
• Experience with GPU and AI accelerator architectures, including compute pipelines, memory hierarchies, interconnects, and parallel execution models.
• Demonstrated expertise in performance profiling, benchmarking, and root-cause analysis, using hardware performance counters, software traces, and workload-level measurements.
• Hands-on experience with kernel-level performance analysis and optimization, and correlating kernel behavior with architectural and system-level performance.
• Strong programming and scripting skills in Python and C/C++ for performance analysis, tooling, benchmarking, and automation.
• Experience with architectural modeling or simulators and correlating modeled behavior with measured hardware performance.
• Experience running and analyzing end-to-end AI models on serving or training infrastructure, with the ability to diagnose performance issues across hardware, runtime, and system layers.
• Hands-on experience with AI frameworks and runtimes, including PyTorch, and familiarity with modern AI serving stacks such as vLLM and SGLang frameworks.
• Ability to communicate complex technical concepts clearly through design documentation, performance reports, functional specifications, and technical presentations.

Hardware Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.