Ballmer has until August 23 2014 at the helm, but the world's largest software and services company has already lined up his successor.
That man is Satya Nadella, currently the president of Microsoft's server and tools business. He joined the company in 1992, a pivotal year for the Redmond giant that saw over 1 million advance orders placed worldwide for its Windows 3.1 operating system.
But for a company of Microsoft's size and stature, being a veteran is not enough. The person who pipped heavyweight CEOs Alan Mulally (Ford), Stephen Elop (Nokia) and Hans Vestberg (Ericsson) to the post, Nadella seemingly has the DirectX factor in the eyes of the board.
The question is: What has them singing his praises?
Engineering a move
Born in Hyderabad, India in 1967, Nadella comes from an engineering background.
He gained a Bachelor of Engineering in Electronics and Communication degree from Manipal University and picked up not one, but two Master's degrees after moving to the US - a Master of Science degree in Computer Science and a Master of Business Administration degree from the University of Chicago.
His flourishing CV caught the eye of bigwigs at internet software pioneer Sun Microsystems, where he worked until joining Microsoft in 1992.
Starting out in the Windows Developer Relations group, he went on to play a helping hand in growing the company's plucky but oft-bemoaned Bing search engine and had a leading role in its now-defunct Office division (now part of the Applications and Services Engineering Group).
In July 2013, the company's "OneMicrosoft" strategy saw Nadella appointed head of the Cloud and Enterprise Engineering Group, where he was tasked with realigning cloud services including the internet-based Office 365 productivity suite.
He also took charge of enterprise-centric endeavors like data-centers, databases and development tools, which headed to the cloud under the guise of Azure, Microsoft's platform-as-a-service (PaaS) and infrastructure-as-a-service (IaaS) offering.
He didn't do badly. Microsoft says its Office 365 Home Premium, which launched in February 2013, has attracted 3.5 million subscribers in its first year. Meanwhile, Azure has seen steady growth which headed into triple figures during the company's most recent financial quarter.
Nadella is only the third CEO in Microsoft's 38-year-history after Bill Gates and Steve Ballmer. It's a huge appointment and one that Ovum analyst Roy Illsley reckons is a plus for Microsoft overall.
"His background and time served with the company are plus points," he says. "On the negative side this also makes him part of the Microsoft machine, and it would have been nice to see a fresh perspective from outside coming into the company."
Illsley also believes his appointment positions Microsoft better in the face of recent moves from its competitors.
He says: "What with IBM's announcement that it was selling x86 to Lenovo and focusing on its Softlayer cloud business, Satya is a good move from Microsoft as he heads up the cloud division. He's also come from the server and tools division so he has that 'future looking' ability coupled with an understanding of how the market delivers value to customers."
According to Illsley, Microsoft's biggest challenge is focusing on its core strengths and plotting a path to move itself from its current position to "be a force in the cloud era."
He also sees more gaming potential on the horizon following the launch of the Xbox One but thinks that desktops are a trickier area.
"While not dead, the desktop needs to be transformed so that it has the applications to deliver what enterprise and individual users want, linking the two together from a skills perspective," he adds.
Are you a pro? Subscribe to our newsletter
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!