The resignations followed a speech by Putin, in which he proposed amending the constitution to increase the powers of prime ministers and Cabinet members.
The maneuver could prove crucial for Putin's future.
Putin, 67, said the government hasn't fulfilled all of its tasks, though he thanked it for its work.
"Together we will certainly change life for the better".
"In this context, it's obvious that we, as the government. should provide the president of our country with the opportunity to make all the decisions necessary for this".
Russian Prime Minister Dimitry Medvedev, shortly after Putin's state of the nation address, announced that his government was resigning to give Putin room to carry out constitutional amendments.
Putin's key proposal is to transfer the power to select the prime minister and cabinet from the president to the parliament, or duma.
The role of governors would also be enhanced, he said, though Russian Federation would maintain its presidential system. "People want development, they are striving to move forward in their careers, in their education, in becoming prosperous".
Putin emphasized that constitutional changes must be put to a nationwide vote.
Putin has led Russia for over 20 years, longer than any Russian leader since Josef Stalin.
He obediently stepped down after just one term as president and let Putin reclaim the top job in what was widely seen as cynical political maneuvering and triggered massive protests in Moscow.
Mr. Putin proposed amending the Russian Constitution to expand the powers of Parliament and a body called the State Council, which now carries little weight.
Under the proposals, the prime minister would present parliament with candidates for the country's deputy prime ministers and government ministers, which parliament would also confirm.
According to the current Constitution, adopted in 1993, the government exercises executive power; it drafts and implements the federal budget and carries out the national policy in the sphere of finance, foreign policy, education, healthcare, culture, science, and so on.
Basically, Putin has removed Medvedev as PM & will take all the decisions DM would normally have taken; he's asked ministers to stay in place until there's a new cabinet.
"In a nutshell, we take this announcement as an attempt by Putin to shake up Russia's polity and refocus the administration on implementing the president's well-telegraphed but slowly progressing public spending program", Citi said in a note.