While it has been illegal to use a handheld phone at the wheel since 2003, using a hands-free device creates "the same risks of collision", the Commons Transport Select Committee said.
Drivers could be banned from using hands-free mobile phones in England and Wales, a group of MPs has suggested.
Alongside the possible new ban, MPs have called for more severe penalties for handheld phone use.
However, trying to ban hands-free devices opens a can of worms, not least in terms of enforcement.
"One moment's distraction from a phone can cause a lifetime of suffering so our advice to drivers is simple - when you're driving, make sure your phone is on silent and placed out of sight and out of reach", he said.
"There is a misleading impression that hands-free use is safe".
They cited data that showed in 2017 there were 773 road casualties, including 43 deaths and 135 serious injuries, in collisions where a driver using a mobile phone was a contributory factor.
However, the Committee said these penalties "still do not appear to be commensurate with the risk created and should be reviewed and potentially increased so that it is clear there are serious consequences to being caught".
Labour MP Lilian Greenwood, who chairs the committee, said "any use of a phone distracts from a driver's ability to pay full attention" to the road.
It has been illegal in the United Kingdom since 2003 to use a handheld phone whilst driving, but the committee is anxious that current laws give the "misleading impression" that hands free options are safe. But MPs want this extended to the use of hands-free devices.
RAC head of roads policy Nicholas Lyes said: "We support the committee's call for the Government to look more closely at the effectiveness of the increase of the penalties in 2017, and key to this is whether enforcement is adequate and whether the police have sufficient resources and technology to be able to crack down on this scourge".
"For example, a very short call saying that you are going to be late will stop people from speeding or driving in a risky manner".
The report has been welcomed by motoring and road safety organisations, including Brake, which has urged the Government to act on the recommendations as a priority, saying that stronger laws and tougher enforcement are needed to make mobile phone use as culturally unacceptable as drink driving. "The law is clear that anyone driving dangerously is committing a criminal offence".