The crack down will also include drivers using a phone that is not fitted to a cradle and will even extend to portable navigation devices. This move will become law from November 1.
Roads Minister Tim Pallas said the new laws were all about common sense and saving lives, based on one simple principle - hands on the wheel, eyes on the road.
"But the message doesn't seem to get through to everybody," said Mr Pallas. "Talking on a hand-held mobile phone while driving is reckless and increases the risk of a crash fourfold."
Drivers breaking the law face a $234 fine and three demerit points.
P-platers will get an even stricter deal with the new rules saying they cannot use a mobile phone at all whilst driving, even if they're in a hands-free set.