The bigger the better... but even the smallest one will push the boat... as long as you aren't fighting a current.
But... unless you are going to places were no one else is at... I wouldn't worry. Get an orange flag, and a rope. Some one will always help.