Eraser wheels are awesome, but they can be pricey. Should work really well on smooth surfaces. Not sure how well they will hold up if you have any non-skid surfaces.
Definitely the cleaner way to try to remove what doesn't simply peel off and it certainly wouldn't hurt to try it.