So long as it doesn't endanger me, I'm perfectly at ease.
It might suck to be stuck behind someone going slow . . . especially when you have a car with lots of ponies that you want to unleash with your right foot. That said, the slow driver isn't adding hours to your journey. It's likely mere minutes (or even seconds). If you're cutting it close, you should've left earlier.