To the OP, if the sound you're hearing (you never described the sound) is a scuffing noise, it's the rear brakes as described in the post quoted here. A couple of things you can try: when first starting out, when it's safe to do so, allow yourself to do a couple of hard, sudden stops to work the brakes. Second, when safe to do so, at a low speed apply the parking brake (it uses the rear brakes only) and allow it to stop the car.
Both of these will help scrape off the rust described by the poster.
If your car tends to set for a few days at a time (you mentioned you're retired), the brakes will be more prone to this surface rust than if it's driven on a daily basis.