My daughters acne products ruined my towels and washcloths also. Benzoyl peroxide, as you suspected is the culprit. The only thing you can do about it is use white towels and wash cloths. God bless!!
Honestly, I just wouldn't use the good towels for wiping your face. Get a cheaper set (but still soft since they're for your face) & use them for after you wash your face. Use the good ones solely for your body & your hair. Just don't waste them & ha...
Ewwww.... Have you smelled towels which haven't been washed in a while? They absolutely reek. We are never completely clean and then bacteria breed on the damp towels after we use them. Add to that mould and mildew... mum2MH