It may be worth trying this stuff: Everbuild PVCu Solvent Cleaner. I find that it helps with my fenders when a water-based approach and scouring pad are ineffective. It is available from ToolStation among others.
+1 for PVCu cleaner but it does take off the original sheen/gloss, as does a scouring pad. After the cleaner I use a clear furniture polish, Pledge or similar.
We find all fenders, regardless of colour pick up stains. Sometimes impossible to remove. Instead we make cheap fleece Fender socks. They look nice and give some protection to topsides as long as you don't get them dirty. There is a risk that they will pick up grit if used on dirty concrete walls, etc but we use a rash pad for that to protect the hull.