It was scratched and sticky, and when I tried to clean it with a MagicEraser + Isopropyl Alcohol it only got worse.

After scratching my head a few times, I figured I got the solution!
I grabbed an (expired) credit card and used that to scrape all the rubber paint off.
After trying different ways, I found the best one was pushing the card at an angle of ca. 30 degrees away from me.
Nice rubber fluff was coming off in no time.
After most of the rubber was removed, I used a paper towel whetted with Isopropyl Alcohol to clean off the rest.
The result?
I'd say magnifico!
Judge for yourself: http://imgur.com/a/sDtfw
Scroll down there to see all the pix.