Is the Reward Written in Invisible Ink?