Which Pixels Are Really Representative of Reality?