After his movie career ended Frank was forced to do children's parties