By 1975, Hayes left Stax Records and formed his own label called Hot Buttered Soul Records. A series of unsuccessful albums led to Hayes' bankruptcy in 1976. The late 1970s saw a major comeback for Hayes, following the release of A Man and a Woman (1977, with Dionne Warwick). In spite of moderate success as a singer, Hayes' records did not sell very well. By the late 1990s, he was best known as the voice of Chef on South Park. A song of his from the series, Chocolate Salty Balls, received some international radio airplay in 1999 and subsequently appeared on the album South Park Chef Aid in 2002.
Hayes was inducted into the Rock 'n Roll Hall of Fame in 2002.