Richards began playing classical piano at age ten, and concentrated on jazz from age 16 after hearing Fats Waller. His first major professional gig was with Tab Smith at New York's Savoy Ballroom from 1945 to 1949. following this he played with Bob Wilber (1950–51) and Sidney Bechet (1951). He toured Italy and France in 1953 with Mezz Mezzrow's band alongside Buck Clayton and Big Chief Moore, also accompanying Frank Sinatra during his time in Italy. He played with Muggsy Spanier on and off from 1953 through the end of the decade, and with Fletcher Henderson in 1957-58. In 1958 he did some time as a solo performer in Columbus, Ohio, then played with Wild Bill Davison in 1958-59 and again in 1962.
In 1960 he formed Saints & Sinners with Vic Dickenson, playing with this ensemble until 1970. He joined Jazz drummer Chuck Slate and his band in 1971 and stayed with him most of the year. He recorded an album with Chuck called "Bix 'N All That Jazz" Following this he did work with Eddie Condon (1975–77), then played with his own trio in 1977-78. He played with Panama Francis's group, the Savoy Sultans, worldwide from 1979 through the 1980s. He recorded with Bill Coleman in 1980. He continued to tour almost up until the time of his death.
- Soft Buns (West 54, 1978)
- In a Mellow Tone (West 54, 1978)
- It's a Wonderful World (Black & Blue, 1980)
- Lullaby in Rhythm (Sackville, 1985)
- Dreamy (Sackville, 1991)
- My Romance (Jazz Point, 1993)
- Swing Time (Jazz Point, 1993)
- Groove Move (Jazz Point, 1994)
- Echoes of Spring (Sackville, 1998)
With Pee Wee Russell
- The Individualism of Pee Wee Russell (1952)
- Doran/Kernfeld, "Red Richards". Grove Jazz online.