Born in Collingwood, Victoria, Richards' passion for Collingwood grew out of family connections – he followed in the footsteps of his grandfather Charlie H. Pannam (shortened from Pannamopoulos after migrating to Australia from Greece), and uncles Charles and Alby Pannam, both former Magpie players. His brother Ron Richards also played for the club. The Richards/Pannam dynasty made Collingwood the only club to have been captained by three generations of the one family. As a family they played over 1200 games between them.
After his retirement from football, Richards managed a number of Melbourne hotels, including the well-known Phoenix Hotel in Flinders Street, whose regular customers included journalists from the nearby Herald and Weekly Times.
Richards also had a long career in the media, beginning as a sport journalist for The Argus and later The Sun News-Pictorial where he gained the nickname of "Louie the Lip". He was a very popular commentator on both radio and television with his great mate Jack Dyer. He also appeared on the popular World of Sport program. In the 1990s and 2000s, he made regular appearances on both The Footy Show and the Sunday Footy Show.
As a football tipster, Richards was known as a Kiss of Death and regularly backed-up his tips with famous dares: "I'll cut Teddy Whitten's lawn with nail scissors" or "I'll jump off St Kilda pier."
In 1972 Richards was appointed Court Jester to King of MoombaJohnny Farnham and was the King of Moomba himself in 1981.
In 1989, he released a memoir, The Kiss of Death: Memoirs of a Sporting Legend; an updated version was released in 2012, entitled Lou: My Wonderful Life.
Richards' wife, Edna, was admitted into care with dementia in 2005. She died, aged 87, in March 2008. At the end of 2008, Richards retired from hosting the handball segment on the Sunday Footy Show, and subsequently made only occasional public appearances.