Waller lived in Stroud, Gloucestershire, for more than seven of his final years. He was well known as a local busker as the "Man in the Red Hat". He recorded an album of his own songs, plus lyrics contributed by Melanie White aka Ruth Whitney ("Qualified" and "Checking Out the Bins"), at the DB Studios in Stroud. His last album was called Last Tracks to Freedom. He collaborated on albums of Melanie White's stories, for example The Cardboard Cut-Out, for which he wrote a song.