Born in Leamington Spa, Warwickshire, England, in 1915, he did not become an artist until he was in his 30s. During World War II, he served in France and the Middle East, before joining the commandos. Whilst serving with the commandos in Crete in June 1941 he was captured and became a prisoner of war. As a prisoner at Stalag 383 in Bavaria, he met and was taught by Adrian Heath. He said of his prison experience that it was a ‘tremendous spiritual experience, a more aware or heightened perception during starvation’.
Frost's academic career included teaching at Bath Academy of Art, the Cyprus College of Art and the University of Leeds. Later he became Artist in Residence and Professor of Painting at the Department of Fine Art of the University of Reading.
He married Kathleen Clarke in 1945. They had five sons and one daughter. His sons Adrian and Anthony also became artists while another son, Stephen, is a comedian and actor. His grandson Luke is also an actor.