He was educated at Winchester College and Balliol College, Oxford. He then became a journalist and writer. During the Second World War he served with the Royal Air Force and from 1944-47 was First Secretary at the British Embassy at Cairo.
In 1938, he married Angela Mary Culme-Seymour (1912-2012), daughter of George Culme-Seymour and Janet (née Orr-Ewing) and former wife of the artist John Spencer-Churchill. Having been separated by World War II when Balfour was posted to Cairo, she started a five-year relationship with Major Robert Hewer-Hewitt by whom she had two sons, Mark and Johnny. Patrick and Angela were divorced in 1942. After breaking up with Major Hewer-Hewitt in 1946 Angela married a French count, René de Chatellus, and moved to France with her two sons. Her grandmother, Trix Ruthveen, was the model for "the bolter" in Nancy Mitford's novel The Pursuit of Love.