Cadogan was born into a middle-class family in Newcastle upon Tyne, where his father was employed by a shipping company. He was educated at The King's School, Tynemouth in the 1930s. After working as an insurance clerk, he joined Royal Air Force Air Sea Rescue in 1941, in which he served until 1946. On his demobilisation, he joined the Communist party, drawing inspiration from its historians' group, which included Christopher Hill, Eric Hobsbawm and E.P. Thompson.
He married Joyce Stones in 1949.
He studied history at King's College, Durham and taught in Northampton and Cambridge.
Cadogan disliked the Communist party's authoritarian style and support of the Soviet Union. He was suspended in 1956 for publicly criticising the party's failure to denounce the Soviet suppression of the Hungarian Uprising. He then joined the Labour Party.
Influenced by Trotskyist ideas, he took part in the 1959 founding conference of the Socialist Labour League (SLL) under the leadership of Gerry Healy. He was expelled from the Labour party when it added the SLL to its list of proscribed organisations in 1959, and expelled in turn from the SLL, whose leadership style he found to be no different from that of the Communist party. In 1960 he joined the editorial board of the Trotskyist publication International Socialism (IS) and contributed subsequently to its more populist paper Labour Worker (now Socialist Worker), only to be expelled from that group as well.
He was national secretary of the anti-nuclear Committee of 100 in the 1960s.
He became an advocate of the breakaway state of Biafra during the Nigerian civil war. In 1968 he set up the Save Biafra Campaign
He was chairman of the South Place Ethical Society from 1970 to 1981. He took the controversial decision, on the grounds of freedom of speech and despite his hatred of fascism, to permit the British National Front to meet at the society's premises.
Cadogan was a long-standing member of the Blake Society and served as both its chairman and president.
He wrote a book on direct democracy in 1974 and many pamphlets and articles. He was a founder of New Consensus/New Dialogue in 1990, co-founder of Values and Vision, 1991, and chairman of the London Alliance for Local Democracy from 1998.