On May 30, 1963, Spence was appointed to the Supreme Court. In 1966, he chaired the royal commission investigating the Munsinger Affair, which involved allegations that an East German spy had been sleeping with Canadian Cabinet ministers. Spence criticised the way the Diefenbaker government had handled the matter, but found no evidence of criminal wrongdoing or security breaches. In the 1970's, Spence frequently joined with Chief Justice Bora Laskin and Justice Brian Dickson on cases involving civil liberties, often in dissent. The grouping was colloquially referred to as the "LSD connection."
Spence retired from the Court on December 29, 1978. In 1979, he was made a Companion of the Order of Canada.