Born in Halifax, Nova Scotia, the son of William Bruce Almon Ritchie and Lillian Stewart, Ritchie was a scion of prominent families — the Almons, Ritchies, and Stewarts were all major families in Nova Scotia. Ritchie's great-uncle, Sir William Johnstone Ritchie, had also been on the Supreme Court, serving as a puisne justice and then as the second Chief Justice of Canada. His brother, Charles Ritchie was an important Canadian diplomat and diarist.
After the war he helped found the law firm, Daley, Phinney & Ritchie. He was a lecturer on insurance law at Dalhousie University, and acted as counsel to the royal commission on the terms of Newfoundland's union with Canada in 1949.
In 1959, without any previous judicial experience, Ritchie was appointed by the Diefenbaker government to replace Ivan Rand on the Supreme Court of Canada.
Ritchie's judgements were typically conservative, which often put him on side with Ronald Martland and Wilfred Judson. He is best known for a pair of conflicting decisions concerning the Canadian Bill of Rights: R. v. Drybones and Attorney General of Canada v. Lavell. In Drybones, Ritchie wrote the majority decision for the Court, holding that a provision of the Indian Act was inoperative because it conflicted with the Canadian Bill of Rights. However, in Lavell, Ritchie wrote the majority decision holding that a federal statute such as the Indian Act could not be held inoperative because of the Bill of Rights.
One of his most significant dissents, co-authored with Justice Martland, was in the Patriation Reference, where they argued that as a matter of constitutional law, the federal Parliament did not have the authority to unilaterally request that the British Parliament enact the proposed patriation constitutional amendments. Although in dissent on the legal issue, Martland and Ritchie were in the majority on the second issue in the Reference, the existence of a constitutional convention which required a significant degree of provincial support for major constitutional amendments.
He served on the Court until his retirement in 1984. He retired due to poor health at age 74, a year before the mandatory retirement age of 75. In 1985 he was made a Companion of the Order of Canada. From 1974 to 1988, he was the Chancellor of the University of King's College.