The reason for the legation's creation had much to do with anti-Asian feeling in the Canadian province of British Columbia during the first half of the 20th Century. Prime Minister Mackenzie King was anxious to limit Japanese migration to Canada, saying "our only effective way to deal with the Japanese question is to have our own Minister in Japan to visa passports.".
The British government was hesitant to anything that might be seen to undermine Imperial unity, but finally in May 1929, the Canadian legation opened. The first "minister" was Sir Herbert Marler. The embassy soon added trade and political roles to immigration. Construction of the chancery was completed in 1934.
In 1938 the minister came back to Canada without being replaced. In 1941 once Canada and Japan were at war the legation staff was placed under arrest and not repatriated to Canada until mid-1942.
In 1952 Canada and Japan had normalized relations and the legation was upgraded to an embassy, and R.W. Mayhew became Canada's first ambassador to Japan.
Canada built a new chancery on Aoyama Avenue, Place Canada, which was designed by Raymond Moriyama and opened in 1991. The embassy is housed on the upper levels while the lower levels are let out for rental income. There is a stone garden at the fourth storey with a view of the Akasaka Palace gardens. At the basement level the embassy hosts a public art gallery, a library, and an auditorium.