Finlayson worked under Charles Ross in the building of Fort Victoria in 1843 to 1844. Upon Ross's death, which he mentions with sympathy in his autobiography, Finlayson became Chief Factor. After James Douglas arrived in 1849 Finlayson was reassigned to second in command. Also in 1849 he married Sarah Work, one John Work's Metis daughters. Finlayson was one of the few to see Victoria grow from bare ground to the Capital of British Columbia.