He was the son of Edward Fernow, a royal councillor, and Bertha de Jachmann, sister of Eduard von Jachmann, a vice admiral in the Prussian service. He was educated at the royal gymnasium of Our Lady at Magdeburg, Saxony, and at a similar institution at Bromberg, Posen, in 1858.
He then emigrated to the United States, served during the American Civil War as lieutenant of the 3rd U. S. colored troops, and subsequently as topographical engineer. For a time, he was one of the state librarians, keeper of the historical records at Albany, New York, and the editor of the 12th, 13th, and 14th volumes of Documents Relating to the Colonial History of New York (Albany, 1877–85), State Archives (vol. i, 1887); and Albany and its Place in the History of the United States (1887). Fernow also contributed to Wilson's History of New York (4 vols., 1892–93), and edited Records of New Amsterdam (7 vols., 1897–98). He wrote many magazine articles on historical subjects.