Gilbertson grew up in Elerch, Cardiganshire before studying at Jesus College, Oxford. He obtained his BA degree in 1836, his M.A. in 1839 and his BD in 1847. He was appointed a Fellow of Jesus College in 1840 and retained this post until 1872. He was ordained deacon in 1837 and priest in 1838, becoming curate of Sheringham near Cheltenham, Gloucestershire. Gilbertson was vicar of Llangorwen, Cardiganshire from 1841 to 1852, becoming known for a connection witih the Oxford Movement. In 1852, he returned to Oxford where he served as junior bursar and lecturer at Jesus College. He became Vice-Principal in 1855, when Henry Foulkes was the Principal. Foulkes died in 1857 and was succeeded by Charles Williams. Gilbertson continued as Vice-Principal until 1872. During his time at the college, he tried (unsuccessfully) to move the college towards Anglo-Catholicism, and he left his mark on the chapel renovation of 1864.
He became vicar of his home village of Elerch (which was a new parish) in April 1869, but exercised his right as patron to appoint a successor in November 1870, becoming rector of Braunston in Northamptonshire (a position within the gift of Jesus College) that same year. He was rector of Braunston until retiring to Aberystwyth in 1893, where he died on 2 April 1896. His widow had a stained-glass window erected in the college chapel in his memory, which includes a picture of Archbishop William Laud.