He was born at Kelvedon Hatch, Essex, on 25 October 1883, the only son of a retired army officer, Arthur Stronge Gilbert, and Melvina (daughter of the Raja of Kapurthala). He attended Cheltenham and Hertford College, Oxford, taking a first in Classical Moderations. Following this, he joined the Indian Civil Service in 1907, and, after military service in World War I, served as a judge in Burma until 1925. He then retired, settling in France with his French-born wife Moune (née Marie Douin). He remained there for the rest of his life, excepting some time in Wales during World War II.
Gilbert was one of the first Joycean scholars. He read Ulysses whilst in Burma, and admired it greatly. His wife gives an account of his introduction to Joyce personally: While she and Gilbert were taking a walk in the Latin Quarter of Paris they passed Shakespeare and Company. Some typescript pages of a French translation of Ulysses by Auguste Morel and Valery Larbaud were being advertised in the window, and Gilbert noted that there were several serious errors in the French rendering. He introduced himself to Sylvia Beach, and she was impressed with his criticisms of the translation. She took his name and telephone number, and suggested that Joyce (who was assisting in the translation) would contact him. This began many years of friendship between Joyce and Gilbert. He published James Joyce's Ulysses: A Study in 1930 and published a collection of Joyce's letters in 1957.
One of his major projects was the translation from French of Roger Martin du Gard's novel sequence entitled Les Thibault. Running to nearly 1,900 pages in translation, it was published by the Viking Press in the US in two volumes, The Thibaults (1939) and Summer 1914 (1941).