He was born on 29 January 1774 at Yaxley in Huntingdonshire. Having been educated by Richard Weston, a Leicester botanist, in 1793 he published a treatise, Lessons Astronomical and Philosophical. Having settled at Cambridge in 1796, Gregory first acted as sub-editor on the Cambridge Intelligencer, and then opened a booksellers shop. In 1802 he obtained an appointment as mathematical master at the Royal Military Academy, Woolwich through the influence of Charles Hutton, to whose notice he had been brought by a manuscript on the Use of the Sliding Rule; and when Hutton resigned in 1807 Gregory succeeded him in the professorship. Failing health obliged him to retire in 1838, and he died at Woolwich on 2 February 1841.
Gregory's books included Hints for the Use of Teachers of Elementary Mathematics (1840, new edition 1853), and Mathematics for Practical Men (1825), which was revised and enlarged by Henry Law in 1848, and again by J. R. Young in 1862. His Letters on the Evidences of Christianity (1815) have been several times reprinted, and an abridgment was published by the Religious Tract Society in 1853. He will probably be longest remembered for his Biography of Robert Hall, which first appeared in the collected edition of Hall's works, was published separately in 1833, and has since passed through several editions. The minor importance of his Memoir of John Mason Good (1828) is due to the narrower fame of the subject.
He was one of the founders of the Royal Astronomical Society. In 1802 he was appointed editor of the Gentlemen's Diary, and in 1818 editor of the Lady's Diary and superintendent of the almanacs of the Stationers Company. Gregory was a co-editor of Pantologia, 1813.
Gregory's eldest son, Boswell was accidentally drowned in the Thames in the summer of 1834. Letitia Landon's poem 'Olinthus Gregory L.L.D., F.R.A.S. &C.' commiserating with this event was published in Fisher's Drawing Room Scrap Book, 1835. Gregory's other son, Charles Hutton Gregory, later became president of the Institution of Civil Engineers.