Born on 4 January 1773 at Quainton, Buckinghamshire, he was the son of James Lipscomb, surgeon R.N., by Mary, daughter of Jonathan George, yeoman, of Grendon Underwood in the same county. After attending schools at Quainton and Aylesbury, and receiving some medical instruction from his father, he studied surgery in London under Sir James Earle.
In 1792 Lipscomb was appointed house-surgeon at St. Bartholomew's Hospital. In 1794 he became lieutenant of the North Hants Militia, and in 1798 captain commandant of the Warwickshire volunteer infantry, for whom he wrote an Address to the Volunteers on their Duty to their King and Country. In 1798 also he was chosen deputy recorder of Warwick.
On 6 June 1806 Lipscomb obtained from Marischal College, Aberdeen, the diploma of M.D. During 1811 he became co-editor of the National Adviser, a newspaper set up by Henry Redhead Yorke. He suggested the plan of the Society for the Encouragement of Agricultural Industry in an essay, for which he received a premium and a silver medal from the Board of Agriculture.
Lipscomb's major work, The History and Antiquities of the County of Buckingham, was mainly based upon his own collections and those bequeathed to him by Edward Cooke (1772–1824). His prospectus produced a good list of subscribers. The first part appeared in 1831, after which the work had to be suspended, owing to Lipscomb's money troubles. Ultimately the publisher John Bowyer Nichols came to his aid, and before he died the last portion (pt. viii.) was in the press. The book fills four quarto volumes, with title-pages dated 1847. His other topographical works were:
- A Journey into Cornwall through the Counties of Southampton, Wilts, Dorset, Somerset, and Devon, Warwick, 1799.
- A Journey into South Wales, London, 1802.
- A Description of Matlock-bath, with an Attempt to explain the Causes of the Heat, and of the Petrifying Quality of the Springs, to which is added some Account of Chatsworth and Kedleston, and the Mineral Waters of Quarndon and Kedleston, Birmingham, 1802.
- A Journey round the Coast of Kent, London, 1818, supposed to have been compiled by L. Fussell.
- The Sandgate, Hythe, and Folkstone Guide, Sandgate, 1823.
His medical writings were:
- An Essay on the Nature and Treatment of a putrid, malignant Fever which prevailed at Warwick in 1798, Warwick, 1799.
- Observations on the History and Cause of Asthma, and a brief Review of a Practical Inquiry into disordered Respiration, London, 1800.
- Inoculation for the Small-pox vindicated, London, 1805.
- A Manual of Inoculation for the use of the Faculty and Private Families … extracted from the Writings of Dimsdale, Sutton, etc., London, 1806.
- A Dissertation on the non-Infallibility of the Cow-pox, with an Examination of the principal Arguments of Drs. Jenner, Pearson, Woodville, Lettsom, Adams, and Thornton, London, 1806.
- Cow-pox exploded, or the Inconsistencies, Absurdities, and Falsehoods of some of its Defenders exposed, London, 1806.
- A Dissertation on the Failure and Mischiefs of the Cow-pox, London, 1806.
- Cautions and Reflections on Canine Madness, with the Method of preventing the Hydrophobia in Persons who have been bitten, London, 1807.
- A History of Canine Madness and Hydrophobia, London, 1809.
- Observations on Contagion as it relates to the Plague and other epidemical Diseases, and refers to the Regulations of Quarantine, London, 1819.
- A Grammar of Medicine, with Plan of the Grammar of Chemistry.
- In 1832 Lipscomb delivered a series of lectures on cholera at the London Mechanics' Institution and the North London Literary and Scientific Institution. He published them as On the Nature, Symptoms, Treatment, and Cure of Cholera Morbus, with preliminary Remarks on Contagion and the Regulations of Quarantine, accompanied by his correspondence with Lord Melbourne on the subject.
Other works included:
- The Grey Friar, or the Black Spirit of the Wye, London, 1810.
- Modern Times, or Anecdotes of the English Family.
- The Capricious Mother.
- Observations on the High Price of Provisions and the Monopoly of Farms.
Articles by Lipscomb appeared in the Gentleman's Magazine, mainly under the signature of "Viator"; and his essays on subjects connected with political economy, statistics, and general literature were published in the Literary Panorama and other periodicals. He edited the Clerical Guide for 1821, and published two volumes of Sermons, also writing them for clergymen. He composed hymns and anthems for charity schools on particular occasions.
Lipscomb married, in 1803, Sarah, the widow of Richard Hopkins of Stratford-upon-Avon, Warwickshire, and third daughter of Thomas Wells, also of Stratford. They had no children. On his wife's death in 1834 her fortune, on which Lipscomb mainly depended, passed back to her own family.
- Lee, Sidney, ed. (1893). "Lipscomb, George". Dictionary of National Biography. 33. London: Smith, Elder & Co.
- Pickford, Chris. "Tattam, Henry". Oxford Dictionary of National Biography (online ed.). Oxford University Press. doi:10.1093/ref:odnb/26994. (Subscription or UK public library membership required.)
- Lee, Sidney, ed. (1893). "Lipscomb, William". Dictionary of National Biography. 33. London: Smith, Elder & Co.
- Lee, Sidney, ed. (1895). "Nichols, John Bowyer". Dictionary of National Biography. 41. London: Smith, Elder & Co.