He was the second son of Michael Millingen, a Dutch merchant originally from Rotterdam and then from Batavia in the western Netherlands, emigrated to England and settled in Queen's Square, Westminster (now Bloomsbury, Camden). James was born there. The physician and writer John Gideon Millingen was a younger brother.
James was educated at Westminster School, and attracted the attention of his father's friend and neighbour, Clayton Mordaunt Cracherode, who encouraged him to study numismatics. In 1790 the family migrated to Paris, where James became a clerk in the banking house of M. Van de Nyver, a connection of his mother. After the events of 10 August 1792, Mrs. Millingen with her two sons left for Calais, but the elder Millingen brought them back to Paris. James obtained a post in the French mint. There he became acquainted with Mongér, the director, a well-known mineralogist, while he made the acquaintance at the Royal (or National) Library numismatic collection of the director, the Abbé Courcy Barthélemy, and of the geographer Barbié du Bocage, and also came to know Charles Athanase Walckenaer, De Non, D'Aumont, and other archaeologists. Late in 1792 Millingen was arrested as a British subject by a decree of the National Convention, and confined first in the prison of the Madelonettes, then in that of the Luxembourg, and finally in the Collège des Écossais, where he remained until released after the Thermidorian Reaction in July 1794. At the Collège des Écossais he became acquainted with two fellow-prisoners, Charles Este, son of the Rev. Charles Este (1753–1829), and Sir Robert Smith of Beerchurch Hall, Essex.
On obtaining his liberty Millingen settled in Calais, but subsequently became a partner in the banking house of Sir Robert Smith & Co. in the Rue Céruti, Paris. The concern failed, and Millingen was thrown on his own resources. An asthma sufferer, he resided in Italy, where he compiled works on coins, medals, Etruscan vases, writing in French and Italian. He bought antiquities, and supplied most of the major museums of Europe with specimens of ancient art. He frequently offered his purchases to the trustees of the British Museum. For some time he lived at Rome and at Naples, where he made the acquaintance of Lady Blessington, but latterly settled at Florence, paying occasional visits to Paris and London. A civil list pension of 100l. a year was granted him, and he was royal associate and later honorary member of the Royal Society of Literature, fellow of the Societies of Antiquaries of London and of France, correspondent of the Institute of France (18 January 1833), and member of other learned academies of Europe.
Millingen, when on the eve of leaving from Florence to visit London, died of a severe catarrhal affection on 1 October 1845.
He married, at Calais about 1797, Elizabeth Penny, daughter of Christopher White of Calais, and had three sons, including Julius Michael Millingen, physician to Lord Byron, and a daughter. He was a staunch Anglican, and when his wife and daughter became Roman Catholics a separation between him and them followed.
- "Literary Intelligence: Notice of the late James Millingen". Classical Museum: A Journal of Philology and of Ancient History and Literature. 4: 91–95, page 91. 1847. OCLC 1554846.
- Mongér was not the director of the mint. Augustin Dupré was director of the Paris mint ( Graveur général des monnaies) from 1791 to 1803. The famous French mineralogist was Jean-André Mongez who died in 1788 in the Solomon Islands.
- Abbé Jean-Jacques Barthélemy was the curator of the numismatic collection at the time. Courcy Barthélemy was the nephew of the curator, and in 1791 went to work for his uncle. They were both arrested briefly on 2 September 1793 but spent less than 24 hours in the Madelonettes prison. "Barthélemy, Jean-Jacques". Encyclopædia Britannica. 4. 1854. p. 480.
- Classical Museum 1847, p. 93