In the early 1780s, Caleb emigrated with his brother, spending some time in York, Pennsylvania and then moved to Leesburg, Virginia in 1786. While in York, Bentley became a Quaker. In the early 1790s, Bentley established himself as a silversmith in Georgetown, Washington, D.C. Bentley was commissioned by President George Washington to make the brass cornerstone used for the White House groundbreaking ceremony in 1792. A year later, Bentley made a silver cornerstone which was used for the United States Capitol.
Bentley relocated to Montgomery County, Maryland in 1794, settling in Brookeville, where his wife's (Sarah Brooke) family owned a large tract of land. In Brookeville, Bentley opened a store and a post office in 1802, becoming the First Postmaster of Brookeville. His first wife died in 1805, and two years later, he remarried to Henrietta Thomas. Bentley was also a founder of the Brookeville Academy, and along with two brothers-in-law, established the town of Triadelphia, on the Patuxent River.
Bentley's wife, Henrietta Thomas, was a close friend of Dolley Madison. In August 1814, during the War of 1812, the White House was set ablaze by British troops during the Burning of Washington, in retaliation for burning Upper Canada's Parliament Buildings in the Battle of York. On August 26, 1814, President James Madison fled Washington, D.C., initially going to Virginia, but then turning north towards Rockville, where he expected to find General John Henry Winder and his troops. Winder's troops had already proceeded on towards Baltimore. Madison continued on eastward and arrived in Brookeville on horseback, where he found refuge in the home of Caleb Bentley. Madison stayed up all night dispatching orders, with Brookeville becoming the temporary capital of the USA.
- Farquhar, Roger Brooke (1952). Old Homes and History of Montgomery County, Maryland. Judd & Detweiler, Inc. pp. 207–208.
- Allen, William C. (1995). In The Greatest Solemn Dignity — The Capitol's Four Cornerstones. Government Printing Office. pp. 12–13.
- Allen, William C. (1995). In the Greatest Solemn Dignity — The Capitol's Four Cornerstones. Government Printing Office. p. 7.
- Allen, William C. (2001). History of the United States Capitol — A Chronicle of Design, Construction, and Politics. Government Printing Office. p. 24. Archived from the original on 2008-12-16. Retrieved 2008-12-12.
- Siegal, Ann Cameron (2007-12-01). "Charms and Challenges Of a Tiny Old Town". The Washington Post.
- "The East Room". White House Historical Association. Archived from the original on 2007-10-07. Retrieved 2007-11-02.
- Landphair, Ted (2007-08-15). "You Won't Need Much Time to Tour This 'Capital'". VOA News. Archived from the original on 2007-10-13.
- "Brookeville: Gem from the Past". Sandy Spring Museum. Archived from the original on 2007-10-07. Retrieved 2007-12-14.
- Browning, Charles Henry (1895). "President Madison's Retreat". The American Historical Register. Historical Register Pub. pp. 857–861.