In 1862 during the American Civil War, much of the land around Point Lookout was transformed into a bustling port, temporary city of civilians and militarypersonnel and numerous buildings, a large army hospital, an army garrison at Fort Lincoln, and a Unionprisoner of war camp to hold Confederate captives. Of the 50,000 soldiers held in the army prison camp, who were housed in tents at the Point between 1863 and 1865, according to the Maryland Department of Natural Resources, nearly 4,000 died, although this death rate of 8 percent was less than half the death rate among soldiers who were in the field with their own armies. The camp, originally built to hold 10,000 men, swelled to between 12,000 and 20,000 prisoners after the exchange of prisoners between armies was placed on hold. The result was crowded conditions with up to sixteen men to a tent in poor sanitary conditions. By far, it was the largest Union-run prison camp in the North and also, one of the worst during the war. Today, a Confederate prisoner of war memorial and cemetery exist on the former grounds of the Point Lookout Prisoner of War Camp. Because of the extensive water erosion of the Chesapeake Bay shoreline, in the last 150 years, half of the original site of the prisoner of war stockade has been obliterated and washed away by the bay.
Today Point Lookout is a Maryland State Park and retains Point Lookout Light, the original light house built in 1830, a fishing pier, boat launch facilities, public beaches and facilities, overnight camping, Civil War historical remains, and, reputedly, ghosts. The Civil War Museum/Marshland Nature Center is open Monday, Wednesday, Thursday and Friday from 9 A.M until 5 P.M. and Saturday and Sunday from 10 A.M. until 6 P.M. until the end of October.
The park is used as the setting in the Point Lookout add-on for the video game Fallout 3, which includes references to its use as a POW camp as well as various features of the park, including Calvert's manor and the lighthouse.