The site was selected partially due to its proximity to the Erie Railway and the Northern Central Railway, which criss-crossed in the midst of the city, making it a prime location for a Union Army training and muster point early in the Civil War. Most of the 30-acre (120,000 m2) Union installation, known as Camp Rathbun, fell into disuse as the war progressed, but the camp's "Barracks #3" were converted into a military prison in the summer of 1864. The prison camp, in use from July 6, 1864, until the autumn of 1865, was dubbed "Hellmira" by its inmates. Towner's history of 1892 and maps from the period indicate the camp occupied an area running about 1,000 feet (300 m) west and approximately the same distance south of a location a couple of hundred feet west of Hoffman Street and about 35 feet south of Water Street, bordered on the south by Foster's Pond, on the north bank of the Chemung River.:265
During the 15 months the site was used as a prisoner of war camp more than 12,100 Confederate soldiers were incarcerated there; of these, nearly 25% (2,963) died from a combination of malnutrition, continued exposure to harsh winter weather, and disease from the poor sanitary conditions on Foster's Pond combined with a lack of medical care. The camp's dead were prepared for burial and laid to rest by the sexton, an ex-slave named John W. Jones, at what is now Woodlawn National Cemetery. At the end of the war, each prisoner was required to take a loyalty oath and given a train ticket home. The last prisoner left the camp on September 27, 1865. The camp was then closed, demolished and converted to farm land.
Woodlawn Cemetery, about 2 miles (3.2 km) north of the original prison camp site (bounded by West Hill, Bancroft, Davis, and Mary streets), was designated a National Cemetery in 1877. The prison camp site is a residential area today, and few of the city's residents are aware that the prison camp ever existed. However, there is a memorial at the site today.
Like many prison camps built hastily in the Civil War to deal with the demands created by the capture of enemy soldiers, Elmira suffered from poor health conditions and poor planning early on. The Camp was originally designed as a training barracks. Camp Rathbun, as originally designed, could support up to two thousand soldiers. As the years passed and Camp Rathbun was used less and less as a training facility, plans to turn it into a prison camp were made. On May 2, 1864, Lieutenant Colonel Seth Eastman, the officer in charge of Camp Rathbun at the time, reported to Brigadier General Lorenzo Thomas that twenty barracks had been constructed and that this, in addition to repairs on old barracks, allowed Elmira to support up to 6,000 troops. On May 19, 1864, Eastman was informed via letter from Commissary General William Hoffman that he was to "set apart the barracks on the Chemung River at Elmira as a depot for prisoners of war." He was also informed that the prison might be needed within ten days and that it might have to welcome 8,000 or up to 10,000 prisoners. According to Eastman's calculations, the camp could only hold half that properly. While the barracks were well kept, they could only house four thousand men, and perhaps another thousand could be kept in tents. In addition to that, Eastman reported that the kitchens could only feed five thousand a day and the mess room could only seat fifteen hundred men at once. To top all of this off, there were no hospital facilities in the camp; the soldiers instead relied on facilities in the town.  Still, Eastman was told to be prepared to receive prisoners, and from the beginning it would seem that the camp was destined to be overcrowded. This led to many charges that the prison camp was designed from the beginning to be not a prison, but a death camp. 
The first Commandant of the Camp was Major Henry V. Colt of the 104th New York Volunteers. Colt was given charge over the prisoners because of an inability to serve in the field, a characteristic that many in his position in similar prison camps shared with him. A man of relatively even temperament, Colt was able to achieve what few officers in the war were able to in that he was liked by both Union and Confederate soldiers. He received his first prisoners on July 6, 1864, when 400 prisoners arrived at the camp. Though Colt was a fair man, and liked well enough by most of his prisoners, he was a man who took his duty seriously as commandant. The best evidence for this can be viewed in the case of Washington B. Traweek. Traweek was a Confederate soldier who made an escape attempt from the prison along with a few other soldiers, all of them members of the Jefferson Davis Artillery Company. The escape plan involved digging a tunnel from a neighboring tent underneath the fence and into town. Later, when a series of hospitals was to be built for the camp, the prisoners involved in Traweek's plot decided to transfer their tunnel to go underneath the hospital, and started work on a new tunnel. Others had a similar idea, and this resulted in the tunnels being discovered. However, the first tunnel had not been discovered, so Traweek and his men returned to work on that tunnel. The next day, Traweek was called before Major Colt. Colt inquired as to where Traweek's tunnel was, and who had been tunneling with him. When Traweek refused to tell, Colt ordered him into a sweatbox and presided over his questioning, willing to go to extreme measures to find and persecute the tunnelers. Traweek held fast however, and Colt was forced to release him. Traweek and his companions eventually escaped.
Five days after the camp opened, Surgeon Charles T. Alexander was ordered to inspect the Camp at Colonel Hoffman's request. Alexander found two major problems with the camp that he detailed in his report. The first was that of the camp's sanitary conditions. The sinks near Foster's Pond contained stagnant water, and he feared if they were not cleaned, they might "become offensive and a source of disease." He recommended the construction of new sinks. Hoffman did not heed these warnings. The other problem that Alexander identified was that of the hospitals. While the camp now had a hospital, in the form of a tent, it did not have an assigned surgeon and instead relied on the services of William C. Wey, a local civilian. Alexander also thought the notion of using a tent as a hospital within the prison was inappropriate, and therefore should be rectified. Hoffman allowed for three pavilion wards to be planned at Alexander's suggestion. While Hoffman was open to suggestions on how to improve his prisons, he was notoriously cheap, and he was also a major proponent of retaliation should the war take a sour turn, and reduced rations to the prisoners. As a result, many prisoners in the Elmira Camp were malnourished, a state of being that harmed many of them, especially during the extreme heat of summer and the extreme cold of winter. 
Another major problem that points to Hoffman's policy of retaliation was the construction of winter housing for the prisoners. During most of their stay, a significant portion of the prisoners in Elmira lived in tents, as there was only room for 6,000 prisoners in the barracks, while there were 10,000 prisoners. And while there was a large work force at the camp, due to the prisoner population, work on winter housing did not start until October when the cold New York nights had started to pervade the camp. The late start resulted in the deaths of hundreds of prisoners as the construction continued into the winter months. In addition to the poor conditions of the weather, winter construction was also delayed by a lack of lumber. In November, it was also reported that the existing barracks were experiencing trouble as well, with roofs falling into disrepair and being unfit to stand the winter weather. Even in late November and early December, there were reports of over 2,000 Confederates sleeping in tents, and a Christmas inspection said that 900 were still in tents.  It was not until October 27, 1864 that work finally began on the draining system that Charles T. Alexander had suggested. By then, the cold weather of November and December kept this project from being completed in a timely manner, and it was not completed until January 1st of the following year. In the meantime, hundreds of Confederate prisoners were subjected to stagnant and unclean water, and many became ill simply because of a lack of clean drinking water. It was during this winter that the tangible signs of Hoffman's reduced rationing began to take their toll on the prisoners, as prisoners were reduced to eating rats in place of the food they would normally get. Indeed, rats became a currency in the trade system of the prisoners for other supplies. 
The propaganda of the Union went to great pains to make sure that the details of Hoffman's retaliations were covered up. The New York Herald denied any mistreatment of prisoners in Elmira, calling the reports a "pure fabrication." To the Herald, all rations at Elmira were sufficient and, though it admitted the unusually high incidence of illness in the camp, the newspaper said that the sickness was "beyond the control of the authorities...there is no lack of medical attendance or supplies." Indeed, this propaganda was so powerful that the belief that the Elmira prison camp was a humane alternative to Andersonville still prevailed in some circles of thought. In a meeting in 1892, John T. Davidson, a captain of the guard detail at Camp Chemung, blamed the high mortality rate on the changing weather, water, and manner of living. He also conceded that the horrors of prison-camp life were numerous, and a condition that all men should hope to avoid. But, he said, "none of these things can apply, with a shadow of truth, to the prison camp at Elmira."  However, by the end of the Civil War it could not be disputed that Elmira had taken a considerable toll on the prisoners who came through its doors. They dubbed the camp "Hellmira," and the mortality rate of about 25 percent was near that of Andersonville (about 29 percent).
In popular culture
Miserable conditions for the prisoners in the camp are depicted in the 1982 miniseries The Blue and the Gray. Tourists, for a fee, are able to gawk at the captives from a viewing stand. Vincent was captured and brought to Elmira Prison in With Lee in Virginia
- "Civil War Prison Camp: Hellmira". City of Elmira, New York. Archived from the original on December 2, 2008. Retrieved February 6, 2011.[dead link]
- Towner, Ausburn (1892). Our County and Its People: A History of the Valley and County of Chemung. Syracuse, N. Y.: D. Mason & Co. Retrieved June 26, 2011.
- Horigan, Michael (2002). Death Camp of the North: The Elmira Civil War Prison Camp. Mechanicsburg, Pennsylvania: Stackpole Books. ISBN 0-8117-1432-2.
- Gray, Michael P. The Business of Captivity: Elmira and Its Civil War Prison. Kent State University Press, 2001. p. 3. ISBN 0-87338-708-2.
- Horigan, Michael. Death Camp of the North: The Elmira Civil War Prison Camp. Stackpole Books, 2002. p. 26. ISBN 0-8117-1432-2.
- Gray, Michael P. The Business of Captivity: Elmira and Its Civil War Prison. Kent State University Press, 2001. p. 7 - 9. ISBN 0-87338-708-2.
- Horigan, Michael. Death Camp of the North: The Elmira Civil War Prison Camp. Stackpole Books, 2002. p. 27. ISBN 0-8117-1432-2.
- Gray, Michael P. The Business of Captivity: Elmira and Its Civil War Prison. Kent State University Press, 2001. p. 12 - 13. ISBN 0-87338-708-2.
- "Recollections of Washington B. Traweek". Elmira Prison Camp Online Library. Retrieved December 2, 2012.
- Gray, Michael P. The Business of Captivity: Elmira and Its Civil War Prison. Kent State University Press, 2001. p. 14. ISBN 0-87338-708-2.
- Horigan, Michael. Death Camp of the North: The Elmira Civil War Prison Camp. Stackpole Books, 2002. p. 86. ISBN 0-8117-1432-2.
- Horigan, Michael. Death Camp of the North: The Elmira Civil War Prison Camp. Stackpole Books, 2002. p. 138, 146. ISBN 0-8117-1432-2.
- Horigan, Michael. Death Camp of the North: The Elmira Civil War Prison Camp. Stackpole Books, 2002. p. 133 - 136. ISBN 0-8117-1432-2.
- Horigan, Michael. Death Camp of the North: The Elmira Civil War Prison Camp. Stackpole Books, 2002. p. 139 - 140. ISBN 0-8117-1432-2.
- Horigan, Michael. Death Camp of the North: The Elmira Civil War Prison Camp. Stackpole Books, 2002. p. 125 - 126. ISBN 0-8117-1432-2.
- The Blue and the Gray, Part 3, Columbia Pictures Television DVD set, 2001, Disk 3.