He is most remembered for writing the poem "Maryland, My Maryland," which is also the reason for his being called the "Poet Laureate of the Lost Cause". It became a war hymn of the Confederacy after the poem's words were set to the tune "Lauriger Horatius" (the tune of O Tannenbaum) during the Civil War by Jennie Cary, a member of a prominent Maryland and Virginia family. It later became the state song of Maryland.
Randall wrote the poem after learning that his friend Francis X. Ward, of Randallstown, Maryland, was killed by the 6th Massachusetts Regiment in the Baltimore Riot of April 19, 1861. The work was first published a week later on April 26, in the New Orleans newspaper The Sunday Delta.
After abandoning his studies at Georgetown University, he traveled to South America and the West Indies. Upon his return to the United States he taught English literature at Poydras College in Pointe Coupee Parish, Louisiana. It was during this time that he penned "Maryland, My Maryland". Tuberculosis prevented him from enlisting in the Confederate Army. However, he was able to serve with the Confederate States Navy in Wilmington, North Carolina. Though a Marylander by birth, he wrote the poem "Maryland, My Maryland" while living in Augusta, Georgia. He considered himself a Georgian by adoption. After the Civil War, Randall became a newspaper editor and a correspondent in Washington, D.C., for The Augusta Chronicle. He continued to write poems, although none achieved the popularity of "Maryland, My Maryland". His later poems were deeply religious in nature.
He died on January 15, 1908 in Augusta, Georgia, and is buried there in Magnolia Cemetery. Augusta honors him on the Monument to Poets of Georgia along with Fr. Abram Ryan, Sydney Lanier, and Paul Hamilton Hayne, all of whom saw Confederate service. The Randall Memorial Committee of Chapter “A” United Daughters of the Confederacy Augusta, Georgia, dedicated a statue to him there in 1936. James Ryder Randall Elementary School in Clinton, Maryland, bears his name.