Born near Stella, Virginia, Penn moved with his parents to Lexington, Kentucky; he then completed preparatory studies and attended Emory and Henry College, Marion, Virginia. Later he moved to the parish of St. Tammany, Louisiana, in 1821 and engaged in planting near Covington. He served in the State house of representatives. Postmaster of New Orleans from December 19, 1843, to April 18, 1849. He served as delegate to the Democratic National Conventions in 1844, 1852, 1856, and 1860.
Penn was elected as a Democrat to the Thirty-first Congress to fill the vacancy caused by the death of John H. Harmanson. He was reelected to the Thirty-second Congress and served from December 30, 1850, to March 3, 1853. He served as chairman of the Committee on Expenditures in the Post Office Department (Thirty-second Congress). He returned to St. Tammany Parish and engaged in planting and the operation of a lumber mill near Covington. At the conclusion of the Civil War returned to Washington, D.C., where he died May 7, 1866. He was interred in Glenwood Cemetery.