Ramey was born in Austin, Texas, and played trumpet in college, but switched to contrabass when playing with George Corley's Royal Aces, The Moonlight Serenaders, and Terrence Holder. In 1932 he moved to Kansas City and took up the bass, studying with Walter Page. He became a fixture on the Kansas City swing jazz scene in the 1930s, and played with Jay McShann's orchestra from 1938 to 1943.
In 1944 he moved to New York City, where he played with Lester Young, Count Basie, Ben Webster, Coleman Hawkins, Charlie Parker, Hot Lips Page, Horace Silver, Thelonious Monk (as a member of Monk’s first trio in 1947, together with drummer Art Blakey), and Miles Davis. He transitioned into the bebop style ably, but also continued to play in more swing-oriented outfits. Later in his life he toured Europe with Buck Clayton, and played with Muggsy Spanier, Teddy Wilson, Dick Wellstood, Jimmy Rushing, and Eddie Vinson, in addition to doing several reunion gigs with McShann. In 1976 he moved back to Texas, playing occasionally up until his death in 1984.
- Texas State Historical Association Retrieved 2012-08-27
- Interview with Gene Ramey, July, 1980. University of Texas at San Antonio: Institute of Texan Cultures: Oral History Collection, UA 15.01, University of Texas at San Antonio Libraries Special Collections.