Ivy J. Bryant, Jr. was born in Moultrie, Georgia, the oldest of 12 children. During the Great Depression he played the fiddle on street corners to help the family buy food, pushed to do so by his father.
After being wounded in World War II, he began working seriously on his guitar playing, influenced heavily by Django Reinhardt. After the war, he returned to Moultrie, then moved to Los Angeles county where he worked in Western films and played music in bars around L.A.'s Skid Row, where he met pioneering pedal steel guitarist Speedy West. West, who joined Cliffie Stone's popular Hometown Jamboree local radio and TV show, suggested Bryant be hired when the show's original guitarist departed. That gave Bryant access to Capitol Records since Stone was a Capitol artist and talent scout.
In 1950 Tex Williams heard Bryant's dizzying jazz/country style and used him on his recording of "Wild Card". In addition, Bryant and West played on the Tennessee Ernie Ford-Kay Starr hit "I'll Never Be Free", leading to both men being signed to Capitol as instrumentalists. Bryant and West became a team, working extensively with each other.
Bryant was a difficult musician to work with. By 1955 he left Hometown Jamboree (retaining his friendship with West) and after various clashes with his Capitol producer Ken Nelson, the label dropped him in 1956. In 1957 Jimmy Bryant was a part of one of the first integrated television shows featuring popular radio and television star Jimmie Jackson who hosted the show along with black Jazz violinist and recording star, "Stuffy" Smith and black jazz percussionist and recording star, George Jenkins. He continued working in Los Angeles and in the early 1960s he and his trio made an appearance in the Coleman Francis film The Skydivers.
During the 1960s he shifted into music production. Waylon Jennings made a hit of his song "Only Daddy That'll Walk the Line". He can also be heard playing fiddle on the Monkees' "Sweet Young Thing". In the early 1970s Bryant ran a recording studio in Las Vegas, but finally relocated to Georgia before settling in Nashville in 1975, the same year he reunited with Speedy West for a reunion album produced by Nashville steel guitarist Pete Drake. Bryant played in Nashville bars and did some recording work but his personality did not mesh well with Nashville's highly political music and recording industry. In 1978, in declining health, Bryant learned that had lung cancer; he was a heavy smoker. He died in Moultrie in September 1980 at the age of 55.