Rosburg was born in San Francisco, California. He played golf as a junior at the Olympic Club, and at the age of 12, he faced the then-retired baseball Hall of Famer, Ty Cobb, in the first flight of the club championship, and beat Cobb 7 and 6. Rosburg says Cobb was gracious in defeat and shook the young Rosburg's hand, but Cobb took so much kidding from the other Olympic Club members that for many years, Rosburg hardly ever saw Cobb back at the club. Rosburg was an outstanding baseball player at Stanford University in Palo Alto, California during the 1940s, and almost chose baseball as a career over golf. He graduated from Stanford in 1949, and turned pro in 1953. He is a member of the Stanford Athletic Hall of Fame.
During his career, Rosburg was one of the most consistent top-10 finishers on the PGA Tour. Rosburg won the Vardon Trophy in 1958 for the lowest average score (70.11) on tour that year. Rosburg's career year was 1959, when he finished seventh on the money list and was named to the Ryder Cup team, after winning the PGA Championship and finishing second in the U.S. Open. In 1969, he won the PGA Club Professional Championship. He won six tour events during the course of his career, before moving into semi-retirement after the 1972 season, his most successful financially. That year, he won the Bob Hope Desert Classic by one stroke over Lanny Wadkins.
The 1959 PGA Championship was played at the Minneapolis Golf Club in St. Louis Park, Minnesota. Rosburg won with a 72-hole score of 277 by one stroke over Jerry Barber and Doug Sanders. Rosburg claimed that he won the 1959 PGA Championship without ever hitting a practice shot during that week, except for a few chips and puts. He came close to winning a second major that year, finishing 2nd at the U.S. Open to Billy Casper. He also finished in a three-way tie for 2nd at the 1969 U.S. Open, one stroke behind Orville Moody.
After his playing days on the PGA Tour finished in the mid-1970s, Rosburg became a commentator for ABC sports television. He pioneered the now-common practice of roving on the golf course and reporting from the fairways. At the time of his death, he was the longest serving active golf announcer on television, with more than 30 years behind the microphone. He is remembered for his catch phrase, "He's got no chance, Jim", which Rosburg would utter whenever he encountered a golfer who had hit his ball into a seemingly impossible position (usually behind a tree or in deep grass), upon which the player would then produce a miraculous recovery. The "Jim" is in reference to ABC commentator Jim McKay. Rosburg is also credited with helping ABC hire Judy Rankin, who was the first full-time female golf commentator to cover men's events, including the major championships. Rosburg worked nearly three decades as a commentator with Dave Marr, who like Rosburg won a single PGA Championship.