Grevey played college basketball at the University of Kentucky, where he was a member of legendary coach Adolph Rupp's last freshman class and played his three collegiate seasons (freshmen were not eligible to play varsity basketball at the time) under Rupp’s successor, Joe B. Hall. He was named First-Team All-Southeastern Conference in all three of his college seasons and All-American in his junior and senior years. In his senior year Kentucky lost to UCLA in the championship game of the NCAA Tournament in what would be the final game in the career of UCLA’s legendary coach John Wooden; Grevey scored a game-high 34 points and was named to the all-Final Four team.
Upon completion of his collegiate career, Grevey scored 1,801 points, which at the time ranked him second in University of Kentucky history behind only Dan Issel's 2,138. His jersey number, 35, is retired by the University of Kentucky.
In 1975 Grevey was selected by the Washington Bullets in the first round (18th pick) of the NBA Draft and by the San Diego Sails in the first round (sixth pick) of the 1975 ABA Draft. Grevey signed with the Bullets and played mostly as a backup small forward and shooting guard his first two seasons. When Phil Chenier suffered a season-ending back injury early in the 1977–78 season, Grevey became the starting off guard and averaged 15.5 points per game. The Bullets won their only NBA championship that season, led by Grevey, newly acquired Bob Dandridge and the future Hall-of-Fame duo of Elvin Hayes and Wes Unseld.
Grevey enjoyed four more solid seasons in Washington, averaging no less than 13.3 points per game. An injury sidelined him for half of the 1982–83 season and reduced him to a reserve for the remainder of his career. He played his final two seasons with the Milwaukee Bucks.
In his 10 NBA seasons Grevey played 672 games and scored 7,364 points, for an average of 11.0 points per game.