Grunfeld was selected to participate as a member of the American basketball team at the 1973 Maccabiah Games, while he was still attending high school. The U.S. team was defeated by Israel in the final game.
He attended the University of Tennessee, where he played basketball with future NBA player Bernard King. Nicknamed the "Ernie and Bernie Show", they averaged over 40 points per game. With 2,249 points, he set a new record as the school's all-time leading scorer. The record was broken by Allan Houston in 1993.
Grunfeld was drafted 11th overall by the Milwaukee Bucks in the 1977 NBA draft. He played with that team for two years and moved to the Kansas City Kings for the 1979–82 seasons. In 1979 he led the NBA in games played, with 82. In 1981 he had a .535 field goal percentage.
The Knicks signed him as a free agent in 1982, and he played there for four years. He retired following the 1985–86 season. Grunfeld averaged 7.4 points per game in his NBA career.  In 1982 he averaged 12.7 points a game, and 21.8 per 40 minutes. In 1986 he was third in the NBA in 3-point field goal percentage, with .426. He finished his career with a .477 field goal percentage and a .770 free throw percentage. His playoff shooting percentages were even better.
After he retired from the NBA, Grunfeld was the Knicks radio analyst for the MSG Network from 1986–89. He then briefly worked under Stu Jackson as an assistant coach for the Knicks before starting his career in administration.
Grunfeld was appointed director of administration in the 1990–91 season and was moved to vice-president of player personnel on April 23, 1991. He was then appointed vice president and general manager on July 21, 1993. He became president and general manager on February 23, 1996. During his time with the Knicks, Grunfeld and his family were residents of Franklin Lakes, New Jersey.
During his eight-year tenure with the Knicks executive, the team had a record of 397 wins and 227 losses (.636), and a 61–44 playoff record. They won the Atlantic Division three times and reached the NBA finals twice.
At the time of his removal from his general manager post, during the 1998– 99 season, the team had a 21–21 record and were on the verge of not making the playoffs. They eventually got in with a 27–23 record. He was responsible for bringing every player on that roster to the team except for Patrick Ewing. Before the start of the season, he organized the trade of Charles Oakley to the Toronto Raptors for Marcus Camby, and John Starks to the Golden State Warriors for Latrell Sprewell. Many people blamed him for the Knicks' poor play. However, they came within 3 games of winning the championship, losing to the San Antonio Spurs in 5 games. At first it was said that he was being temporarily relieved of his duties as general manager. When the season ended with the result that came about, it was said that all was forgiven and he would be reinstated.
However, he took the job as the Bucks' general manager on August 13, 1999. He held the position for four seasons, during which the Bucks made the playoffs three times and enjoyed 14 playoff wins. The team won 177 regular season games and lost 151 (.540 average).
He was hired by the Washington Wizards as President of Basketball Operations in June 2003. As Wizards GM, Grunfeld signed free agent point guard Gilbert Arenas, who has gone on to have one 2nd team All-NBA and two 3rd team All-NBA seasons. Grunfeld also traded Kwame Brown for All-Star Caron Butler (who was later traded in a deal for Josh Howard).