Before his stint in Philadelphia, O'Brien was the head coach of the Boston Celtics from 2001–2004, replacing Rick Pitino. He worked to rebuild the struggling Celtics and led them twice to the playoffs. During the 2003–04 NBA season, however, O'Brien consistently fought with Celtics' general manager Danny Ainge over short-term versus long-term goals. Ainge was looking to completely redo the roster, and traded Eric Williams and Tony Battie, two of O'Brien's favorite hardworking players in December 2003. As a result of the conflict, O'Brien shocked everyone in the Celtics community by resigning in January 2004.
The Indianapolis Star reported on May 31, 2007 that he would coach the Indiana Pacers.  He replaced Rick Carlisle, who was fired after four years, when the team failed in 2006–07 to make the playoffs for the first time in a decade. On January 31, 2011, Pacers President of Basketball Operations Larry Bird relieved O'Brien of his coaching duties. He was replaced by assistant Frank Vogel on an interim basis, but Vogel was retained after the season and is currently the Pacers' head coach.
In the 2012–13 season, O'Brien was an assistant coach for the Dallas Mavericks. After the season, O'Brien retired from the team to spend more time with family.