The Seat Pleasant, Maryland native served as head men's basketball coach at Loyola-Maryland from 1994 to 1997, and then at the University of Michigan from 1997 to 2001, where he posted a 62-60 record. However, all of Ellerbe's wins in his first two seasons at Michigan were later vacated from official records as a result of the University of Michigan basketball scandal in which four players received money from booster Ed Martin. These infractions dated to the previous coaching staff, and Ellerbe himself was cleared of wrongdoing.
He also Coached at the University of Michigan. Less than five months after taking an assistant coaching job at Michigan, Brian Ellerbe was given the reins to the team as interim head coach for the 1997-98 season, a position that was made permanent the following year. In his four years, Ellerbe compiled a 28-32* record with a 10-22* mark in the Big Ten. Following two years of rebuilding, Ellerbe began building his team and found himself again with a very young team for the 1999-2000 season. The Wolverines got off to a 9-2 start to the season, but went on to a 15-14 overall mark after a 6-10 conference season, which gave them the No. 8 seed in the Big Ten Tournament and a berth in the NIT. Ellerbe also coached Jamal Crawford, who left early and became the No. 8 overall pick in the NBA Draft to the Chicago Bulls. Continuing with a young team, the 2000-01 season saw the Wolverines finish with a 10-18 overall mark and a 4-12 finish in the Big Ten. The Wolverines earned the No. 10 seed in the Big Ten Tournament.