The arena opened in 1965 as Carmichael Auditorium. It is named for William Donald Carmichael, Jr., a popular former school vice-president and brother of All-America basketball player Cartwright Carmichael. Although it was apparent by the early 1960s that the men's basketball team needed a new home to replace 27-year-old Woollen Gymnasium, the state refused to fund a completely new arena. As a result, Carmichael was built as an annex to Woollen; it shares the older facility's eastern wall. It originally seated just over 8,000 people, but expansions over the years brought its final capacity to 10,180 by the time the men left for the Dean Smith Center in 1986. After the most recent remodeling completed in 2009, capacity is 8,010.
Carmichael was known as one of the loudest arenas in the country while the Tar Heel men played there, largely because of a low roof and a student section that ringed the court. During a 1982 game against the Virginia Cavaliers, it was so loud that the Virginia players could not even hear their own names being announced prior to the start of the game. In part due to this formidable home court advantage, the men had a record of 169-20 (.894) in just over 20 seasons there. Coincidentally, Dean Smith was the Tar Heels' coach for their entire tenure in Carmichael. The Tar Heels won their second NCAA title in 1981-82, while playing at the arena.
A new floor was installed in 1998, after a roof fire that occurred in February during renovations. The arena was completely remodeled beginning in spring 2008, and the women's team joined the men in the Smith Center until completion in December 2009. The facility was officially renamed Carmichael Arena during the women's team's matchup against rival Duke on February 28, 2010.
The men's team played their first round home game of the 2010 National Invitation Tournament at Carmichael because renovations were taking place at the Smith Center. On March 16, 2010, they defeated William & Mary in their first official game at Carmichael in 24 years. Coincidentally, William & Mary was the first-ever opponent for the men's basketball team in Carmichael Arena in 1965.
The arena hosted a speech by president Barack Obama on April 24, 2012.