Brown was selected in the 2002 Major League Baseball Draft in the first round (35th selection overall) by the Athletics, at the behest of Billy Beane and Paul DePodesta, over the strong objections of the scouting department. He is most remembered for a game in 2002 where he, planning for a triple, slipped and fell on first base, and while scrambling to get back to base, was notified that he in fact hit a home run. The home run was reenacted in the film Moneyball.
Though spending the majority of his time with the Oakland's Double-A Midland RockHounds and Triple-ASacramento River Cats, "Badge" (short for "Badger," a nickname for his ample body hair) had a .379 OBP over his first 4 years. Brown made his Major League debut on September 3, 2006, against the Baltimore Orioles. He had 3 hits in 10 AB, including two doubles, and a .364 OBP in the Major Leagues for the 2006 season.
Brown was designated for assignment by the Athletics on May 23, 2007 and subsequently outrighted to the minors.
On February 15, 2008, Brown announced his retirement. Brown was the team's third-ranked catcher behind Kurt Suzuki and Rob Bowen and was unlikely to make the major league roster.