Conger graduated from Huntington Beach High School in Huntington Beach, California in 2006. Conger was drafted 25th overall by the Angels in the 2006 Major League Baseball Draft.
Conger was selected to represent the United States in the 2010 All-Star Futures Game. He hit a three-run home run, earning him MVP honors.
He was promoted to the major leagues as part of September call-ups on September 7, 2010. He made his major league debut on September 11, 2010 as a pinch hitter for Hideki Matsui. His first hit in the major leagues came off of Cleveland Indians pitcher Jeanmar Gómez on September 15, 2010. His first home run came off of Jeff Niemann on April 5, 2011.
On July 19, 2011, Conger was optioned to AAA Salt Lake to make room for Tyler Chatwood. At the time, Conger was hitting .194 and opponents had been successful at stealing bases 48 out of 56 attempts against him. On August 18, 2011, the Angels recalled Conger.