Phegley then began the 2011 season at Double-A Birmingham but was promoted to Triple-ACharlotte for the last month of the season. He hit a combined .242, 9 home runs and 56 RBI in 116 games in 2011. Phegley spent the entire 2012 season at Triple-A Charlotte where he hit .266, 6 home runs, 48 RBI in 102 games.
On July 5, 2013, Phegley was promoted from Triple-A after hitting .316 with 15 home runs, 18 doubles and 41 RBI. He collected his first Major League hit and RBI on a single off of Tampa Bay Rays pitcher Jeremy Hellickson on July 5, 2013. On July 7, 2013, Phegley recorded his first major league home run off of David Price. Phegley's first grand slam came four days later against Detroit's Aníbal Sánchez.