Allen was born in Fayetteville, North Carolina. He attended Terry Sanford High School in Fayetteville, and played high school football for the Terry Sanford Bulldogs. During his career he had 68 receptions for 1,257 yards.
Allen attended Clemson University, where he played for the Clemson Tigers football team from 2008 to 2011. He was redshirted in 2008. In 2009 he started six of 14 games and had 10 receptions for 108 yards and three touchdowns. He started all 13 games in 2010 and was a second-team All-ACC selection after recording 33 receptions for 373 yards with a touchdown.[3]
Through the first seven games of his junior season, Allen had 27 receptions for 381 yards and four touchdowns.[4]
Allen was named the 2011 recipient of the John Mackey Award on December 8, 2011 at the Home Depot College Football Awards Red Carpet Show. He graduated from Clemson on August 9, 2014.[5]
Allen was drafted 64th overall by the Indianapolis Colts in the 2012 NFL Draft. He was the second tight-end taken in the draft as well as the second taken by the Colts. He helped fellow rookie Andrew Luck lead the Colts to an 11-5 record.[6] Despite their draft positions, Allen actually bested his teammate (taken 34th overall), Coby Fleener, in overall production ranking 7th best among the league's rookie receiving leaders.[7] Allen and fellow rookie teammates T.Y. Hilton, Coby Fleener, Vick Ballard, and LaVon Brazill combined for an NFL record 3,108 yards - the most by any rookie class playing for an NFL club since the 1970 NFL merger. Interestingly, the second best performance was the Colts 1999 rookie class with 2,751 yards.[8]
Allen was intended to be a large part of new Colts offensive coordinator Pep Hamilton's offense, but he was placed on the injured reserve following a hip injury in the first game of the season. He finished the season with one catch for 20 yards and a touchdown.