Armitage was born in Mackay, Queensland. Despite growing up in a rugby league stronghold city, Armitage played Australian rules football in his younger years. His father, Greg, was an avid Australian rules football fan and introduced David to the game at a young age. David played his junior football for the Eastern Swans in the AFL Mackay competition. He made his senior debut for the Swans in 2002 at 14 years old and kicked five goals against Bakers Creek. In 2005, at the age of 16, he made the decision to move to Brisbane and play for Morningside in the hope of increasing his chances of getting drafted to an AFL club. While at Morningside, he represented the under 18s Queensland team several times and played alongside future AFL players Courtenay Dempsey and Lee Spurr. He also competed in the 2005 QAFL Grand Final. The decision paid off as he was drafted to St Kilda with the 9th pick in the 2006 AFL draft.