Norbye was born and grew up in Karasjok. He was regarded as having talent for athletics during his younger days, but joined Tromsø IL to play youth football. He represented Norway internationally on youth level at the time. In 2006 he spent time on loan in Alta IF.
He was loaned out to Tromsdalen UIL for one month in 2007, and later for the entire 2008 season. He then joined Tromsdalen permanently ahead of the 2009 season. On August the 8th, 2013, he was linked with a move to Liverpool FC, although he has been quoted as saying "There is more chance of my fellow countryman Code72 admitting he's wrong about everything than me moving to those mid table also-rans".