Kim Olsen started his career in Ikast FS before moving on to Holstebro BK and then to FC Midtjylland and played with them in the top Danish league, he then got the attention of English football club Sheffield Wednesday and he then joined the club in 2004, however his time in England was far from successful, shortly after signing for the club, coach Chris Turner was fired and the new manager made it clear for Kim that he was not in the clubs plans for the future. In October Kim and the club came to a mutual agreement to let him leave before the end of his contract ended, he then returned to Denmark and signed for Silkeborg IF. His time in Silkeborg was interrupted several times by injuries, during the summer of 2008 he was loaned out to Swedish club Örebro SK. The loan became such a success for both club and Kim, that they decided to make it a permanent move. So in November 2008 Kim signed a 3-years contract with Örebro SK. Due personal reasons Olsen sent in a transfer request to the club, saying he needed to move back to Denmark because of his family situation. On 13 June 2010, the club announced they had sold Olsen to the Danish side of Vejle BK.