Koeman finished his playing career with Groningen in 1998 and became youth coach at PSV. In October 2001, he was promoted to assistant manager under Eric Gerets, and for the 2004–05 season became manager in RKC Waalwijk. He managed RKC for one season before moving to Feyenoord. In March 2006, he extended his contract to the summer 2009. However, on 3 May 2007, Koeman announced his immediate resignation due to motivational problems, after a troublesome season, where Feyenoord eventually finished seventh.
Before the 2011–12 season, he was appointed as the new manager of FC Utrecht. However, on 18 October 2011 he quit the job citing bad working conditions and frustration after several key players had been sold.
On 16 June 2014, he was appointed as the assistant head coach of Southampton, where he works alongside his brother, Ronald, for the first time.