Koller played his entire career for Swiss club Grasshoppers Zürich. In these 24 years he won seven Swiss championships and five Swiss cups. For the Swiss national team he got 56 international caps, scored 3 goals and participated at Euro 1996.
Koller's career as coach started at the Swiss club FC Wil in the season 1997/98. He led his club to a promotion place in the Swiss Challenge League. In January 1999 he went to FC St. Gallen in the Swiss Super League. One year later the club won its first championship for almost 100 years. Also on European level FC St. Gallen could celebrate successes: in the UEFA Cup the Swiss side could eliminate English top club FC Chelsea. Koller was voted Swiss manager of the year 2000.
On 9 January 2002, he went to his old love Grasshoppers Zürich. Koller won his second championship as a coach in the 2003 Season. After failing to qualify for the Champions League (against AEK 1-0/1-3) and a losing streak in the league he resigned on 3 October 2003.
The next step in his career was a move to German Bundesliga. Koller's first station in Germany was 1. FC Köln. However, this spell was not a very successful and took only seven months. Koller took the club in a desperate position and could not improve the results. In his term the club only won four matches and relegated eventually to Second Bundesliga. During this period he gave young talents the opportunity to debut on the highest level. The most prominent example is German international Lukas Podolski. 1. FC Köln ended at last position. Koller was sacked after the season ended.
From 23 May 2005 on Marcel Koller was coach of German side VfL Bochum. The club was relegated into the Second Bundesliga the season before and its coach Peter Neururer was sacked as a result. Koller got the challenge to lead the club directly back to the highest level. Already five matches before the end of the season VfL Bochum was sure of its return to the Bundesliga.
The goal of season 2006/07 was to stay in the Bundesliga. VfL Bochum did not start very well and many supporters demanded the dismissal of the coach. But the board kept faith in the coach and Koller's contract was even extended in that difficult period. VfL Bochum bought a few new players in the winter break and started a winning streak. Highlights were two memorable home victories against arch rivals Borussia Dortmund (2–0) and FC Schalke 04. (2–1). Bochum even had a winning streak of four away victories in a row at the end of the season, a new club record. Already two rounds for the end of the season VfL Bochum was sure of survival in the Bundesliga. Eventually the club finished at the 8th place, the third best season for Bochum ever.
The VfL Bochum lost three important players in the summer break 2007 so that most observers were rather skeptical about the club in the coming season. But the club played a solid season and ended, without too much relegation troubles, in the midfield. However, at the beginning of the season 2008/09 Koller defined 45 points as target. Surprisingly Bochum did not have to sell key players and could even afford some relatively spectacular transfers. With the return of former key players Paul Freier and Vahid Hashemian, and the purchase of the talents Daniel Fernandes and Austrian international Christian Fuchs Koller called his team the best team he had at Bochum ever. Nevertheless the club came into relegation troubles and could save itself rather late in the season. At the end Bochum finished 14th in that season. Again many supporters asked during the season for the dismissal of the coach. Especially the release of crowd pleaser Tomasz Zdebel in the winter break caused many frictions with the fans.
Due to the relatively poor results and the boring style of football the team played in the season before Koller started under great pressure in the 2009–10 season. With only four points out of the first six matches VfL Bochum started again rather disappointing and the public pressure grew continuously. After a 2–3 home defeat by FSV Mainz more than two thousand supporters gathered at the stadium to demand his dismissal. As a reaction one day later, at 20 September 2009 Koller was sacked as head coach of VfL Bochum.