In 1985, Veh resumed playing in his home town for FC Augsburg, moving in 1987 for a few months to local rival TSV Schwaben Augsburg. Due to injuries he played only eight games in the second Bundesliga for SpVgg Bayreuth before retiring in November 1990.
He scored two goals in 65 Bundesliga games, one goal in 60 second Bundesliga games, and appeared in 18 games of Swiss NLA.
Veh was hired by VfB Stuttgart to replace Giovanni Trapattoni on 10 February 2006. Yet, on 18 April 2006 his contract was extended until summer 2007, and on 19 January 2007 for another year.
In May 2007, Veh and VfB Stuttgart scored their biggest successes by winning the German championship on 19 May 2007 and qualifying for the DFB Cup final of 26 May 2007 in Berlin, which gave Veh and his team the chance to win "the double". The opponent, 1. FC Nürnberg, had beaten Stuttgart twice in regular season, and prevailed again by beating VfB Stuttgart 3–2 in overtime. In the 2007–08 season Stuttgart found it tougher as champions and finished sixth.
On 23 November 2008, Veh parted ways with VfB Stuttgart, due to the club's current performances. His dismissal came in the wake of a 4–1 defeat away at VfL Wolfsburg and a 5 match winless streak.
On 23 May 2009, it was announced that Veh will take over newly crowned Bundesliga champions VfL Wolfsburg from the hands of Felix Magath in July. Veh was sacked as manager of VfL Wolfsburg on 25 January 2010.
On 24 May 2010, he was announced as the new manager of Hamburger SV. On 13 March 2011, Veh was sacked by HSV, following a 6–0 hammering by Bayern Munich in the Bundesliga, six points adrift from a spot that would have allowed them to play in the UEFA Europa League the following season.
On 30 May 2011, Veh was announced as the new manager of Eintracht Frankfurt. After achieving promotion to the Bundesliga in his first season, Veh renewed his expiring contract with Frankfurt for another year. Veh demanded that that the club invest more money in the squad or he'll leave the club.