Herrhausen fell victim to a sophisticated roadside bomb shortly after leaving his home in Bad Homburg on 30 November 1989. He was being chauffeured to work in his armoured Mercedes-Benz, with bodyguards in both a lead vehicle and another following behind. The bomb had been hidden in a saddle bag on a bicycle next to the road that the assassins knew Herrhausen would be traveling in his three-car convoy. In the bag was a 7 kg bomb that was detonated when Herrhausen's car interrupted a beam of infrared light as it passed the bicycle. The bomb targeted the most vulnerable area of Herrhausen's car – the door where he was sitting – and required split-second timing to overcome the car's special armour plating. The bomb utilized a Misznay-Schardin mechanism. A copper plate, placed between the explosive and the target, was deformed and projected by the force of the explosion. It is unlikely that this improvised explosive device had the precise engineering required to form the liner into a more effective slug or "carrot" shape (as in a shaped charge or an EFP) but in any case, the detonation resulted in a mass of copper being projected toward the car at a speed of nearly two kilometers per second, effectively penetrating the armoured Mercedes. Herrhausen's legs were severed and he bled to death.
No one has ever been charged with the murder. For a long time, the German federal prosecutor office listed Andrea Klump and Christoph Seidler of the Red Army Faction as the only suspects. The Federal Criminal Police Office (Germany) presented a chief witness Siegfried Nonne who later retracted his statements in which he claimed to have sheltered four terrorists in his home. His half-brother Hugo Föller (died 23 January 1992) furthermore declared that no other persons had been at the flat at the time. On 1 July 1992 German television broadcast Nonne's explanations of how he was coached and threatened by the Verfassungsschutz, the German internal intelligence agency, to become the main witness. In 2004 the federal prosecutor dropped the charges against the Red Army Faction; the investigation was to continue without naming a suspect. Certain German and US media connected the assassination of Alfred Herrhausen to the Staatssicherheitsdienst (Stasi) of the GDR.
The award-winning German documentary movie of 2001 Black Box BRD retells the lives and deaths of Alfred Herrhausen and Wolfgang Grams, a radical activist (who was a major suspect in the attack on Herrhausen). 
- 2007 Wall Street Journal article speculating about communist assassins
- Gravesite at Waldfriedhof Bad Homburg, Germany. "We must say what we think. We must do what we say. We must also be what we do."
- Thomas Moser: Andreas Veiel: Black Box BRD. Alfred Herrhausen, die Deutsche Bank, die RAF und Wolfgang Grams. Deutschlandfunk, 23 December 2002