Trotter had originally joined Charlton as a trainer twenty two years earlier under Jimmy Seed, but after Seed's sacking he was appointed manager in 1956, although he was unable to prevent relegation that season in 1957. In the following season saw the extraordinary match against Huddersfield Town F.C. at The Valley on 21 December 1957. Charlton played most of the match with 10 men after their captain Derek Ufton was injured, and Huddersfield were leading 5–1 with just 27 minutes remaining. At that point, Johnny Summers began an extraordinary passage of play in which he scored five goals and assisted with two others to allow Charlton to win 7–6. Huddersfield become the first, and still the only, team to score six goals in an English Football League match - or indeed any other professional football match - and still be on the losing side. Trotter came close to taking the Addicks back into the top flight that season, but lost 4–3 to Blackburn Rovers on the final day of the season, when a draw would have seen Charlton promoted.
The remaining seasons under Trotter's charge saw finishes of eighth, seventh and tenth before he was fired in 1961 after just one victory in twelve games.