He retired prior to Torquay achieving league status, but became the club's trainer and with Percy Mackrill in charge saw United promoted to the Football League. In May 1932 he was appointed manager of Torquay United and during his six seasons at the helm at Plainmoor, Torquay finished 10th 3 times, and 20th (out of 22 teams) 3 times.
In May 1938, Brown moved to the manager's chair at Chester, leading them to 6th place in Division Three (North) and a second replay against Sheffield Wednesday in the 4th round of the FA Cup in his first season. The war intervened in September 1939 with Chester unbeaten after 3 games of the new season, and Chester joined a regional league alongside the likes of Manchester United, Liverpool, Everton and Manchester City. When league football resumed, Chester finished the 1946-47 season in 3rd place and won the Welsh Cup the same year. However, the following seasons saw mid-table finishes at best as he struggled with the limited resources available. He eventually left Chester in May 1953, and was manager of Stafford Rangers in 1954-55. He died shortly after finishing his football managerial career.