Brown started his career with First DivisionMiddlesbrough, where he came through from the juniors team. He joined Burnley in August 1952, but his time at the club was restricted to just six appearances because of a serious back injury. Picking up a slipped disc in his back at age seventeen, he recovered after the operation to salvage his playing career. He joined Bournemouth in June 1954, where he made a total of 215 appearances and 5 goals in the league. He later moved onto Aldershot in July 1960.
Brown re-joined Burnley as a coach in 1961 and was in charge of the Burnley team that won the FA Youth Cup in 1968. Almost two years later, he became first team coach when Jimmy Adamson took over as manager. Brown became manager of Burnley in January 1976, but lost his job in February 1977. Despite being offered another job at the club he decided to leave. He joined Manchester United where he worked with the youth set-up. He eventually retired and lived in the Burnley area.