Goss served in the Royal Navy during World War I, and played football for the naval side. On leaving the navy, he joined the staff of Southampton F.C. as assistant secretary to Ernest Arnfield, taking over as secretary on Arnfield's retirement.
Following Jimmy McIntyre's surprise resignation as manager in December 1924, Goss assumed responsibility for team affairs under the supervision of the board of directors for the remainder of the season, continuing into the following season until Arthur Chadwick took over the manager's position in October 1925. During the period when he was acting as manager, the Saints embarked on a run in the FA Cup, defeating Liverpool in round 4, to reach the semi-final at Stamford Bridge, where they were defeated 2-0 by Sheffield United on 28 March 1925. In the semi-final Tom Parker had a dreadful afternoon, first scoring an own-goal, then suffering a rare miss from the penalty spot (shooting straight at the 'keeper) before a mix-up between him and goalkeeper Tommy Allen gave Sheffield their second goal.
In June 1936, the club were suffering serious financial difficulties and the entire board of directors resigned. The new board of directors asked manager George Kay to resign in order to trim the payroll, and in August Kay moved on to Liverpool, taking long-serving trainer Bert Shelley with him. Goss then combined the roles of secretary and manager, with club captain Johnny McIlwaine taking on the role of assistant manager.
Following the upheavals in the boardroom, the club were able to raise sufficient finances to purchase new players, most prominent of whom was the former Irish international forward Jimmy Dunne who was signed from Arsenal for £1000.
The 1936–37 season was not a conspicuous success and Saints struggled to avoid relegation from Division 2. On 16 January 1937, Saints played Sunderland in a First round FA Cup tie. Although Saints lost 2–3, this game was attended by a crowd of 30,380 which remained an all-time record attendance at The Dell.
By March, Goss and McIlwaine were feeling the pressure and Goss resigned his position as manager and in March 1937 was replaced by former Saints and Arsenal player Tom Parker.
At the end of the season Goss left the club completely, after over 13 years service, initially to run the Railway Hotel in St Denys. On the outbreak of World War II he re-joined the Royal Navy, taking command of a minesweeper before settling in Australia.