Walker was a key position player and a dual Best and fairest winner at St Kilda, with back to back wins in 1942 and 1943. In the fall of 1944, Walker was hit by a truck, while driving home from a party and was taken to hospital. He had broken his arm in three different places and suffered massive internal bleeding. Walker's career seemed to be over; in 1949, however, was able to fully recover and play football again. In the spring of 1950, Walker brought his team up to the towards the Australian football League again, but was not able to win. Despite the loss at St. Kilda, Walker decided to quit football as a player and become a coach.