Fellow Ipswich player Terry Butcher wrote of Hunter in his autobiography, describing an incident where Butcher had "crossed himself" before a reserve match:
Big Allan Hunter was sitting in the stand and after the game he grabbed me and asked me if I was a Catholic. I told him I wasn't. I was an English Protestant. Why, then, he asked, did I cross myself? I told him it was for luck but he told me to remember I was a Protestant and warned me never to do it again – if I did, he would really sort me out.
In 1982, Hunter accepted the role of player-manager at Colchester United where he appointed former Ipswich coach Cyril Lea as his assistant. Hunter did not last long in the position, and following the suicide of John Lyons in November 1982, he resigned from the club, leaving Lea in charge.