Carey played his club hurling with his local Roscrea club. He played a key role in the full-back position when the club won its very first senior county championship title in 1968. He won a second county title the following year before later adding a first Munster club title. Carey made it three county titles in-a-row in 1970, before collecting a second consecutive provincial title as well. These wins were later converted into an All-Ireland club medal. Carey won his last Tipperary county medal in 1972. He also won a Laois county title with Knock.
Carey first came to prominence in 1958 when he won his first Munster Championship title with the Tipperary senior inter-county team. Later that year he won his first All-Ireland medal following a win over Galway. Two years later in 1960 Carey captured his second Munster title, however, Wexford accounted for Tipp in the championship decider. The following year Carey captured his third Munster and second All-Ireland medals with victories over Cork and Dublin respectively. 1962 saw the Roscrea man win a fourth provincial medal. A victory over Wexford gave him his third All-Ireland medal from four attempts. Collectively, with fellow inner-defenders, Michael Maher (Holycross) and John Doyle (Holycross), he completed a very formidable trio as Tipperary's last line of defence. Their marshalling territory in front of goal was famously known as "Hell's Kitchen" because of the often tempestuous nature of the exchanges which greeted the dropping ball arriving from mid-field.
Two years later in 1964 Tipp continued their dominance with Carey adding a fifth Munster title to his collection. Once again this was later converted into a fourth All-Ireland medal following a win over their old rivals Wexford. In 1965 Carey captured another set of provincial and All-Ireland medals following easy wins throughout the championship. Two years later in 1967 Carey won a seventh Munster title. In the All-Ireland final Kilkenny emerged victorious over Tipp for the first time since 1922. Carey retired from inter-county hurling following this defeat.