Born in Nashville, Tennessee, he began playing on a local semi-pro team at age 14 and dropped out of high school in his senior year to pursue his career. He was nicknamed "Junior" during his time playing in the Negro leagues with the Baltimore Elite Giants, where he was voted an All-Star three straight years from 1948 to 1950; veteran George Scales taught him to switch hit.
Gilliam made his debut with the Dodgers in April 1953, with the formidable task of taking over second base from Jackie Robinson, who was shifted to the outfield and third base; he proved capable, batting .278 with a team-leading 125 runs for the NL champions. His 17 triples led the NL, and remain the most by a Dodger since 1920; he was second in the league (behind Stan Musial) with 100 walks, and third in the NL with 21 stolen bases. For his excellent season he earned NL Rookie of the Year honors, as well as The Sporting News Rookie of the Year Award.
He continued to play well during the team's Brooklyn years, batting .282 in 1954 with a career-high 13 home runs before slipping to a .249 average for the 1955 champions; he scored over 100 runs both years, as well as in 1956. With the 1956 pennant winners, he batted a career-best .300 and made his first major league All-Star team, also finishing fifth in the MVP voting; he was again second in the NL in walks (95, behind teammate Duke Snider) and steals (21, behind Willie Mays). On July 21 of that year, he tied John Montgomery Ward's 1892 major league record of 12 assists in a game. In the Dodgers' last season in Brooklyn in 1957, he batted .250 but led the NL in putouts and fielding percentage and again finished second behind Mays in stolen bases.
He continued to star with the team after their 1958 move to Los Angeles, California, though he gradually shifted to third base; for the 1959 champions he led the NL in walks (96), along with 23 steals, and was again an All-Star, hitting a home run in that year's second All-Star Game. During the team's Los Angeles years, he moved back to second base from 1961 to 1963, batting .282 in the 1963 pennant year and placing sixth in that year's MVP vote; he also relinquished the leadoff role to Maury Wills in the 1960s, instead batting second in the order.
Gilliam was named a coach after the 1964 season, and intended to end his playing career, but team injuries resulted in his seeing substantial play at third base in 1965 and 1966, with the team again winning the NL championship in both seasons. In 1965 he was part of the major leagues' first all-switch-hitting infield, with shortstop Wills, first baseman Wes Parker and second baseman Jim Lefebvre. On September 5, Gilliam hit a 2-run pinch triple in a road game against the Houston Astros, giving the Dodgers a 3–2 lead in the 9th inning; the Los Angeles Rams, playing a preseason game against the Philadelphia Eagles at the Coliseum, were playing so poorly despite their 10–0 win that the biggest cheer from the stands came from people listening to portable radios tuned to the Dodger game who cheered when Gilliam got the hit.
He finally retired as a player following the 1966 season with a .265 career batting average, 1889 hits, 1163 runs, 65 home runs, 558 runs batted in, 304 doubles, 71 triples, 1036 walks and 203 stolen bases over 14 seasons.
Gilliam played in seven World Series with the Dodgers, four of them against the New York Yankees. In the 1953 World Series he singled to lead off Game 1, and had a solo homer in the fifth inning batting left-handed. He hit three doubles, scoring once and driving in two runs, in the 7–3 Game 4 victory; he had another homer, this time batting right-handed, in the 11–7 loss in Game 5. In Game 3 of the 1955 World Series, he drew a walk with the bases loaded in the second inning to give the Dodgers the lead for good, and he drove in the first run of the 8–5 Game 4 win; the Dodgers won in seven games for their first Series championship. In the 1956 World Series, he walked with one out in the tenth inning of Game 6 and scored on a single by Robinson to give the Dodgers a 1–0 victory, tying the Series; in Game 5 he had struck out and grounded out twice in the perfect game pitched by the Yankees' Don Larsen. In the 1963 World Series he scored the only run of Game 3 in the first inning after walking and advancing to second base on a wild pitch; after advancing all the way to third base on an error by Joe Pepitone in the seventh inning of Game 4, he scored on a Willie Davissacrifice fly to give the Dodgers a 2–1 win and a Series sweep. He was also on Dodger teams which won the Series in 1959 against the Chicago White Sox and 1965 against the Minnesota Twins. His final major league appearance was in Game 2 of the 1966 World Series against the Baltimore Orioles.
Gilliam served as a player-coach beginning in 1964, and became a full-time coach in 1967. He continued as a coach with the Dodgers until his death, including three more Dodger pennant teams in 1974, 1977 and 1978; they lost the World Series in each year.
Gilliam was respected for his personal qualities and sportsmanship, in addition to his playing ability, over his 28-year career with the Dodgers. Quotations about him include the following:
"What a great team player he was. He'd hit behind Maury, take pitch after pitch after pitch. And when Maury got to second, he'd give himself up by hitting the ball to the right side, even with two strikes, which most hitters won't do." - teammate Jeff Torborg, describing Gilliam as the ideal #2 hitter.
"He didn't hit with power, he had no arm, and he couldn't run. But he did the little things to win ballgames. He never griped or complained. He was one of the most unselfish ballplayers I know." - manager Walter Alston
"Father, friend, and locker room inspiration that will never be forgotten." - Davey Lopes, Dodgers second baseman from 1972 to 1981