He signed with the Hiroshima Toyo Carp in the Japanese Central League in 1977, and immediately became a vital part of the team in his first year, playing right field, and batting cleanup along with Koji Yamamoto and Sachio Kinugasa. The Carp won consecutive league championships from 1979 to 1980, and Lyttle contributed immensely to the team's Japan Series victory over the Kintetsu Buffaloes, where he was awarded the series MVP award. His strong arm won him the outfield Japanese golden glove award four years in a row from 1978, and he played his best season in 1981, leading the league in hits while slugging 33 home runs and 100 RBIs, with a .318 batting average. He also played in 472 consecutive games until 1981. He played six total seasons with the Carp; the longest of any non-Japanese player, and is regarded as the best non-Japanese player ever to play for the team. He spent the 1983 season with the Nankai Hawks before retiring.