At the 1948 Olympics in London, Whitfield won the 800 m and was a member of the winning 4 × 400 m relay team. He also earned a bronze medal in the 400 m. At the 1952 Olympics in Helsinki, Finland, he repeated his 800 m victory. He also earned a silver medal as a member of United Statese 4 × 400 m relay team. He set a world record in 880 yd (800 m) of 1:49.2 in 1950 and dropped it to 1:48.6 in 1952. In 1954 Whitfield won the James E. Sullivan Award, given annually by the Amateur Athletic Union of the United States (AAU) to the outstanding amateur athlete in the country. He was the first black athlete to win the award. Whitfield narrowly missed making the 1956 Olympic team while a student at California State University, Los Angeles and he retired from track competition shortly thereafter. After graduating he worked for the United States Department of State, conducting sports clinics in Africa.
In his 47 years in Africa, Whitfield trained and gave consultation to dozens of athletes who represented their countries as Olympians and All-Africa Games champions. Whitfield also arranged sports scholarships for over 5,000 African athletes to study in the United States. During his career as a diplomat, he traveled to over 132 countries and played a key role in training and developing African athletes. The President Ronald Reagan wrote of him: "Whether flying combat missions over Korea, or winning gold medal after gold medal at the Olympics, or serving as an ambassador of goodwill among the young athletes of Africa, you have given your all. This country is proud of you, and grateful to you." Shortly after his retirement from government service in 1989, Whitfield was invited to the Oval Office, where President George H.W. Bush recognized his service to the nation and the world.