The club was founded on 29 January 1983, upon the remnants of the previously well-known and successful club Highlands Park, which Sono opted to buy when he ended his playing career in the United States. The club was named Dion Cosmos in its initial 1983 season, with the first part of the name representing the previous sponsor of Highlands Park and the second part being the fingerprint of Jomo Sono, who decided to name his newly bought club after his former NASL club, the New York Cosmos. Since 1984, the name of the club has been Jomo Cosmos.
Sono's policy for development has always been to recognise and build upon raw talent. Sono's team accentuated and developed a strong youth policy and through the years has gained a reputation for discovering and developing some of the finest talent to have played in the league, for the South Africa national team and abroad. Under Sono's ownership, the club went on to achieve several successes: it won the National Soccer League in 1987, the Bob Save Super Bowl in 1990, the Coca-Cola Cup in 2002 and 2005 and the SAA Supa 8 in 2003.
In 2008, Jomo Cosmos were relegated from top-flight football for the first time since 1993. After one season in the National First Division, they returned to the PSL for the 2009–10 season, having won the Inland Stream and the promotion play-off against Carara Kicks. They were immediately relegated again, only to win the 2010-11 NFD championship and promotion back to the PSL, for the 2011-12 season. They were again relegated, and as of 2013-14, play in the National First Division.