Rubini started to play basketball for his high school team in his native Trieste, where he graduated in 1941. The same year he began to play for Olimpia Milano, the most prestigious Italian League basketball club at that time. He however had a long-lived passion for water polo: this led him later to become one of the rare world sportsmen to play at the highest level in two different team sports.
As a basketball player, Rubini won 5 national titles in a row, from 1950 to 1954. After 1956, he devoted only to the coach role, winning 10 national titles with Olimpia. In these years, he set an unparalleled record of 322 victories and 28 defeats. As coach of the Milan team, Rubini totaled 488 victories, including the European Champions Cup (Euroleague) in 1966, and two European Winner's Cups in 1971 and 1972: these were the first international victories of Italian basketball clubs.