"Start!" is based on both the main guitar riff and bass riff of The Beatles' 1966 song "Taxman" from the album Revolver, written by George Harrison. Likewise, The Jam's "Dreams of Children" had featured the same "Taxman" bassline, played then as a lead guitar riff.
The album version of the song runs at 2:30 and features trumpets in the final section. The single version, also featured on the "Snap!" compilation, is edited and slightly remixed, and omits the trumpets.