The subject matter of this song is surfing; this would be the group's last surfing-themed song until 1968's "Do It Again". In the lyrics, the surfers would try to impress the girls by not backing down from the waves.
AllMusic critic Richie Unterberger called the track a "relatively little-known treasure" and praised it for its "uncommonly anxious lyrics." Writer Andrew Hickey called the song "one of [All Summer Long's] best tracks." Author Philip Lambert said that "'Don't Back Down' positively explodes with surfing energy and vitality."