The Psalm uses a variety of imagery, memorably in verse 12: "...As far as the east is from the west, so far he removes our transgressions from us..." 103:12.
An interesting note appears to the modern reader when considering verse 12. The significance of the compass directions being east and west in verse 12 instead of north and south, is that when traveling north you will eventually cross the North pole and begin traveling south, and the same vice versa. This does not happen when traveling east or west, and thus the difference between our transgression and ourselves is considered infinite. Although this line of reasoning is powerful for modern readers, the culture that produced this Psalm had a cosmology that viewed the world as flat and having four corners. In the original context, east and west function as a merism that implies infinite distance.
In the Eastern Orthodox Church, this Psalm is known as the "First Antiphon" and is sung at the beginning of the Divine Liturgy after the first litany. In some Byzantine traditions, this Psalm, and the other two Antiphons (Psalm 145/146 and the Beatitudes) were dropped in favor of different Antiphons with petitions to Christ and the Theotokos. The monastic communities of Mount Athos and most Slavic Churches have retained the ancient practice of singing these prayers during the Divine Liturgy.
The song Bless The Lord in the musical Godspell is based on this Psalm.