In the late 19th and early 20th centuries, a shotgun messenger was a private "express messenger" and guard, especially on a stagecoach but also on a train, in charge of overseeing and guarding a valuable private shipment, such as particularly the contents of a strongbox (on a stagecoach) or safe (on a train). The express messenger for stagecoaches typically rode in a seat on top of the coach, on the left next to the driver (who typically sat on the right side, operating the wheel brake with right arm). In the Old West of the 1880s, if a stagecoach had only a driver and no Wells Fargo messenger, this meant the coach carried no strongbox, and was thus a less interesting target for "road agents" (bandits).
Wells Fargo Co. express messengers typically carried a short (or sawed-off) 12- or 10-gauge double-barrelled shotgun, loaded with buckshot. This was a most effective weapon in use against pursuing riders. Such weapons were sometimes referred to as "messenger shotguns" or, more commonly, "coach guns" (a name still used today). To some extent these weapons also carried over to use by private guards in trains with strongboxes or safes, where they were again effective.
Like "gunslinger", the actual term "riding shotgun" first appeared in fiction about the Old West, dating back as far as the 1905 book The Sunset Trail, by Alfred Henry Lewis. See also "calling shotgun" which dates from use in autos to about 1954, at a time it was being used in the popular TV series Gunsmoke.