Tepper met Beefheart (alias Don Van Vliet) by chance while studying as an art student in Northern California in the mid-1970s. Van Vliet had already started to focus on painting and although they shared this in common it was when Van Vliet learned Tepper knew guitar parts to Trout Mask Replica that he became interested in putting together a new band around Tepper's unique guitar sound. This was done later in Los Angeles after Van Vliet and Frank Zappa had reunited briefly for the Bongo Fury tour. The result of the new band led to arguably some of Captain Beefheart's most creative music efforts including albums Shiny Beast (1978), Ice Cream for Crow (1982) and Doc at the Radar Station (1980). Tepper joined The Magic Band in 1976 and stayed in the band until 1982. He is one of the longest serving (without a break) Magic Band members and the only one to receive production credit for Ice Cream for Crow (1982)
Tepper's first released solo work beginning with Big Enough to Disappear (1996) and then Moth to Mouth (2000) came after having been a sideman for many years. His music has been reviewed as "accessibly avant-garde" and although it may be difficult to shed his sideman cult status he remains inspired.Stingray in the Heart (2008) was described as an album that never pauses in a familiar territory.
Tepper is also a painter and he divides his time between music and painting.