A former staff writer for PolyGram Publishing in Nashville, Pettis' musical career was started in 1979 when Joan Baez covered one of his songs, "Song at the End of the Movie", on her album Honest Lullaby. Following that release, Pettis became heavily involved in the "Fast Folk movement" in New York in the 1980s alongside artists such as Shawn Colvin and Suzanne Vega.
In 1984, Pettis released his first independent solo album, Moments. Signing with High Street Records in 1989, he made three albums with them: While the Serpent Lies Sleeping in 1989; Tinseltown, produced by Mark Heard in 1991; and Chase the Buffalo, produced by David Miner in 1993. None of these releases made Pettis a household name, but his music became extremely popular with other artists. The production on While the Serpent Lies Sleeping is erratic, apparently trying to balance a folk-rock sound with Pierce's mostly introspective and introverted lyrics. Pettis and producer Doug Jansen Smith argued often over the production, and did not work with each other subsequently. Mark Heard, Pettis's own choice as producer for Tinseltown gave that album a more straightforward folk sound, with the occasional touch of bluegrass or rock. The lyrics are also more provocative, and the album includes a few tracks that are basically protest songs. Heard and Pettis became close friends, and after Heard's untimely death in 1992, Pettis made a decision to include a Mark Heard song on every subsequent album of his own until Heard's songwriting abilities gained greater attention, a practice Pettis continues to this day. Chase the Buffalo, undoubtedly the most lyrically rich album of the High Street years, established Pettis firmly as a "songwriter's songwriter" and further developed the solid folk atmosphere of the previous album, adding more prominent bass and percussion instruments and starting to move away from keyboard sounds. Lyrically the album struck a fine balance between songs looking inward and looking outward.
When Pettis's contract with High Street ended, he signed with Compass Records, where he has remained since. 1996 saw his first release with them, Making Light of It, a low-key collection of songs, the majority returning to an introspective demeanor and tone, produced by David Miner (T-Bone Burnett, Elvis Costello), and featuring Derri Daugherty and Steve Hindalong of The Choir. Musically, "roots folk" would not be a bad description, though the tone is not old-timey in any way. Everything Matters followed in 1998, with an increased tempo overall and a few regionally oriented songs that explored and celebrated Southern cities and personalities. The music of this record was a delicate and successful blend of a more sparse "roots folk" sound backed by solid bass and percussion and produced by Grammy award winning artist Gordon Kennedy (best known for co-writing Eric Clapton's "Change the World"). 2001 saw Pettis's most regionally oriented album, State of Grace released, with a fuller, more straightforwardly folk tone and atmosphere. 2004's Great Big World record saw Pettis collaborating with a number of other songwriters for the majority of the tracks, with a still-present regional tendency, and similar sound musically to the previous album. The album's cover art was painted by the southern folk artist Terry Cannon. Great Big World featured musicians like Kenny Malone on percussion and bassist Danny Thompson of Pentangle fame. In 2009 That Kind of Love included less of a regional focus with a collection of mostly mid-tempo, personal and contemplative songs, although the three cover tracks on the album, from Mark Heard, Jesse Winchester, and Woody Guthrie, are uptempo blues or bluegrass. 2013 saw Pettis, along with Tom Kimmel and Kate Campbell, form The New Agrarians and release a debut album on the independent Due South label. Currently Pettis tours frequently, alternating between solo shows, concerts with The New Agrarians, and a double bill with his daughter Grace Pettis.
Pettis's songs have been covered by artists like Dar Williams ("Family" on Mortal City), Garth Brooks ("You Move Me" on Sevens), Dion & the Belmonts, Sara Groves, Randy Stonehill, Pat Alger, and others. Pettis himself has covered one of Mark Heard's songs on every album since 1993. These are: "Nod Over Coffee" on Chase the Buffalo; "Satellite Sky" on Making Light of It; "Tip of My Tongue" on Everything Matters, "Rise from the Ruins" on State of Grace, "Another Day in Limbo" on Great Big World; and "Nothing but the Wind" on That Kind of Love. Pettis's cover of "Nod Over Coffee" also appeared on a 1994 tribute album to Heard entitled Strong Hand of Love.
- Moments (Small World, 1984)
- While the Serpent Lies Sleeping (Small World, 1988, original release)
- While the Serpent Lies Sleeping (High Street, 25 September 1989, full release)
- Tinseltown (High Street, 18 June 1991)
- Chase the Buffalo (High Street, 13 July 1993)
- Making Light of It (Compass, 8 October 1996)
- Everything Matters (Compass, 16 June 1998)
- State of Grace (Compass, 10 July 2001)
- Great Big World (Compass, 3 August 2004)
- That Kind of Love (Compass, 27 January 2009)
- Due South Co-op (independent, 2013) as a part of the trio The New Agrarians
- Legacy: A Collection of New Folk Music (Windham Hill, 1989)
- A Winter's Solstice, Vol. III (Windham Hill, 1990)
- featuring "In the Bleak Midwinter"
- Strong Hand of Love, tribute to Mark Heard, 1994
- featuring "Nod Over Coffee"
- Orphans of God, tribute to Mark Heard, 1996
- featuring "Nod Over Coffee"
- Aliens and Strangers (Silent Planet, 2000)
- featuring "Kingdom Come"
- Beat (Silent Planet, 2001
- featuring "Absalom, Absalom"