In 1992, Bertram joined the indie rock band Lync, in which he played bass and sang. The band went on to release singles on Magic Pail, Candy Ass, Landspeed and K Records, and one full-length album on K Records, These Are Not Fall Colors, before disbanding in 1994. In 1997, Troubleman Unlimited and K Records collected the band's singles and unreleased tracks and released them on one compact disc, Remembering The Fireballs (Part 8).
In early 1995, Bertram formed the band Red Stars Theory with Tonie Palmasani, Jeremiah Green and Jason Talley. Red Stars Theory never officially disbanded, but have not released a record since 2000's Touch and Go release Life in a Bubble Can Be Beautiful. Suicide Squeeze Records released a compilation CD combining the band's out-of-print Rx Remedy 7-inch inch and Deluxe Records 10-inch. In 2000 Suicide Squeeze released the "Naima" 7-inch, a minimalistic cover of a song from John Coltrane's Giant Steps album. The b-side of "Naima" is "North to Next (exit)", a remix by Sientific American featuring prominent samples from the first two tracks of Life in a Bubble. In 2006 the band contributed the track "Evergreen and Ivorbean" to Suicide Squeeze's 10th-anniversary compilation, the CD/LP Slaying Since 1996.
For a brief period after Built to Spill's There's Nothing Wrong With Love was released in 1994, Bertram and Dave Schneider joined Doug Martsch as the touring version of Built to Spill. This lineup is captured on the live track "Some" on the album The Normal Years.
In the late 1990s Bertram had a short lived solo project called Pennsy's Electric Workhorse Songs. This project released two 7" records: a self-titled 4 song EP on All City Records in 1997, and a 7" single for the song Fransse in 1999, which was released by Suicide Squeeze Records.
In 1998 James was asked by the two members of 764-HERO, John Atkins and Polly Johnson, to play bass for a session on a radio station. The session went so well that Bertram was made a member of the band. He recorded two albums with the band, 1998's Get Here and Stay and 2000's Weekends of Sound. Bertram left 764-HERO shortly after a tour opening up for Modest Mouse in 2001. He was replaced by Robin Perringer.
Other recorded appearances
- Bass on One Foot in the Grave by Beck
- Piano on "Lie for a Lie," from the album Ultimate Alternative Wavers by Built to Spill