In November 1973, guitarists Malcolm Young and Angus Young formed AC/DC and recruited bassist Larry Van Kriedt, vocalist Dave Evans, and Colin Burgess, ex-Masters Apprentices drummer. The Young brothers decided that Evans was not a suitable frontman for the group because they felt he was more of a glam rocker like Gary Glitter. The band had recorded only one single with Evans, "Can I Sit Next To You, Girl" with "Rockin' in the Parlour" as the B-side. In September 1974, Ronald Belford "Bon" Scott, an experienced vocalist and friend of producer George Young, replaced Dave Evans after friend Vince Lovegrove recommended him. The addition of Scott redefined the band; like the Young brothers, Scott had been born in Scotland before emigrating to Australia in his childhood and loved rock and roll, especially Little Richard. Scott had played in the Valentines and Fraternity. In a 2010 interview with Mojo's Sylvie Simmons, Angus Young recalled that Scott "moulded the character of AC/DC...Everything became more down-to-earth and straight-ahead. That's when we became a band."
The album was produced by Vanda & Young at Albert Studios in Sydney, Australia. George Young is the older brother of Angus and Malcolm, and also plays bass guitar on a number of the album's songs. Harry Vanda was a bandmate of George's in The Easybeats and the pair were the main songwriters of The Easybeats later hits, including their international hit "Friday on My Mind". When George Young heard what his younger siblings were up to, he was quite impressed, telling VH1's Behind the Music in 2000, "All of a sudden the kid brothers were still the kid brothers but, my God, they knew how to play. There was no sort of, 'Do they have it or don't they have it?' It was obvious that they had something." AC/DC was still developing its sound when High Voltage was recorded in November 1974, and singer Bon Scott and the Young brothers were backed by a different rhythm section than the Mark Evans/Phil Rudd combination featured on their next three full-length studio recordings. Rob Bailey and Peter Clack were the band's bassist and drummer, respectively, at the time. According to Murray Engleheart's book AC/DC: Maximum Rock N Roll, bass duties were shared by Malcolm and older brother George, who also played live with the band infrequently, as well as Bailey. Clack played drums on "Baby, Please Don't Go" and the rest of the tracks were recorded by Tony Currenti, although John Proud also did some session work. Malcolm and Angus traded-off lead guitar parts on "Soul Stripper" and "Show Business," and Malcolm played the solo on "Little Lover." In the book Highway to Hell: The Life and Times of AC/DC Legend Bon Scott, author Clinton Walker quotes Angus Young: "It was actually recorded in ten days in between gigs, working through the night after we came off stage and then through the day. I suppose it was fun at the time but there was no thought put into it."
While the songs on High Voltage showcase a glam rock influence that the band would soon discard in favor of a more ear-splitting hard rock sound, the foundation for the band's songwriting structures are clearly evident. As Angus told Benjamin Smith of VH1 in 2014, "I think the ‘60s was a great time for music, especially for rock and roll. It was the era of The Beatles, of the Stones, and then later on the Who and Zeppelin. But at one point in the ‘70s it just kind of became...mellow. When Malcolm put the band together, it was obvious what was missing at the time, another great rock band. So it was basically a reaction to that because the music at that point had just turned into that soft, melodic kind of period, and that seemed to be all over the world. For us it was a pretty easy choice, especially because Malcolm and myself – we’re two guitarists – so from the get-go it was going to be a guitar band." Six of its eight songs were written by the Youngs brothers and Scott, with "Soul Stripper" being credited to the Young brothers alone. "Soul Stripper" evolved from an older song called "Sunset Strip", written by Malcolm and the band's original singer Dave Evans, revamped for the album with new lyrics by Scott, and is similar in theme and structure to "Squealer," a song that would be included on Dirty Deeds Done Dirt Cheap in 1976. "Baby, Please Don't Go" is a cover version of a Big Joe Williams song and was chosen as the LP's first single, leading to the first of many AC/DC appearances on Australia's Countdown music program. The band's earliest appearances included a now-legendary live performance of "Baby, Please Don't Go" (featuring Scott dressed as a blonde schoolgirl) and a filmed performance of "Show Business." "Love Song" evolved from an unrecorded song called "Fell in Love", also written by Malcolm and Evans. This earlier version of the song had different lyrics, and the finished lyrics as heard on the album were added by Scott. "Love Song" was released as the album's first single (under the title "Love Song (Oh Jene)") and was backed with "Baby, Please Don't Go", but radio preferred the flip. In 1994, Bon Scott biographer Clinton Walker speculated that the uncharacteristically maudlin lyric to "Love Song" was likely a leftover from Scott's previous band Fraternity. "She's Got Balls" (about Scott's ex-wife Irene) was the first song that Scott and the Young brothers put together, while "Little Lover" had been a song Malcolm Young had been tinkering with since he was about 14 and had been originally titled "Front Row Fantasies" (Scott, who wrote the song about Angus, mentions glam rock star Gary Glitter by name in the song). 
High Voltage was originally released on Albert Productions only in Australia, and has never been reissued by another label in this format. The international version of High Voltage, that was issued on Atlantic Records in 1976, has a different cover art and track listing, with only "She's Got Balls" and "Little Lover" appearing overseas. "Baby Please Don't Go", "Soul Stripper", "You Ain't Got a Hold On Me" and "Show Business" were later released on '74 Jailbreak in 1984. "Stick Around" (about Scott's inability to hold onto a lover for more than one night) and "Love Song" have been released on Backtracks in 2009. The title and artwork were the suggestion of Chris Gilbey of Albert Productions. In the 1994 Scott biography Highway to Hell, Gilbey expalins that he came up with the concept of "an electricity substation with a dog pissing against it. It's so tame now, but back then we thought it was pretty revolutionary." Released in May 1976, and featuring a different cover, the international version of High Voltage was a compilation of the original Australian edition and its follow-up, the Australia-only T.N.T. album. Despite its title, the second High Voltage features only two tracks from the original: "She's Got Balls" and "Little Lover", with the other seven songs coming from T.N.T.
AllMusic deems this version of AC/DC "a very young band who were still coming into their own at the time, and that process of self-discovery is what makes the original version of High Voltage both the most inconsistent and unique of all the Bon Scott albums."