With the Music I Die is the debut album and second extended play by American singer-songwriter Wynter Gordon, who subsequently adopted the stage name Diana Gordon. The full album was released through Big Beat Records exclusively in Australia on June 17, 2011. A cut-down EP version of the album was released in the United States and United Kingdom. It was preceded by the release of three singles: "Dirty Talk", "Til Death", and "Buy My Love", which all peaked inside the top three of the Hot Dance Club Songs Chart.
"Dirty Talk" was released as the lead single from the album and was well-received from critics. It was praised for its sensual lyrics and fun production. "Dirty Talk" topped the US Hot Dance Club Songs chart and Australian singles chart. It became a top 20 hit in Belgium, Scotland and Ireland, and a top 40 hit in Denmark. The song peaked at No. 25 in the UK, where it stayed for 3 weeks and peaked at No. 6 on the UK dance chart. It also charted in France and Romania. The album's second single was "Til Death". The song was remixed by Denzal Park and was released as the official single across Europe and Australia. It debuted at No. 24 in Australia, and peaked at No. 16. It peaked at No. 3 on the US Hot Dance Club Songs Chart.
The album's third single was "Buy My Love" and a music video for the song was released online on September 8, 2011. The song debuted on the US Hot Dance Club Songs Chart at number 45 and has so far peaked at number 2 and is the third single to chart in the top 5. "Still Getting Younger" was released on June 1, 2012, as the fourth and final single from the album. The music video for the song was released online on June 11, 2012. It peaked at number 24 on the US Hot Dance Club Songs Chart.