Kallman was born in Brooklyn of Jewish ancestry. He received his B.A. at Brooklyn College and his M.A. at the University of Michigan. He published three collections of poems, Storm at Castelfranco (1956), Absent and Present (1963), and The Sense of Occasion (1971). He lived most of his adult life in New York, spending his summers in Italy from 1948 through 1957 and in Austria from 1958 through 1974. In 1963 he moved his winter home from New York to Athens, Greece, and died there at the age of 54. Kallman had been the beneficiary of the entirety of Auden's estate, but himself died intestate, with the result that the estate was inherited by his next-of-kin, his father, a New York dentist in his eighties.
Together with his lifelong friend (and sometime lover) W. H. Auden, Kallman wrote the libretto for Stravinsky's The Rake's Progress (1951). They also collaborated on two librettos for Henze, Elegy for Young Lovers (1961) and The Bassarids (1966), and on the libretto of Love's Labour's Lost (based on Shakespeare's play) for Nicolas Nabokov (1973). They also wrote a libretto "Delia, or, A Masque of Night" (1953), intended for Stravinsky, but never set to music. They were commissioned to write the lyrics for Man of La Mancha, but Kallman did no work on the project, and the producers decided against using Auden's contributions.
He and Auden collaborated on a number of libretto translations, notably The Magic Flute (1956) and Don Giovanni (1961). Kallman also translated Verdi's Falstaff (1954), Monteverdi's The Coronation of Poppea (1954) and many other operas.
- An Elegy (1951). New York: Tibor de Nagy Gallery. (pamphlet poem)
- Storm at Castelfranco (1956). New York: Grove Press.
- Absent and Present: poems (1963). Middletown: Wesleyan University Press.
- The Sense of Occasion: poems (1971). New York: George Braziller.
- The Rake's Progress (1951, with W. H. Auden, for music by Igor Stravinsky) New York: Boosey & Hawkes.
- Delia, or A masque of Night (1953, with W. H. Auden; published in Botteghe Oscure XII; never set to music)
- Elegy for Young Lovers (1961, with W. H. Auden, for music by Hans Werner Henze). Mainz: B. Schott's Söhne.
- Love Propitiated (pbd. 1963, for music by Carlos Chavez; first performed as Panfilo and Lauretta, 1957, then as Love Propitated, 1961). New York: Mills Music.
- The Bassarids (1966, with W. H. Auden, for music by Hans Werner Henze). Mainz: B. Schott's Söhne.
- Love's Labour's Lost (1973, with W. H. Auden, for music by Nicolas Nabokov). Berlin: Bote & Bock.
- Translations (published)
- Bluebeard's Castle (1952; translation of the libretto by Béla Balázs for the opera by Béla Bartók). New York: Boosey & Hawkes
- Falstaff (1954; translation of the libretto of the opera by Boito). New York: G. Ricordi.
- The Magic Flute (1956, with W. H. Auden, for an NBC Opera Theatre production of the opera by Mozart). New York: Random House.
- Anne Boleyn (1959; translation of the libretto by Felice Romani for the opera by Donizetti). New York: G. Ricordi.
- The Prize Fight (1959; translation of the libretto by Luciano Conosciani for Vieri Tosatti's opera Partita a Pugni). Milan: Ricordi.
- Don Giovanni (1961, with W. H. Auden, for an NBC Opera Theatre production of the opera by Mozart). New York: Schirmer.
- The Rise and Fall of the City of Mahagonny, by Bertolt Brecht. (pbd. 1976, with W. H. Auden). Boston: David Godine.
- Arcifanfano, King of Fools (pbd. with a recording, 1992, with W. H. Auden, after the opera by Carl Ditters von Dittersdorf).
- "The W. H. Auden Society: news, links, books, notes, etc.". Retrieved 2007-01-21.