Merkin's family fled Germany to escape Nazi persecution and came to New York City in 1940. Soon after coming to the United States, Merkin joined the Army as an intelligence and counterintelligence officer. After the war he purchased a seat on the New York Stock Exchange, and founded Merkin & Company, an investment firm. He met and married Ursula Breuer in 1950 in New York City and had six children (three sons and three daughters); at the time of his death, he had 20 grandchildren. He lived in New York City. Although he was known to many people as an investment banker, the actual bulk of his fortune came from his ownership of 37 per cent of Overseas Shipholding Group, which owns and manages the second largest fleet of transatlantic oil tankers in the world. Consequently, he was actually not so much a banker, as a shipping magnate.
Merkin was a good friend of Herman Wouk and the two were very involved in the founding of the Fifth Avenue Synagogue in Manhattan. He and his wife "Ulla" sponsored the well-known Merkin Concert Hall in New York City and also gave generously to Mount Sinai Hospital, Yeshiva University- sponsoring a chair in memory of his father Leib Merkin and a school in memory of his late father-in-law Isaac Breuer. The school (IBC) or Isaac Breuer College of Hebraic Studies is an undergraduate program. He served on Yeshiva University's Board of Trustees for over three decades and for a time held the position of chairman. He also gave generously to Re'ut at the Wayback Machine (archived February 8, 2002) and many other Jewish charities; the Merkins were particularly interested in furthering Jewish education through philanthropic gifts.