He became friends with osteopath Doctor Stephen Ward. MI5 saw Ivanov as a potential defector and asked Ward to try to convince him to shift his allegiance to Great Britain.
Ivanov was at the party at the Cliveden estate when Christine Keeler met John Profumo, the British Secretary of State for War. Keeler's subsequent affair with Profumo went on at a time when she was also having sex with Ivanov. This was at a time when Cold War tensions were already heightened, just before the Cuban Missile Crisis. Ward and Ivanov are said to have asked Keeler to quiz Profumo as to when American nuclear missiles would be taken to then-West Germany.
When the Profumo affair became public, the ensuing scandal of Britain's defence minister having an affair with the mistress of a Russian spy resulted in a number of far-reaching changes. On a personal level, Ivanov's relationship with Keeler caused his wife to leave him, while back in Moscow, the Kremlin failed to show him much recognition. That double rejection manifested itself in heavy drinking by Ivanov.
Nearly three decades later, in early 1993, Ivanov met Keeler for dinner in Moscow. He later sent her a letter apologising for the way he had treated her in her attempt to get military secrets from Profumo. He also admitted there was a lot of envy in the KGB, because of his assignment.