Kholmyansky became interested in Jewish revival and, above all, in learning Hebrew. Gradually, he became a prominent Hebrew teacher in the late 1970s and 1980s in the USSR, heading a clandestine network of Hebrew teachers in different cities across the USSR. At the peak of its activities, the network was active in 20 cities of USSR, with contacts in a further 30 cities. The KGB began close surveillance of Kholmyansky and arrested him in Tallinn, capital of the Estonian Soviet Socialist Republic on 29 August 1984.
While Kholmyansky was in preliminary detention, the KGB conducted a house search of his Moscow apartment, where they claimed to have recovered a shotgun, but his supporters among Soviet Jews and campaigners claimed that he was framed and that the charges were false. During the subsequent criminal investigation, however, the accusation against him of illegal possession of a weapon was dropped, leaving only the charge of illegal possession of ammunition.
The defendant proclaimed an unlimited hunger strike, which he also continued after he was sentenced. When Kholmyansky was hospitalized in the prison hospital, his weight was 42.5 kg (he is 177 cm tall). There, he was force-fed, spent a lot of time in the special punishment cell. It is widely believed that it was due to Kholmyansky’s staunch struggle that he received a relatively short term of imprisonment – 1 and a half years.
Some time after completing his term of imprisonment, Kholmyansky was granted an exit visa to Israel, where he arrived with his family. US Secretary of State George P. Shultz, three US Senators (including Ted Kennedy), ten US Congressmen, Margaret Thatcher, Bob Hawke and other public figures took part in the struggle to free Kholmyansky. His case may be seen as a valuable historical document about the struggle for human rights in the USSR in the period of the Brezhnev stagnation.
- "Refusenik", documentary movie by Laura Bialis. Clip: 
- "The Sound of Silence", memoir of Ephraim Kholmyansky (Russian: Эфраим Холмянский. Звучание тишины. — Иерусалим, 2007. При поддержке КЕРООР Hebrew: ק"ק מוסקבה)
- Reuter 15.10.84 «Hebrew teacher force-fed in Soviet jail»
- The New York Times, February 14, 1985 «Crime and punishment. The ordeal of a quiet Soviet Jew» by Anthony Lewis.
- The New York Times, February 5, 1985 «Soviet said to sentence popular Hebrew teacher to labor camp» by Seth Mydans.
- Washington Jewish Weekly 14.02.1985 «Kholmyansky Sentenced, Sharansky Ill»
- Jerusalem Post 07.12.1984 «English bishop supports imprisoned Soviet Jew»
- Enn Tarto. Estonian: Riigikogu liikme arupärimine