The next major sporting event she took part in was the 2004 Athens Olympics where she finished second. The winner was the home hero Athanasia Tsoumeleka, who deeply moved the ecstatic Greek crowd by getting her country's first ever medal in the event (time 1:29:12). However according to many people Athanasia Tsoumeleka switched to running, going against the main rule of race walking. Ivanova finished four seconds later and could not hide her disappointment. She did, however, win the gold for the 20 km walk in the 2005 Helsinki World Championships, beating the world record. For this record she was added to the Guinness World Record book of 2007.
Ivanova was stripped of her silver medal in the 10 kilometer walk at the 1997 World Championships in Athletics after she had tested positive for stanozolol, and she was banned for two years.
Olimpiada Ivanova is married and has a daughter. Her passion is cooking.