The algorithm is based on a Fermat's little theorem
p prime, a & p coprime.
So, if we choose d such that
Then for a message m
So a message that is encrypted by raising m to e can be decrypted by raising the result to d.
In order to provide security, RSA actually calculates
(p, q prime) and then d such that
We can then continue to calculate
And likewise for q
Now if and then , p, q coprime.
An attacker would need to factor n into p and q in order to determine d, and this is a hard problem.
Note that while an attacker could easily calculate f as
because n is not prime.