"If anything be against justice... it is cruelty." ~ George MacDonald
Just laws are merciful.
Do we say a man is being unjust when he forgives someone? No. Perhaps it is just to forgive? If so it is just to be merciful.
And if it is true that "If anything be against justice... it is cruelty," then it is also merciful to be just.
So should the state simply forgive criminals? No, it is not the states role to forgive. The state should enforce just laws. (Which is a mercy, even for the criminal because eventually God will set everything right. See here). And the individual should choose to forgive.