You do not have to be religious to forgive. Forgiving others is a good, useful, positive and functional behavior. Doing so helps to unburden your mind. I do not accept you have to be religious to in order to forgive.
It seems to be the religious who expect forgiveness from God, without caring or acknowledging the wrong to the person/people they wronged.
I have no problem forgiving someone who acknowledges the fault, and is sorry for it. Unfortunately, there are a few instances in my life where someone wronged me and refuses to acknowledge their part in the rift. I don't need those people around me - glad to cut them out of my life.
A religious person would say "forgive them and let it go." But if they don't ask for forgiveness from me, only their god, why should I? They don't care. In their eyes, they are good with God and that's all that matters to them.
I don't want to set the example that you can shit all over me and I'm perfectly fine with that. I have to show myself some love and set about never letting that happen again. I have to stand by my principles and I don't need to be in the company of anyone who doesn't respect that.
On the other hand, I'm the least religious of my siblings. We had an abusive father. When he died, I think I was able to forgive his treatment of us better, as I came to understand he just didn't know how to show affection appropriately, due to the way he was raised. It was a different time.
I seem to have come through it with more introspection and reflection, rather than not understanding how an outside source (god) could have let it happen.