Removing people from "Friends" List not a Crime

I recently invited a friend on my friends list to join my party who was in a party with a person who used to be on my friends list with the intention of having them both, hopefully, join. They did, and the former friends list member was understandably wondering why he wasn't on my friends list. I thought for a second and tried to remember. I said that the reason was because of the cussing. He said something to the effect of "I don't want to deal with you anymore" and left the party.

I felt no guilt or shame for removing him, though I did wonder why he was so offended by this. I know what it's like to be rejected, and I'm sorry for hurting him if I did, but it's entirely up to me who's on my friends list and who's not. I got tired of the cussing so I removed him--not to make him mad, to make me not mad. It's all about having a good time on LIVE, and he should play with people who are okay with his cussing, especially since there are people who are okay with it.

In any case, I hope that he will still have a good time on LIVE and not be mad at me for removing him from my friends list, a thing that appeared to him to be an unjustice towards him, but a thing that had nothing to do with hurting him. I wish him good times with his Halo 2 playing in the future because I know that there are people who are entirely okay with it and he should not have to change for anyone.