That’s not easy to implement. (You would need to share with everyone the list of contacts you blocked).
Also you are tying to solve a social problem with technology here.
The closest thing to this is having a non democratic group-type where only admins can add/remove members.
See New feature: Group Types