This is very strange.
What does your mother think of this? I would talk to her privately when he is not around. Try to find out if he is really going to do it.
Once you are on campus apparently you won’t be living with them, even if they are living a short distance away. There is at least some hope that things will get better. For example, when you are in university he shouldn’t be able to get into your dorm without your permission, and you can at least say that you are too busy with classes to see him. He might be less controlling if you are just not there (in his house) for him to see every day.