The question you have to answer is whether what you are seeking are credentials or an educational experience? If the latter you should stick with Wesleyan. You went to a lot of trouble choosing it; it’s the best fit. You haven’t said that you can’t afford it, only that Berkeley would be cheaper.
But, you knew that going in. What’s changed?
If the next four years is just about graduating on time with the proper credentials, then definitely go with the less expensive alternative. FWIW, unless you plan on working overseas, where the concept of a liberal arts college is difficult to explain, Wesleyan and Berkeley CAS have comparable reputations.