If you are going to try to stay in academia, it is the reputation of your adviser that is most important.
If you are going to look for a career outside of academia, the reality is the “name brand” where you got your degree can help you on the job market.
And in general, if you are going to go get a PhD it should be because you want to get a PhD. It is a long and arduous process and has no guarantee of high incomes like some other graduate degrees. You have to enjoy doing research in your field.