Transferring to a Worse Highschool

I guess its not “a lot worse” , but it’s definitely a tier below the school I currently attend. There aren’t any differences in terms of course offerings, in fact I think they offer the exact same classes. I’m mostly worried that colleges will look at this negatively because it is very rare for students to switch schools in the same district, especially from a good school to a worse one, since it’s not due to family relocation or moving.