You’re from a competitive high school, so it follows that most schools from your district outside of your high school will be “worse” academically. That doesn’t mean they’re “worse” high schools, and rankings don’t matter in college admissions as far as I can tell. A college won’t compare your current high school to your new high school and go “well, we would have admitted them, but because they transferred to a school ranked lower than their previous one, we’re going to deny them”. You get what I mean? As long as you do well on your tests and keep your grades up, participate in extracurriculars etc. I think you will be fine.
By “a lot worse” (referring to the language in your original post) do you mean like 36 ACT to 30 ACT drop or 36 ACT to 12 ACT drop? I would be worried if the difference is more like the latter. I thought you meant the area was unsafe, or there are less extracurriculars or something. Are there major differences in resources and course offerings?