Drop a class?

I’m a senior and I’m currently taking 5 AP classes, 1 Bible class (I go to a Christian school), and I also TA. I’m struggling with one of my AP classes and really want to drop, but my counselor is telling me not to because I’d end up with only 5 actual classes and won’t be competitive, given the schools I’m applying to. However that class is also hurting my performance in other classes, and I don’t want to ruin my GPA the first semester. what should I do? Will it affect me by a lot if I dropped it?