Transferring Senior Year

I’m a current junior in a very small public high school that has about 300 students.This year has been extremely hard especially since I’m basically not learning anything and am having to teach myself at home, it’s currently at a point in which I’m unhappy in my school and feel like transferring schools.The problem is that I already transferred my freshman year to the school Im currently in and I’m planning on transferring to a higher challenging private school.My biggest concern is would my transferring affect the way colleges will view my application next year,especially ivy school will they view it as something negative?