Should I leave IB for an arts school?

Hello, I’m a sophomore currently enrolled in the IB Program, and I’m wonder if the benefits of IB are really worth it instead of being at a nearby arts school?
I really want to get into a good college, but nothing major like the Ivies. I feel like somehow I can struggle through IB to get the diploma, but if I’m already struggling so much and rather despise my current school’s setting, I think I would be a lot happier at the arts school focused on my interest in writing instead of being up until 3 in the morning working on assignments and being stressed out over all the IB requirements, like CAS and EE. I don’t want to regret not pursuing something i love, but i also don’t want to put my future at risk.
I realize that being in the Program is supposed to prepare you for college, but I’ve managed to get straight A’s throughout my duration in IB (but I know the genuine struggle comes during junior year), so I think I’m ready to struggle for college when the time comes? Do the benefits really outweigh all of this work we’re doing?
I don’t know, I’d really like to enjoy my high school life being surrounded by people who love writing and take writing classes for 3 periods a day, but I also want to get into a good college and have a good transcript.

Thank you so much for your help!

Isn’t the IB alllll about writing LOL? You cite the EE as a problem, but you want to write? If anything, the IB system works against the kid who wants to do 3 HL sciences and HL math, the IB program is MADE for the kid who wants to write. CAS should never be an issue to a US high schooler, it is very minimal and you shouldn’t really have to think about it beyond the reflections.