Do I have to be passionate about math/science to be a doctor?

I’m not bad at either subject, but I am much more interested in the humanities(and art, but I’m not good enough to really be serious about it).

To go to medical school, you can major in anything you want, but you have to take certain med school pre-req classes
Minimum course requirements include one year each of biology, general (inorganic) chemistry, organic chemistry, physics, and related lab work for each. In addition, about two-thirds require English and about one quarter require calculus.
So you don’t have to be passionate about science but med school is applied science so you need to be able to do it.

You need to be passionate about people in all their states and most of all be really, really hard working.
Majoring in the Humanities (especially Philosophy or English, plus to a certain extent foreign languages) can be an advantage for Med School as long as you can handle the premed pre-reqs.