PE is mandatory freshman year. Last semester I missed a quiz that I couldn’t retake which dropped me to a B. I had all A’s in my other classes and am currently going strong with all A’s this semester. How much will that B affect my chances when applying to college?
Zip. Zero. Zilch. Nada.
In my school, PE wasn’t even factored into the GPA, so I’m sure you’re fine
Why even give grades in PE? Should be Pass/Fail.
An I never understood why they make kids wear those uniforms or otherwise get a 0 for the day. Make them sweat in their street clothes and sit in it the rest of the day. They’ll change next time. 
Does PE matter? Yes
Does getting a B in PE make a difference? No
All classes matter and should be taken seriously.