Lack of AP classes

I am a junior in high school and I have not taken any AP classes yet. Furthermore, college board has been recommending me to take AP classes. However, my high school only offers roughly 6 AP classes, and 4 of those can really only be taken senior year. But due to the way my class schedule works, at most I would probably only be able to take 3 AP classes next year. I am a bit worried to do that considering I have never taken any AP classes before, and suddenly taking 3 AP classes might be too difficult.

I have always tried to balance my classes with taking some less challenging courses with the highest level courses. But my school does not have any “honors” classes, just higher level classes with other names. Additionally, my school does not weight grades or even have class rankings. I’m also pretty sure students at my school do not get to join an honor society, at least until senior year. However, people I know in lower grades than me at other schools are already in honor society and take tons of AP classes. Even people on the forums here say their schools offer 16 AP classes, that they are 2nd in their class in terms of rankings, and that their weighted grade average is 5.41 or something similar. Basically, there is no way I can even compete with these people in these areas. I am a really good student with a 4.0 unweighted grade average, but I am worried that because of what my high school offers, I will not have anything extra to put on my resume for college.

Can anyone relate to this problem? Should I be worried about applying to colleges with barely any AP classes?

You have to compete against people from your school and other “lower performing” schools (schools with few AP’s, honors, etc.) NOT against schools that offer 16 or 20 AP’s :). it wouldn’t be fair otherwise.
However, if your school offers 6 AP’s, you should do your utmost to take them all.
Also, make sure your guidance counselor, in the school profile, indicates your school offers 6 Ap’s, that they can only be taken jr/sr year, and that the “advanced” classes are called “…” .
Most schools have such a profile, you can ask to read your school’s - but if it doesn’t or your guidance counselor hasn’t got one, you can always write down the information on your “brag sheet” senior year for your guidance counselor to use for his/her recommendation.