Is it really that hard?

Is it really as difficult as they say it is to get into college?

Depends upon which colleges you are targeting. There are thousand of colleges that will gladly take a good student, you just have to spend the time researching which ones will fit your stats, interests and budget.