Should I be worried about this?

If I got accepted to Cal Poly Pomona for Computer Science, but missed the deadline for the Keloggs Honors Program, should I be worried? What are the cons of this mistake.

No. Your acceptance will still stand regardless if you applied to the Honors college or not. Not applying means you do not get the privileges of being an Honors college student which the biggest perk is priority class registration.