It’s not just hearsay… USC’s Honors College is among the 7 best in the nation for breadth of offerings, grant support, amenities, opportunities, scholarships, etc, etc. Calhoun is an excellent Honors College (ranked just below the “top7” - ie., as good for peers, but not as good for opportunities/support offered) and Clemson as a whole is more STEM-focused with fewer weaker students, but USC Honors is a the real deal. Personally I like Columbia’s environment better than Clemson’s, but that’s very subjective as Clemson’s campus is very nice too.
Honestly you should apply to both and see what you like best once you’ve been admitted.
http://publicuniversityhonors.com/new-top-programs-by-category/