According to gradeinflation.com, Harvey Mudd’s average GPA was 3.31 in 2008 (the most recent year for which it is reported). In that year (or in the closest year with a reported GPA) , the averages were equal or lower at Berkeley, Case Western, JHU, Lafayette, Lehigh, Michigan, MIT, Princeton, Reed, Rensselaer, St. Olaf, UCLA, UVa, Virginia Tech, Washington & Lee, Wellesley, and William & Mary.
Peer colleges with GPAs above 3.40 in the same year (2008) included Brown, Dartmouth, Duke, Northwestern, NYU, Rice, Swarthmore, and Vassar.
I wouldn’t be surprised to find some pre-med courses/professors at Rice and Swarthmore that have equal/heavier workloads and equal/lower average grades than similar courses at Harvey Mudd. But I’m just speculating. It’s very hard to assemble good data to support these comparisons.