They’re more similar than they are different. But I think Pomona trumps Amherst in just about every respect save athletics (which I didn’t care about), racial diversity (very slightly), and general reputation (Pomona’s is rising, though).
I visited both schools and ended up choosing Pomona (among other choices as well). I would say despite looking on paper to be more diverse, Amherst felt a bit cliquey and not nearly as intimate or welcoming. The students seemed more independent and less happy than those at Pomona. I visited during the weekends for both schools so I didn’t really see many student studying. Someone at Pomona told me it was uncharacteristic because people study hard Sunday-Thursday and I wouldn’t really be seeing that.
I sat in on a humanities course at Amherst and a science seminar at Pomona. Both were fantastic- engaged students, brilliant professor, small and highly intellectual. They’re likely to have an equally superlative education. That being said, Pomona’s facilities on the inside stood out as much better than Amherst or any of the LAC peers. The dining hall and the dorms were better as well. They seemed to value undergraduate research, interdisciplinary studies, and independent study more than Amherst, but that may be due to the number of interdisciplinary programs and STEM majors they have. Pomona is almost 40-45% STEM majors, whereas Amherst is around 20-25%, so that changes the academic culture and priorities at both schools.
The two biggest points were the location and the consortium. While Amherst’s surrounding area is pretty awesome (much better than what immediately surrounds Pomona College, though Claremont and Amherst are similar as college towns), California and Los Angeles are just a whole different ball park. It’s actually very easy to get to LA and I’ve been to most famous sites here despite not having a car. LA is the type of city that offers much more the longer you stay, rather than for a 1-2 day visit. Cheap tickets at the world renowned Walt Disney Concert Hall, Broadway musicals at Pantages Theater, just about every famous singer or performer stopping by, amusement parks such as Disneyland and Universal Studios…the list goes on and on. Outdoor excursions and diversity is something in favor of California too. We have a really strong outdoor education center that takes students to world-renowned places like Yosemite, Joshua Tree, and Big Sur. Also don’t forget our beaches and snow-capped mountains for skiing, and the perpetually sunny weather 
For a longer trip, it’s pretty easy to get to San Diego, Las Vegas, or San Francisco, with the $2-30 Megabus running straight from Union Station. It’s easy to get to most places in LA after arriving at Union Station, and there is a train station 2 streets south of Pomona that takes you there. That being said, a longer trip (2-3 hours) will take you to Boston and NYC from Amherst as well, but it seems that Pomona students use LA more than Amherst students use larger cities. We actually have a competition called the 47 Things (http://www.pomona.edu/magazine/pcmfl04/47thingsLIST.shtml) in which students can earn $2000 for going out the most.
The consortiums at both schools are very different. Pomona’s is adjacent, with all 5 schools no more than a 15 minute walk from the south most part of Pomona. Amherst’s relies on a shuttle system, which, while convenient, is not the same in terms of accessibility. Pomona’s system has the more diverse schools- specialized schools like Pitzer, CMC, and Mudd offer great offerings in their areas of interests and unique programs that other small schools may not have, such as urban studies (Pitzer), accounting (CMC), and engineering (Mudd)- whereas Amherst’s 3 sister LACs are similar to each other and Amherst in academic emphasis. There is more bureaucracy involved to sign up for courses in Amherst’s consortium and as such the academic benefits are a bit more limited. At Pomona, every 5C course is listed in your registration and most are open to students from the 5C’s, with a few restrictions. The dining halls are shared among the 5C’s, as are some resources such as a library and the health center (under the Claremont University Consortium Umbrella). Events at all 5C’s usually tend to be open to all students, which greatly enhances campus life. 5C groups such as acapella, ballroom dance, and club sports make it easy to make friends across the colleges. I would say Amherst students are elitist about their institution compared to the rest of the consortium and are thus less likely to use it academically, but Pomona is surrounded by 3 of the most selective LACs in the country (Mudd, CMC, Pitzer, and Pomona all were in the top 5 for lowest acceptance rates of LACs) and the best women’s college on the West Coast. The atmosphere feels more mutual and tight-knit, hence why Pomona students often include “The Claremont Colleges” in their resume or list themselves as a Claremont Colleges alum in addition to Pomona alum. It’s not a perfect consortium, but it’s probably the best in the country.
Drug culture is the same at both schools. Many students drink alcohol, some do hard drugs, but at either school you can enjoy a good social culture without having to partake.
I think Amherst is more pre-professional than Pomona and more of their graduates go into fields such as consulting, investment banking, and business. Meanwhile, more Pomona students go into non-profits, social justice, and academia per capita. Amherst went co-ed in 1975 while Pomona was co-ed from the start, and that discrepancy is pretty apparent in the alumni networks at both schools. I think this is why Amherst does significantly better than Pomona on PayScale (which only looks at alums with bachelor’s degrees), because less students at Pomona go into super high paying jobs. That being said, I think in the last 3 or 4 years Pomona has really upped its offerings in pre-professionalism after realizing the changing market. While career development is supplementary to academics here, it’s becoming a stronger consideration. We have an internship program on campus that links students to internships around LA during the school year, a summer experience funding program, and an alumni winter break program. Student initiative has created an alumni mentoring program modeled after Amherst’s. Amherst also has a summer internship program hosted by alums and parents across the country, and a variety of school year internships in the surrounding area. Both schools have tons of grants available for students from various academic backgrounds to travel and find funding for independent projects. Their endowments are comparable, and both schools put in a lot of money for their students.