Academically speaking, these schools are roughly equivalent, one from the Philly-NYC region and the other from the Boston region. There often is a bias that emerges when asking on this site about particular colleges, so make sure you take half that is said with a grain of salt.
There are some differences. The first is size. Haverford is one of the smaller LACs, and is significantly smaller than Colby. That allows for professorial mentoring, but it also becomes increasingly small as you complete the years. Size is less an issue at Colby.
The second is location. Haverford is in an affluent suburban area outside of a major city. Colby is more isolated, and may appeal to the more outdoorsy types. Waterville is not as affluent as Haverford, but some great improvements are going on in Waterville and on Campus. For that reason, if you have an interest in community service, Waterville may have more options than Haverford. That’s not to say that there aren’t lots of options in Philadelphia, but you have to be committed. Philadelphia is “close” but not really. Its about a 25 minute train ride to Philly, but you have to get to the train station. While the Haverford train station is very convenient, if you are walking to the station, add a ten minute walk from the college to the train (not to mention the predictable delays on SEPTA).
Haverford’s athletic facilities are fine, but Colby has a new facility slated to open soon. When it does, Colby will have it hands down in that department over Haverford.
Haverford is solid in the arts, as is Colby which has a very impressive art museum and is expanding its teaching based on its museum collection.
Haverford is very strong in the sciences, although Colby also is very good.
Colby has an excellent economics department, but chances are, so does Haverford. Again, ask a professor you know which is better regarded.
I strongly suspect the consortium is less feasible than many people think. Don’t get me wrong, I do believe that many Haverford students cross matriculate at Bryn Mawr. If you plan this out well, it can be done without owning a car. Taking a class at Swarthmore is going to be more of a challenge, since it is a 20 minute drive, depending on traffic. Taking the train would be a huge pain, as it would involve taking the train to Philly and then a separate train out to Swarthmore. Can it be done? Yes, of course, but it is not as easy as advertised. The same goes for taking a class at Penn, because after that 25 minute train ride, you need to walk another ten minutes to Penn’s campus. In fact, the same could be said for Colby students taking classes at Bowdoin or Bates.
The good news is that both of these school are really good. You won’t make a mistake which ever you choose.