^ colby is in waterville, maine.
anyway, i go to middlebury and i love it.  it's in bumblef*ck vermont, but it doesn't sound like you're looking for a big city.  the social scene is mainly dorm parties and social houses (like frats without the greek) so i can see why some people might not find the "fun" party on a given weekend.  i have a lot of fun.  burlington is a cool place too, it's about 45 minutes away and we go there for concerts and nightlife when the great outdoors become a bit too much to bear.  i've got nothing but love for uvm, but for the record i'm not rich and i don't work all the time.  a fair bit though.
i'm studying abroad in valparaíso, chile right now.  we got some great skiing in at portillo and el colorado down here.  if you're interested in study abroad, middlebury has some of the best programs among the schools you listed.
skiing-wise, the snow bowl is a college owned resort 25 minutes from school with free buses running on the hour.  it's not big, but it makes skiing for a couple hours between classes very doable.  some exciting glades, too.  sugarbush is 45 minutes, along with mad river glen.  i also ski stowe a lot, which is a little over an hour away.  backcountry in the adirondacks is very popular, as is tuckerman's.
as far as the other schools you're looking at, i almost went to bowdoin (very similar) but the skiing isn't nearly as good.  in fact, i chose middlebury largely because of the skiing.  i'm from the west coast so i'm not familiar with the new york schools, except colgate, which i hear is a bit preppy, but fun.
any other questions about midd?