Having taken the Vermonter numerous times, southbound is by far the most scenic (especially in the fall and winter!) because it runs through Vermont during the daylight hours. Depending on the season, you might get through part of VT during the daylight hours on the northbound run, but certainly not this time of year. I've never had a bad experience on the Vermonter, the conductors have always been fantastic even when we're delayed (one time was when we struck a very large tree on the tracks at full speed- 79mph- which delayed us about 3 hours). The train now runs quite a bit faster through VT with New England Central RR recently redoing the track through the length of the state, allowing nearly an hour to be shaved off the schedule if I'm not mistaken. w
I always get on in Waterbury (WAB) since it's the closest stop to where I live, and personally I think it's one of the nicest stations in the state since it was recently renovated by Green Mountain Coffee (which has it's corporate offices and their main roasting facility right next door, and their visitor center inside 2/3rds of the train station). So if you're at all into coffee, definitely check out their cafe!
If you're looking for two routes to knock off your list, in the summer it's possible to take the Adirondack northbound to Port Kent, NY (PRK), and take the Lake Champlain ferry over to Burlington. From there you could get on the southbound Vermonter at Essex Junction (ESX), which is about 15 minutes from Downtown Burlington and accessible by bus. Also, sometime in the next few years the Ethan Allen Express will be extended northward to Burlington from it's current terminus of Rutland, once track improvements are made.
Also, if you're looking to explore multiple towns along the Vermonter route, you can travel on the Vermonter between any two stations in VT for $12 (or less if the standard fare is less than that). You can go all the way from St. Albans to Brattleboro, or even just Essex to Montpelier for the same fare. More info is available
here