If you search this forum for something like "sleep coach seats" or "sleeper vs coach," there are an abundance of threads with comments from all kinds of people who have varying views on this subject. Some will ONLY travel by sleeper no matter what the cost, and others would never DREAM of spending the money for a sleeper.
In the end, it does come down to "How comfortable do you want to be?".
Personally, I sleep mostly fine in a coach seat (although as a railfan, I'm excited by the train travel itself, and so I consequently spend less time sleeping on a train than I would normally). It's been said around here (I think it was the_traveler who first coined it) that if you can sleep in your recliner at home, you'll have no problem sleeping on a train. While an Amtrak coach seat isn't quite as roomy or plush as a La-Z-Boy, the point is: if you can sleep in a mostly-sitting-up position, you'll probably do fine; if you're the type that MUST have a flat bed, then you'll probably end up regretting braving it in coach.
Personally, I have traveled by sleeper only twice, and neither was for an overnight journey (rather, I was booking the sleeper primarily to take advantage of the "first class" benefits of traveling in a sleeper on the newly-relaunched Coast Starlight service, which included access to the Pacific Parlour Car and an afternoon wine tasting). Having now traveled by sleeper (and I did lay the chairs down into the flat-bed position and try it out for a half-hour nap), I will say that I'm more likely to look at booking a sleeper in the future. However, because of the cash outlay required, there will have to be something extra to push me over the edge and go for it, like one or more of the following:
1) Upgrading for one segment or part of one segment on a long cross-country journey in order to take advantage of the bed and/or shower
2) An extra first-class experience such as a wine tasting, access to a special car, lounge access (i.e. the Metropolitan Lounge in Chicago)
3) Traveling with another person so as to split the cost of the upgrade (the upgrade charge is for the room itself, regardless of the number of occupants, which makes it a better deal as costs can be shared and all meals are included for both people)
4) A chance occurrence where coach seats are selling at a high bucket and sleepers are, strangely, selling at a low bucket, and so the difference between the two is too low to pass up
5) An on-board upgrade is available (which always sells for the lowest upgrade bucket price) and I'm so tired and out of it that, against my better judgment, I decide I really want the room!
Outside of a tempting-enough combination of the above factors occurring, I will likely continue to travel solely in coach. In your situation, given the prices you quoted and my inference that you'll be traveling alone, I personally would not upgrade the entire itinerary. I might consider upgrading on the Capitol Limited so that I could arrive in Washington, D.C. refreshed with a good night's sleep and a shower--factor #1. That upgrade would include factor #2 above, as I'd obtain use of the Metropolitan Lounge in Chicago and the Club Acela in D.C. I would be skipping, however, the chance for the wine tastings on the Empire Builder, but I'm not willing to pay that much extra for that privilege!
So, it's hard to determine whether the $800 upgrade cost gives you $800 in value, since each person's perception of the sleeper's value is different. For me, it wouldn't be an $800 value. $200 extra? It'd be a no-brainer, but Amtrak's not going to sell it for $200. $400--I would consider it but wouldn't be likely to book it. $800 for two people? It's getting more tempting. $350 for two people for an upgrade on the second train only? Now we're starting to get serious. But your tastes and financial situation may be different than mine, and so only you can really determine whether it's worth the $800.
Maybe you want to do what I would do and upgrade the second train only, and possibly even only on the outbound, since arriving in D.C. refreshed is more important than arriving home refreshed--if you sleep miserably in coach on the way home and feel all grimy, you can always take a shower and crawl into bed when you get there--you're not losing much there, whereas arriving in D.C. in miserable condition will ruin your first day and quite possibly set a negative tone for your entire trip. But this way, you can try out both worlds and determine your preferences, and you'll know which class to book in next time.
Hope this helps somehow!