On January 20, 2005 the Silver Meteor I rode from D.C. to Jacksonville, FL was likely the roughest ride in my Amtrak experience. On the occasion of the 2nd inauguration of president George W. Bush, WAS was even more congested than usual. Then there was a derailment in the station on top of that, which put a halt to anything arriving from or departing for points north for much of the day.
The time required for clearing the backlog of arrivals and departures (there were also some cancellations) resulted in the Meteor departing WAS about 90 minutes late. The first couple hours of the trip were uneventful, and the Meteor remained behind schedule by about the same amount of time as far as Richmond, VA, where I was kinda surprised to find snow on the ground.
After RVR the train started making up time. Oh man. The Meteor was going like a bat outta hell, which of course magnified the impact of the really rough track. It was more like a wooden rollercoaster than a train ride. In addition, who knew there were so many grade crossings in the middle of nowhere in the South? With the rocking, rolling, bucking, and horn action, I seriously doubt anybody on that train got so much as a wink of sleep that night. Walking into the diner for breakfast the next morning (first one there) I remarked to the crew there that we'd made up some serious time overnight. Their responses made it pretty clear that they hadn't slept either.
The train was only 15 minutes late at JAX. No doubt some passengers were not exactly pleased with the overnight experience, but as a railfan with a serious need for speed, for me it had been one of the coolest rides of all time.
Some stretches out here might give the Silver Meteor a run for its money on the rough track front. Two Coast Starlight segments in California leap to mind: At the edge of the Salinas Valley between Salinas and Paso Robles there's some pretty bad track. The train doesn't generally run very fast there, so not much bucking, but those Superliners do some major to-and-fro swaying over some sections.
On a southbound starlight just north of Santa Barbara in 2008 the train was also making up time. Due to rough track, instead of passengers being asked to gather their belongings and be ready to detrain at SBA, everybody was told to remain in their seats until the train came to a stop. It was indeed quite a ride, though much shorter than the Silver Meteor's marathon rodeo.