Besides the railroads already mentioned, I think both the Reading and the Chicago South Shore & South Bend were eligible and stayed out, probably because of the limited scale of their operations (I'm just guessing, I have no idea why).
I took a ride on the Reading in 1975 (or was it 1976?) from Pottsville, PA to PHL. The train was either a single, or maybe two, Budd RDCs. I don't know the story of pax rail on the Reading, but Pottsville to PHL would be an awfully long commute, so the limited scale of their operations might be the reason they stayed out...
You've happened upon one of the other odd situations. My understanding is that (most of) Reading's operations were ineligible for Amtrak because they were classified as "commuter" rather than "intercity" operations; there's a "magic" break point at 100 miles, IIRC, that served to divide the two (hence some of the rail situations in CT). The LIRR was also locked out of Amtrak even though a few lines served similar roles and ran over 100 miles.
With Pottsville, I think it was more Pottsville-Allentown and Allentown-Philly commuters than Pottsville-Philly. These services carried over to Conrail (which did substantial cutting back) before being transferred to SEPTA/NJT/etc.
Notably, the Penn Central was also locked out, and was stuck with random commuter operations all over the place that it couldn't shed until Conrail finally got rid of them in the 1980s (witness Amtrak's "Michigan Executive" as an example of the back end of one of these operations).
Chicago South Shore and South Bend was for some reason eligible to join (the commuter operations might have been just long enough to "clear the bar", but they ultimately declined. I don't know why exactly, but my best guess is that the commuter passenger operations were probably a large enough share of their business that they'd have had trouble keeping up the tracks without them. Thus you may have had a case of either a small profit or a small enough accounting loss (i.e. the operation was cash flow positive but it was weighed down by shared expenses and depreciation) to not bother joining.