The Old Town stop is walking distance to the Old Town State Historic Park ("Birthplace of California"). It includes a number of historic buildings dating to when the first Spaniards came to California, plus some replicas. Some house museums, there is an operating theater, and most buildings contain shops and restaurants. They have regular days where they have a blacksmith working in the blacksmith shop and let the burros out. Father Serra's first mission was on the hill overlooking the plaza here, Commodore Stockton's headquarters were here, the original graveyard is here, the last Mexican governor of California lived here, there is a famous haunted house here, some events from Ramona (influential novel) happened here, etc.
I haven't been for a couple of years, but there used to be a kitchen shop that had really high quality Mexican/Spanish seasonings. There are at least two top-notch restaurants, one in the plaza and one in the Cosmopolitan Hotel (which also has a pretty reasonable bed & breakfast).
If you have never been to California or San Diego, then the main Santa Fe depot is where to start. Otherwise, Old Town is a unique and worthy stop.