But for more than a year after this celebration, local Centerville citizens complained about the boxcar that served as the town depot. Before any improvement could be made, property had to be purchased and Niles Road (later renamed Peralta Boulevard) had to be relocated to make room for a full-sized depot. Finally, more than a year after train service began to the town of Centerville, the Southern Pacific Railroad completed a new wood depot in September of 1910. The depot was built as a variation of what Southern Pacific called its "One Story Combination Depot No. 23." 

The Centerville depot was constructed at the height of popularity of rail travel, at a time when Southern Pacific trains provided a nearly universal web of transportation throughout California. For thirty years, the depot hosted passenger trains operating between Oakland, San Jose, Santa Cruz, Stockton, and even Redwood City for a few years. From 1910 through the early 1920's, the station served two or three daily milk trains, carrying the area's milk and cream to dairies in San Jose and Oakland. Local residents used the depot to commute to work, out-of-town relatives stepped off the train to visit, and immigrants arrived from the East Coast to begin new lives. 

But the automobile quickly made inroads into Southern Pacific's "local" train service. Ridership on passenger trains dropped, even though the area's population had steadily grown. On March 29, 1940, the last passenger train stopped briefly at Centerville, and then departed nearly empty, leaving public transportation to buses. Throughout World War II, up to 1958, the depot was used as a freight agency and Railway Express Agency, supporting the shipment of seasonal crops from the large Williams packing shed that stood about where the restored depot now stands. But to the public, the old depot -- like the railroad that served it -- slowly became invisible. 

Southern Pacific officially retired the depot on September 30, 1961, fifty-one years after it 
 was opened. Over the next thirty years, the building housed a furniture store, a spice store, a toy store, and an electronics store. By 1991, the depot was vacant, boarded up, and in a state of disrepair and deterioration. 

The inspiration to preserve the historic depot began with Fremont's petition to bring Amtrak service to Centerville in 1991. The petition worked. On June 4, 1993, Fremont welcomed the return of passenger trains to the Centerville station after an absence of 53 years. In December of 1993, the city of Fremont acquired ownership of the depot itself and began to plan its restoration. Of more than sixty "No. 23"-style depots constructed by Southern Pacific between 1896 and 1916, less than a dozen exist today and the Centerville depot is the only one used in rail passenger service.