Note: If you can't find the data that you need below, check out Starting the Hunt for a comprehensive list of data sites.

Data Available for All States.

Data for Specific States:
Arkansas | California | Hawaii | Illinois | Louisiana | Mississippi | Pennsylvania
All States
ArcData The good folks at ESRI provide free Census 2000 TIGER/Line data, as well as shaded relief images generated from the National Elevation Dataset (NED).
TerraServer TerraServer contains 3.3 tera-bytes of high resolution USGS aerial imagery and USGS topographic maps, with data available for virtually anywhere in the U.S. Check to see if your state has its own site for this data (below) before going here - it will probably be faster for you.
Seamless Data Distribution System Download data from the National Elevation Dataset (NED), the National Land Cover Dataset (NLCD), and the Shuttle Radar Topography Mission (SRTM). NED data is valuable for doing 3D work.
GISDataDepot A commercial site that offers free downloads of much of its data, including Digital Elevation Models (DEM), National Wetlands Inventory (NWI), and Landuse/Land Cover. Note that DOQs (aerial photos) are not free.
Census Data from ESRI

Download shapefiles of census blocks, block groups, roads, rivers, etc for your county.
Download a table with demographic data (number of people/race/age/sex/etc) and join to blocks, block groups, tracts, & county boundaries. See tutorial:

Geospatial Data Gateway By the USDA: NRCS. A great resource for natural resource data including aerial imagery, soils data, climate data, and more.
Arkansas
GeoStor Arkansas' interactive seamless warehouse of geodata, for all of your Arkansas geospatial data needs. Consider using the "Live Connection" option instead of downloading data layers.
California
California Spatial Information Library (CaSil) This is the best place to find goespatial data for California, however the data is in different projections. Stay tuned - we're developing a tutorial on how to handle that.
Statewide Data Index Most of this data is in the "Teale" Albers projection, NAD 1927.
Download the correct GeoMedia Coordinate System File (.csf) for the Teale Albers projection. Use your GeoMedia INI Wizard to associate this .csf with your data.
DOQ Data TIFF format in UTM Zone 10 or 11, NAD 1983 (If your longitude is west of -120, you're in zone 10; east of -120 you're in zone 11.) Be sure to download the World File (TFW) as well as the TIFF for each DOQQ.
Download the Index map for all California DOQQs - This index is a shapefile in Decimal Degrees, NAD 1983. The attribute "cell" contains the name of the TIFF and TFW files you'll need to download from the above DOQ Data link.
DRG Data Digital Raster Graphics - scanned USGS topographic map including the collar information
Hawaii
Hawaii Statewide GIS Program A comprehensive selection of GIS data for all the islands. Whenever possible, we recommend downloading and using the data that is in the "NAD 83" datum; "Old Hawaiian" is bad news. Note that many layers have been recently converted to NAD 83.
Illinois  
Illinois Natural Resources Geospatial Data Clearinghouse Includes DRG files (scanned topo maps), DOQ data (aerial photos), geology, land use, political boundaries, nature preserves, Public Land Survey, roads, water resources, wildlife areas, and year 2000 Orthoimagery for the Des Plaines River Watershed.
Louisiana  
ATLAS - the Louisiana State GIS Download a variety of useful raster data layers, including color-infrared aerial photographs (DOQs), GAP land cover data, scanned topo maps (DRGs), and (really cool!!) LIDAR data, which is high-resolution elevation data - great for 3D models and analysis!
Mississipi  
Mississippi Automated Resource Information System (MARIS) Tons of data for the lucky folks in Mississippi!!
Pennsylvania  
Pennsylvania Spatial Data Access (PASDA) Follow the link to "Access Data" for all kinds of good GIS data, including aerial photographs (DOQQs), soils data, and more. Use the Data Access Wizard to easily find what you're searching for.