Building Dynamic Data Centers for Fast Delivery of New Data and Data Updates

Abstract

In this presentation I will demonstrate and explain the code that I wrote to build a dynamic data center – the CISER Data Archive Census 2010 Summary File 1 (SF1) Download Center – and its importance in the timely delivery of updated datasets at minimal cost. The code is simple, efficient, and easy to use. When updates are released by the U.S. Census Bureau (CB), I only need to update the source files and run the code to make current all files on the Download Center that are for consumer download. The code is designed to eliminate the multi-step process that consumers would have to undertake to get the information they want, apply the updates to or enhance the 49 file segments provided by the CB, create full data sets by merging the segments; make them available in SAS, SPSS, STATA, and CSV format and zip them for download; and automatically update the download center’s website including information about the size of the compressed (zipped) and uncompressed versions of the datasets. The process to build this dynamic data center is scalable and may be used as a model or guide by other repositories who are planning to develop their own.