Getting at Free Data
I’ve talked quite a bit over the past few years about how one can make data easily accessible and usable by users. TIGER/Linehas been one of my biggest examples. The web interface is a nightmare in usability and the FTP site is hard to use (and who really wants to download 150 GB of data?) unless you’ve got those FIPS codes memorized. There has to be a better way to access this data.
OK So Let’s See What This Thing Can Do
When I joined WeoGeo almost 2 years ago, we were working on adding vector support to the already powerful raster customization options. We’ve had vector support for a while now, but only recently had we added what we call ToC support so that you can manage large datasets such as this TIGER/Line 2009. ToC gives WeoGeo the ability to deliver many different shapefiles (or any other supported data type) as individual layers allowing someone to customize their order. In the case of TIGER, one could only get the hydro layer and only have to download that dataset. This means that you don’t need to get gigabytes of data to access lakes in your area of interest. One could of course download everything, but many times you just need one part of the dataset.
So I Work Across County Boundaries
TIGER County data is of course organized by county. In many regions, the reasons why these county boundaries were created have no bearing on reality or the facts on the ground. So organizing data by county might make sense to some mathematician sitting in a cubicle in the bowels of some government building, but to those of us who need to work with the data it is nothing but an impediment to building our products. The ToC (which is just a JSON file) gave us the means to offer up TIGER/Line 2009 data for the USA. Unlike the TIGER/Line website, you can easily select across counties and states for your data in one download. If I want Portland, OR and Vancouver, WA, it is a simple order. I don’t have to navigate through Oregon to Multnomah County and then back out to Washington and THEN back down to whatever county Vancouver, WA is in (seriously how do you figure that out on their website?). The clicks involved are just nuts.
Get Multiple County TIGER Downloads for Free
Take this example below; I can order all the TIGER/Line data between New York City and Philadelphia (including all those counties in New Jersey), select only the layers I want, clip it to my area of interest, re-project it and even deliver it in another format (such as TAB files).
How many trips through the TIGER/Line website would it take to grab all that data?
Grab only the TIGER layers you want, leave the junk behind.
Thus, WeoGeo is now hosting the entire TIGER/Line 2009 (TIGER/Line 2010 will come out as more states are added) for everyone to use. Because this dataset is about 150 GB, we are subsidizing the cost of hosting and delivery ourselves for basically anything smaller than the State of New Jersey1. This means that I can grab the Los Angeles Metropolitan area and get every TIGER/Line shapefile for free. Not a bad resource considering that you can place that order in less than one minute.
What is that? Census data for like 17 million people. For free!
So Yea, I’m Excited
Yup! Having the TIGER/Line 2009 Layers in WeoGeo and making it available free to users solves some of the problems I’ve seen with federal data sharing websites. While I do think hosted basemaps such as Bing and Google Maps are the future, having some free local data is still very valuable to those who need data unencumbered by licensing. And you can use our REST API to access this data outside of the website or with our MapInfo and ArcGIS connectivity.
Amazon put TIGER/Line 2008 on AWS a couple years ago and then let it die. We’ll keep our archive of 2009 and soon 2010 up for those who need it in the Amazon Cloud.
In addition, we’ve got lots of other datasets available for free on our Market including the whole FCC geo-database, natural hazard rankings, National Land Cover and Natural Earth Data. If you’ve got some good suggestions of free data that we should be including in our WeoGeo Market, please don’t hesitate to contact me via my email address which is located at the top of the sidebar on the right.
1: As much as I’d love to let you all download the whole thing as much as you wish, I still have my son who wants aniPad this Christmas. We just want to cover our costs here.