Free data a GFC casualty

The US government has been a proponent of free data for quite a while now and over the years it established a number of national programs to allow easy access to wast resources of public information. However, the annual budgets for e-government initiatives were slashed by 75% last month, putting in question the survival of such programs like data.gov (it is the repository for publicly available data that was promised as a platform to power software and analysis created by and for the public). Comments from federal CIO Vivek Kundra indicate that data.gov will not be shut down but “…there will be no enhancements or other development to address needs for improvement”. So, although the policy of free data remains unchanged, significant cost of delivering that policy may be its ultimate “undoing”.

Meantime, in Australia, the progress towards opening up government data vaults has taken another step forward. Earlier this week Australia’s Information Commissioner, John McMillan, unveiled eight new rules for Federal agencies to adhere to when considering the publication of government data. These rules are:

  • Open access to information – a default position,
  • Engaging the community,
  • Effective information governance,
  • Robust information asset management,
  • Discoverable and useable information,
  • Clear reuse rights,
  • Appropriate charging for access, [So, not entirely free access!]
  • Transparent enquiry and complaints processes

The Principles are not binding on agencies, and operate alongside legal requirements about information management that are spelt out in the FOI Act, Privacy Act 1988, Archives Act 1983 and other legislation and the general law.

Despite the launch of data.gov.au portal, there is no federal program in Australia to facilitate access to public data on a large scale (ie. the US style) and the onus so far is on individual agencies to manage the dissemination of public information in their possession. State and Territory governments are pursuing their own initiatives. This “piecemeal approach”, although slower in implementation, may prove to be a more sustainable model for enabling access to public data, considering the vulnerability of large scale initiatives to budgetary pressures of the government of the day in these uncertain times.

The Eye of the Storm

We all know the cliché that in the United States the vacations are limited to christmas eve, president’s day and the fourth of July. Bosses get nervous when their staff asks for a full week off, and two weeks is generally called sebatical… You can imagine what happens after a four week vacation that ends on the day that Hurricane Katrina makes landfall in Louisiana.

It is like hitting the wall of the storm again, right after being in the quiet eye of the storm for four weeks. My mind was set to complete the new version of the Geospatial One-Stop Portal (http://www.geodata.gov) for which I am project manager at ESRI and to think about the next steps in this project.

Little did I know how my schedule would be affected by Hurricane Katrina that cost many lives, cost business owners their life’s work, and destroyed many local governments databases, IT infrastructures, and buildings.

Apart from the human rescue operations, many GIS volunteers went to the affected areas and helped recreate the data and maps needed by first responders using equipment and software donated by private industries from accross the United States.

Within a week the United States Geological Survey initiated a concerted and focused effort aimed at creating a comprehensive seemless database of geographic information for the affected areas. This meant collecting data from many sources, in many different formats, assessing which data was usefull for the new database, creating mapping of the individual data models to a common data model, loading this data into the new database, and making this data available to those people who needed it in the first place. Apart from this effort, data is being collected in the field by first responders, areal photography is being acquired, and satellite imagery of the area is becoming available.

The amount of data that has become available todate after Hurricane Katrina made landfall is enormous. With the initial response activities changing in to a recovery operation. The amount of reports, data, and analysis results of the effects of the hurricane and of the reconstruction of the area that will become available within the next year is mind blowing.

To find ones way in this sudden wealth of information, metadata catalogs and search capabilities on those catalogs will play a key role. Here is where portals suchs as the Geospatial One-Stop Portal can have a significant contribution. These portals act as the card catalog for a library, allowing you to browse through a description of the information rather than requiring you to walk through the entire library in search of that one book. The new generation portals such as the Geospatial One-Stop portal provides more than just a card catalog with a search interface. These new portals provide for collaboration tools that allow users and producers of geospatial information to communicate with each other about a specific topic. This collaboration can be through the sharing of working documents, listing of and linking to relevant news feeds from a variety of agencies, or by participating in chat rooms.

Directly after the storm made landfall, a community of interest was set up on the Geospatial One-Stop Portal that was populated with information as it became available. This unstructured collection of resources was useful at the time (it was all there was!). However, as the stream of avaiable data grew, and after the realization that this information channel would exist for many months, a more structured approach would be necessary. This structure was provided by metadata about the datasets, clearinghouses, and mapping applications, that was published to the Geospatial One-Stop Portal. Existing resources already available on the Geospatial One-Stop Portal was updated to include a simple keyword ‘Katrina’ (and later on ‘Rita’) to indentify resources as relevant for the response and recovery effort. The Geospatial One-Stop Portal has proven its tremendous value as a mechanism for the dissemination of key geospatial resources to users of these resources.

The hurricanes also made it clear that although we all love the magic of web services, there are situations in which the good old floppy is unbeatable. The fact that 1.4 Megabyte floppydisk, has been replaced with a 120 GB Firewire Drive aside…

Appeared in GeoInformatics Magazine (www.geoinformatics.com) in October 2005