ogc | Mapsys.info (3) Mapsys.info

The New GeoServer 2.1

GeoServer 2.1 (Major Update)

“GeoServer is a Java-based software server that allows users to view and edit geospatial data. Using open standards set forth by the Open Geospatial Consortium (OGC), GeoServer allows for great flexibility in map creation and data sharing.”

18 months in the making:WMS Cascading

New Features

Something many users have asked for since the very addition of WMS support is cascading, the ability of GeoServer to proxy for another remote WMS server (be it GeoServer, MapServer, or ArcGIS). This feature has many uses, such as pulling in a remote base layer and overlaying local vector data onto it or securing a locally unsecured map server. Special thanks to the University of Perugia for sponsoring this feature.

Virtual Services

Anyone who has published a large number of layers or feature types with GeoServer has probably at some point been annoyed by the fact that every single layer is published by a single global service. WMS has the ability to group and nest layers but WFS and WCS have no such equivalent. Thanks to virtual services, one can create multiple service endpoints within a single GeoServer instance. Special thanks to Landgate for funding this work.

Layers from SQL

GeoServer has always been good at publishing a flat database table but users often need to do more—such as pre-filter the data in a table, join two tables together, or generate column values on the fly with a function. Before this feature, the recommendation was to create a view, yet views can be a maintenance burden and are at times problematic.

Now one can create a layer directly from an SQL query and query definitions can be parameterized to create dynamic queries on the fly. These parameters can be restricted with regular expressions to prevent an SQL injection security hole.

Special thanks to Andrea Aime for spending much of his personal time on this one and to OBIS for providing the funding for the parametric component of the work.

Web Processing Service (WPS)

With 2.1 and the arrival of WPS we welcome a new OGC service into GeoServer. The Web Processing Service is an OGC service for performing geospatial analysis functions over the web. The specification is extensible in nature and allows for simple processes ranging from buffering a geometry to more complex processes such as image processing.

Historically, GeoServer has been focused on data delivery and did not provide tools for performing analysis on spatial data. WPS fills that gap, making GeoServer a more compete solution for geospatial web services.

Thanks to Refractions Research for the initial contribution of the WPS module and to Andrea once again for taking personal time to bring WPS support to its current state.

Unit of Measure

Support for units in SLD allows one to specify values in measurements other than pixels such as feet or meters. This adds a very powerful capability to SLD that may alleviate the need for multiple scale-dependent rendering rules and may greatly simplify complex SLD documents.

Special thanks to Milton Jonathan for the initial GeoTools work to make unit of measure support possible and to Andrea for working with Milton to improve the initial patch. Note that this feature has also been backported to the stable 2.0.x branch thanks to support from SWECO and Malmö City of Sweden.

DPI Scaling

GeoServer renders images at a resolution of 90 DPI by default. While this is acceptable for most standard screen resolutions, it is unacceptable for higher-resolution printed materials. Now it is possible to supply a format option to a WMS request to control the DPI setting on the fly.

Special thanks again to SWECO and Malmö City of Sweden for sponsoring this work. Note also that this feature has also been backported to the 2.0.x branch.

Graphical File/Directory Chooser

Ever found it difficult to remember the full path when loading a shapefile or GeoTIFF? This new improvement brings an easy graphical file and directory selection tool for browsing the file system on which GeoServer resides. This is definitely a great enhancement to make GeoServer even easier to configure.

Core improvements to support a database-backed catalog

GeoServer’s core catalog interfaces received some tweaks to more easily support different backend storage formats. The current in-memory implementation has a number of drawbacks—the most notable being that it is memory-bound, meaning it cannot scale to accomodate large numbers of layers. Support for specific new storage formats is still only available as a community module but these core improvements make it possible to more easily swap between different backends.

Font Improvements

Adding new fonts to maps should now be much easier, as one can drop font files directly into the GeoServer data directory and have them be recognized by GeoServer. The admin interface will list all fonts that are currently available, including those made available by the Java Virtual Machine.

Upgrade to Spring Security 2.0.6

GeoServer has always had Acegi Security at its core but since that library was absorbed by the Spring community it has been improved and upgraded into Spring Security, the official security module of the Spring portfolio. This brings a number of new security protocols to GeoServer, including OpenID and Windows NTLM. With even more powerful options, it should now be easier to customize the security setup.

WCS Limits

While WFS and WMS have both had the ability to limit what a user can request, now similar controls are in place for WCS calls as well. Thanks to MassGIS for funding this improvement.

WMS 1.3.0

WMS 1.3.0 is the newest version of the Web Map Service protocol. Special thanks to Ordnance Survey, Britain’s national mapping agency, for providing OpenGeo with funding to complete its implementation in GeoServer. With WMS 1.3 mandated by the INSPIRE Initiative, Ordnance Survey opted to fund the GeoServer project so that other organizations in the UK and the rest of Europe could meet their INSPIRE requirements and everyone across the globe could benefit.

SLD 1.1 / SE 1.1 enhancements

Though not every new option is fully-supported, it is now possible to use most SE 1.1 documents in GeoServer. User feedback on which new options should be supported first is greatly appreciated. Also funded by Ordnance Survey is a community module to implement the WMS extensions for INSPIRE View Service compliance—namely the language parameter and several extended capabilities fields.

GeoWebCache Integration

GeoWebCache integration allows clients to enjoy the benefits of tile caching through the regular GeoServer WMS endpoint. This enables GeoWebCache to transparently proxy for the GeoServer WMS without the need for a separate service endpoint. Taking advantage of the recently-added disk quota functionality, GeoWebCache now provides the ability to set limits on the amount of disk space used for storing tiles, allowing users to control and limit the size of the tile cache on disk. Big thanks to Gabriel Roldán of OpenGeo for the great GeoWebCache improvements.

Improvements to RESTConfig

This release also brings some improvements to RESTConfig, which is now shipped with GeoServer by default. Improvements to the API include a file upload operation that now allows for uploading files into an existing data store. This addition allows users to upload a shapefile and have it automatically converted into a PostGIS database and published as a PostGIS layer rather than as a Shapefile layer. The API also supports recursive DELETE operations, making it more convenient to remove resources that contain other resources such as stores or workspaces. Thanks to David Winslow and Justin Deoliveira of OpenGeo for these improvements.

Improved Raster Reprojection Performance

Thanks to some great work from GeoSolutions, raster reprojection performance has been significantly improved by using linear appoximations of transformation functions.

WCS Request Builder

Thanks to Andrea Aime, there is now a Web Coverage Service request builder for graphically building WCS requests to test a coverage service. As clients for WCS have always been sparse, this tool goes a long way towards making the service more usable.

Run multiple GeoServer instances from a single data directory

There now exists a new parameter that will once again allow multiple GeoServer instances to run from a single data directory. This parameter, named “GWC_DISKQUOTA_DISABLED”, will disable the GeoWebCache disk quota module and prevent it from maintaining a lock on the data directory.

Source:
http://blog.geoserver.org/2011/05/12/geoserver-2-1/

Convert, Reproject, and Load Spatial Data to SQL Server with OGR2OGR

I used OGR2OGR to join a set of shapefiles together prior to loading them into SQL Server using Shape2SQL. But OGR2OGR can do much more than simply appending shapefiles – it can convert data into different formats, transform coordinates into different spatial references systems, read and write a whole variety of spatial datasources, and (with a bit of fiddling) load them into SQL Server 2008.

To demonstrate, I thought I’d repeat the objective of my previous post, but instead of simply appending the Ordnance Survey open data shapefiles together, I would reproject them into a different SRID, and import them into SQL Server too using OGR2OGR alone, without using Shape2SQL or any other tools.

Use OGR2OGR to create a CSV file containing WKT

OGR2OGR can’t insert spatial data directly into SQL Server 2008, but it can create CSV files that can be read into SQL Server. To create an output file in CSV format, set the –f CSV output option. You can also manually set layer creation options to dictate the field delimiter and line return character of the CSV file, using the LINEFORMAT and SEPARATOR layer creation options.

By default, the CSV file created only contains columns containing the attributes associated with each shape (i.e the data in the .dbf file associated with a shapefile), but it doesn’t include the shape itself. To include the shape of the geometry in Well-Known Text format, you also need to state the –lco “GEOMETRY=AS_WKT” layer creation option.

But that’s not all – OGR2OGR can also reproject data between coordinate systems as it converts it. To convert to another spatial reference system, include the -t_srs parameter, together with either the full WKT definition (hard to escape properly,i.e. GEOGCS(ELLIPSOID(“WGS 84”, ……) or the EPSG code (much easier, i.e. EPSG:4326 for WGS84).

Putting this all together, you can use the following command to load a shapefile (in this case, the Ordnance Survey OS Vector Map settlement_area polygon shapefile), reproject the data into SRID 4326, and then save the result as a CSV file containing a column with the WKT of each resulting polygon:
[php]ogr2ogr -f "CSV" "Settlement_Area" "Settlement_Area.shp" -t_srs "EPSG:4326" -lco "GEOMETRY=AS_WKT" -lco "LINEFORMAT=CRLF" -lco "SEPARATOR=SEMICOLON" [/php]
NOTE – I’ve noticed some problems with the quote/speechmark characters in this blog being changed to “smartquotes”, even when I use the code-formatting option. OGR2OGR isn’t happy about this, so I recommend that you don’t try and copy and paste the line above into a CMD window – if you get an error about “Couldn’t fetch request layer uT” or something similar then be sure to retype using normal speech marks.

More information on CSV options for OGR2OGR can be found here: http://www.gdal.org/ogr/drv_csv.html

Create a Format File

To import the CSV into SQL Server, we have a range of options – you could use SSIS, or you could use the Tasks –> Import Data wizard. Or you could use BCP or BULK INSERT. However, I’m going to use OPENROWSET. This will allow me to write a single query to access the CSV file directly from T-SQL as if it were an existing table.

In order to do this, we first need to define a format file which specifies the datatypes of each of the columns in the input csv file, and how they should be mapped to datatypes of columns in a sql table. More information on format files can be found here.

The following shows the XML format file required for the OS VectorMaps Settlement Area shapefile just converted:

[php]<?xml version="1.0"?>
<BCPFORMAT xmlns="http://schemas.microsoft.com/sqlserver/2004/bulkload/format" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<RECORD>
<FIELD ID="0" xsi:type="CharTerm" TERMINATOR="""/>
<FIELD ID="1" xsi:type="CharTerm" MAX_LENGTH="2147483647" TERMINATOR="";"/>
<FIELD ID="2" xsi:type="CharTerm" TERMINATOR=";"/>
<FIELD ID="3" xsi:type="CharTerm" TERMINATOR="\r\n"/>
</RECORD>
<ROW>
<COLUMN SOURCE="1" NAME="WKT" xsi:type="SQLVARYCHAR"/>
<COLUMN SOURCE="2" NAME="FEATCODE" xsi:type="SQLDECIMAL"/>
<COLUMN SOURCE="3" NAME="FEATDESC" xsi:type="SQLVARYCHAR"/>
</ROW>
</BCPFORMAT>
[/php]
Query the CSV from SQL Server using the Format File

Finally, you can write a SELECT query in SQL Server to query the CSV file using OPENROWSET in conjunction with the format file above. Since the CSV contains column headers, I’ve included the FIRSTROW=2 parameter to skip the header row. I’ve also used the geometry::Parse() method in the SELECT to dynamically create the geometry data from the Well-Known Text representation contained in the WKT column of the supplied CSV file.
[php]SELECT
FEATCODE,
FEATDESC,
geography::Parse(WKT)
FROM OPENROWSET(
BULK ‘C:Settlement_Area.csv’,
FORMATFILE=’C:Settlement_Area_format_file.xml’,
FIRSTROW = 2
) AS CSV;[/php]
And here’s the results showing my Ordnance Survey data, originally provided as a shapefile using projected coordinates in SRID 27700, now loaded as latitude/longitude coordinates (EPSG:4326) using the geography datatype:

Testimonials as Reviews – A View from the Field

Last week after publication of the new Google Rich Snippets for Local FAQ and follow up comments by Carter Maslan, there was a great deal of discussion about what was best practice vis a vis testimonials on a SMB web site.

Clearly, when you focus on the client, having this type of content can be a good practice and as David Mihm pointed out there are a number of reasons that semantic markup of the testimonial information on an SMB website makes sense going forward, Carter’s ambiguous comments not withstanding.

All that being said, the question of how Google is actually handling testimonial information in Places is still of interest. Are they or are they not including it in the Places Pages as reviews?

The answer is yes, in a limited way.

The Data

I have been looking for examples of Google’s use of owner website pages as reviews since October of 2010. Some of these examples were discovered by research but many are user contributed (I apologize for the lack of credit but I just lost track of them all). I am sure that there are more examples of Google including testimonials on the Places Page and I would encourage you to send them along. It will add to the very limited view we now have of Google’s use of this content in practice.

I am presenting the data in two groups of results, hotels and SMBs, as I think that Google is handling the two cases differently.

Google is broadly including positive content from hotel websites as reviews on their Places Page. Using the newly updated Local Search Tool Kit, I examined the results for the 7 Pack for hotels in Buffalo, New Orleans and New York. Over 40% of the hotel’s Places Pages contained testimonials from their own websites as reviews. However most of the results were not true testimonials but rather positive marketing language that Google had scraped and included. The one exception is the Embassy Suites Chicago Downtown (shown to me by Steven King of SimPartners). Interestingly the reviews shown for them on their Places Page are TripAdvisor reviews presented on their website via the TA review widget.

My conclusion: Google trusts large hotel websites and it appears that a large number of them now contribute their own content to the Places Review area. Google however is still having trouble parsing them for appropriate review content.

Hotel	Content Page URL Referenced by Places Reviews	Places Page
Buffalo Marriott Niagara	/bufam-residence-inn-buffalo-amherst/	Places Page
Adam’s Mark Hotels – Buffalo	/about/amenities.html	Places Page
Hampton Inn & Suites Buffalo Downtown	/index.jhtml?ctyhocn=BUFDTHX	Places Page
Windsor Court Hotel – New Orleans	/windsor-court-hotel.visit-new-orleans.com/	Places Page
Omni Royal Orleans	/NewOrleansRoyalOrleans.aspx	Places Page
InterContinental New York Barclay Hotel	/new_york/new_york/7775.html	Places Page
The New York Palace Hotel	/index.html%3FpropertyID%3D97502	Places Page
The Standard, New York Hotel	/new-york-city/-hotel-new-york-city/ /new_york_city/the_standard_new_york	Places Page
New York’s Hotel Pennsylvania	/popular-new-york-attractions.html /all-hotel-offers.html	Places Page
Embassy Suites Chicago Downtown	/taste_of_chicago/index.cfm	Places Page

It is much, much harder to find examples of SMB testimonial pages that Google has included in Places Pages. I would take that as a signal of the lower trust that Google has in this market compared to hotels as well as poorer optimization.

That being said, the content showing on the Places page was much more accurate with most of the examples actually including testimonials. The exception below is LA Fitness that, like many of the hotels above, is not actually showing review content. In the case of Crest Wood Painting while Google shows their testimonials as reviews, they are nothing more than his Google and Yahoo reviews repurposed onto the review page on his site.

Generally speaking, the ones below have a page name of testimonial or review. Whether Google needs that signal to more easily parse and include the page is unclear.

Business Name	Content Page URL Referenced by Places Reviews	Places Page
Cleveland Comfort Solutions	/reviews/	Places Page
LA Fitness	/gym/lincoln-south-west/	Places Page
Crestwood Painting	/reviews.html	Places Page
Person + Killian Photography	/testimonials/	Places Page

Some History

In late 2009, Google started using sentiment analysis to improve review summaries in Places. Shortly thereafter, in early in 2010, Google started analyzing scraped blog and general news content and where appropriate including this free form content into Places as reviews.

In late September of 2010 Google announced their support of Rich Snippets for Local. At the time the Rich Snippet for Local FAQ indicated that testimonials would in fact be included as review content on a businesses Places Page. At some point, Google obviously came to the realization that a broad movement in this direction would lead (or did lead) to a flood of review spam and revised their FAQ.

Regardless, Google does seem to be indicating that when there is high enough trust they would still include testimonials as reviews (“The FAQ below was intended to convey that we try to classify reviews wherever they’re found on the Web…”).

It appears, that it is both Google’s willingness to, on occasion, accept a businesses testimonials as trust worthy and their ability to parse sentiment that has led to own website testimonials showing in some Places Pages.

While Google is accepting and including semantically marked up reviews from large directories sites and a few lesser sites, it does not appear that Google has yet applied their understanding and use of semantically notated review content to SMB websites. When they will is unclear. Note that none of the above examples that I found are using Rich Snippet mark up.

The Takeaways

Do a testimonial page if it makes sense for the client website.

Will Google include your testimonial page in Places? At this point, we just don’t know how many or whose they will include. So don’t build the page solely for that reason. I have a client site (Barbara Oliver Jewelry) that has long had testimonials on it. She essentially includes every comment that comes to her offline. In the fall, we implemented hReview on the page and it has yet to be picked up by Google.

If you are a hotel then it behooves you to provide Google will legitimate reviews via a page on your site. Although having marketing happy talk show as reviews on your Places Page is not so bad, it would seem to me preferable to have real testimonials and reviews there. If I were a hotel I might even think about showing independently generated reviews on the site.

It appears from the SMB samples to make sense to name the page Reviews or Testimonials to give Google the as man clues possible in finding your page.

I assume that Google judges the page on its trust, content and its prominence. If you are going to build a testimonial page be sure to pass it some strength via your site architecture.

What are you seeing?

What has your experience been?

Have you seen other SMB testimonial pages showing? Are you seeing other types of industries beside hotels regularly showing a testimonial page?