BigQuery, meet Google Spreadsheets

Since announcing BigQuery at Google IO last May, we’ve been very excited by the response and feedback we’ve received from the developer community, enterprises and academia. The one consistent request we heard from everyone is the ability to interactively analyze large volumes of data without having to worry about provisioning, maintaining and scaling infrastructure.

Today, we would like to announce the integration of BigQuery with Google Apps Script and Google Spreadsheets, a feature we first demoed at Google IO. With this integration users now have the power to query multi-billion row tables, visualize the results and share them with others. Below you can see a simple script that queries a sample dataset and plots the results. A simple tutorial is available here with more to come soon.

We’ve seen a big uptake of the APIs (released in October) which let you create, populate and delete tables in BigQuery. Users have been loading more and more data in BigQuery. For instance the current M-Lab dataset in BigQuery stands at 240B rows!

The details of BigQuery and new features are available on the BigQuery website. We are gradually adding more developers during this free preview period. Please sign up for an invitation, and let us know about the creative and valuable ways you’re using BigQuery.

Augmented Reality for SketchUp

The first time I saw Augmented Reality in action, I wondered if I’d accidentally fallen through a wormhole on the way to work; it’s the kind of thing you’d expect to see on Boba Fett’s BlackBerry. AR is downright futuristic.

In the 3D modeling sense, AR involves combining a live video stream with a 3D model to create the illusion that the model is a physical object in the real world. All you need is a webcam (the ones that are built in to many laptops work just fine), a 3D model (SketchUp takes care of that), a printed-out paper “target” and a piece of AR software that can put everything together. This video shows AR in action:

And here’s an illustration that shows the setup:

A simple Augmented Reality setup for SketchUp, using an external monitor to show the video output.

Thanks to an Italian outfit called Inglobe Technologies, SketchUp users have been able to ride the AR wave for a while now. They’ve just released version 2 of their AR-media Plugin for Google SketchUp. Three great things about this shiny, happy piece of tech:

  • It’s available for both Windows and Mac OSX.
  • It’s available in three flavors: Free (Personal Learning Edition), Professional Lite and Professional.
  • I was able to use it, which means that it can’t be that hard to figure out.

Start out by grabbing the Quick Start Guide; you’ll find the relevant links about halfway down the plugin’s webpage. Follow Steps 2 and 3 to download and install the software; the free Personal Learning Edition will let you see how everything works without spending any money. After that, achieve instant gratification (my favorite kind) by skipping ahead to Step 6 in the Quick Start Guide: “Creating your first Augmented Reality Scene”.

The High Replication Datastore for App Engine

When App Engine launched over two years ago, we offered a Datastore that was designed for quick, strongly consistent reads. It was based on a Master/Slave replication topology, designed for fast writes while still allowing applications to see data immediately after it was written. For the past six months, as you are probably aware, we’ve been struggling with some reliability issues with the App Engine Datastore. Over the course of the past few months, we’ve made major strides in fixing these issues. However, our experience with these issues has made us rethink some of our design assumptions. As we promised you in some of our outage reports earlier this year, we wanted to give you a more fundamental solution to the problem.

Today I’m proud to announce the availability of a new Datastore configuration option, the High Replication Datastore. The High Replication Datastore provides the highest level of availability for your reads and writes, at the cost of increased latency for writes and changes in consistency guarantees in the API. The High Replication Datastore increases the number of data centers that maintain replicas of your data by using the Paxos algorithm to synchronize that data across datacenters in real time. One of the most significant benefits is that all functionality of your application will remain fully available during planned maintenance periods, as well as during most unplanned infrastructure issues. A more detailed comparison between these two options is available in our documentation.

From now on, when creating a new application, you will be able to select the Datastore configuration for your application. While the current Datastore configuration default remains Master/Slave, this may change in the future.

Datastore configuration options when creating an app.

The datastore configuration option can not be changed once an application is created, and all existing applications today are using the Master/Slave configuration. To help existing apps migrate their data to an app using the High Replication Datastore, we are providing some migration tools to assist you. First, we have introduced an option in the Admin Console that allows an application to serve in read-only mode so that the data may be reliably copied between apps. Secondly, we are providing a migration tool with the Python SDK that allows you to copy from one app to another. Directions on how to use this tool for Python and Java apps is documented here.

Now, a word on pricing: Because the amount of data replication significantly increases with the High Replication datastore, the price of this datastore configuration is different. But because we believe that this new configuration offers a significantly improved experience for some applications, we wanted to make it available to you as soon as possible, even though we haven’t finalized the pricing details. Thus, we are releasing the High Replication Datastore with introductory pricing of 3x that of the Master/Slave Datastore until the end of July 2011. After July, we expect that pricing of this feature will change. We’ll let you know more about the pricing details as soon as they are available, and remember, you are always protected when pricing changes occur by our Terms of Service. Due to the higher cost, we thus recommend the High Replication Datastore primarily for those developers building critical applications on App Engine who want the highest possible level of availability for their application.

Thank you, everyone, for all the work you’ve put into building applications on App Engine for the past two years. We’re excited to have High Replication Datastore as the first of many exciting launches in the new year, and hope you’re excited about the other things we’ve got in store for App Engine in 2011.