App Engine 1.5.3 SDK Released

 

New App Engine release. You might have noticed that the rate of releases has gone up slightly in the past few months. We’ve made some changes internally so we are looking to push out a new release every month. This month includes a few Datastore updates, some changes to Blobstore API and Memcache API, and finally a new feature for the Java developers.

Python and Java Changes

  • Blobstore API – We’ve removed the limits on the size of blob uploads. You can now upload files of any size, allowing your app to serve images, video, or anything your internet connection can handle.

Datastore Changes

  • Index retrieval – We’ve added the ability for you to programmatically retrieve the list of indexes you’ve currently defined in the datastore, as well as their statuses.
  • Datastore Admin – You can now enable the Datastore Admin function from the Admin Console. This will allow Java users to make use of this functionality, like deleting all entities of a certain kind, without having to upload a Python version of their application. And for Python developers, you no longer need to enable this in your app.yaml file.
  • HRD Migration Trusted Testers – We are seeking early adopters to try out an improved HRD migration tool that requires a read-only period relative to your datastore write rate (as opposed to your datastore size, which is how the current version behaves). Please see the release notes for more information.

Python Updates

  • Memcache API – We now support the CAS (compare-and-swap) operation in our Python Memcache API (Java already had it). This can be used to update a value in Memcache only if no other requests have updated it between when the value was retrieved and when you go to update it.

 

  • Download app – Using the AppCfg download_app command, you can download any files that were uploaded from your war directory when you last updated the app version.

This release also contains small updates and bugfixes for both Python and Java so be sure to check out the full release notes. Feedback, discussion, and questions can be posted in our Google Group.

Beauty On the Outside, High Replication on the Inside

Today we launched Google Art Project in collaboration with 17 of the world’s most renowned museums. Google Art Project is built on top of App Engine and lets you take virtual tours of famous museums using internal Street View technology, view high resolution images of famous art work, and create personal virtual artwork collections.

When Art Project started development several months ago, the team built the application using Java and the Master/Slave Datastore. However, as their launch date approached, we released the new High Replication Datastore configuration and, with a scheduled maintenance period so soon after the site’s launch, they decided to switch over to the High Replication Datastore.

Before switching, they ran a load test to set a performance baseline for comparison after the application’s data was migrated. Now that the application has launched, we wanted to share the results of the test with you as an example of what to expect after a switch to the High Replication Datastore. Below are the mean numbers for latency of different parts of the site.

Here’s a description of what each page does behind the scenes:

Homepage: This is the landing page that just serves a static webpage for site navigation. Since this page does not pull information from the datastore, the latency is stable.

Collections: Art Project lets users create individual museum collections. These load tests specifically targeted adding and deleting paintings from a user’s personal collection, as well as rendering those collections. We notice the slightly increased latency from saving and deleting entities in the datastore.

Level Maps: These pages simply performed get() calls on the datastore using entity keys. Latency on these pages is consistent across instances.

Info Spots: This handler performs the most data intensive calculations of all of the handlers. It calculates all line of sight interest points for a user’s map position in a museum gallery room, and saves the points of interest to the datastore for that location. The good news is, this calculation doesn’t have to happen for every user. Once this data has been calculated for a given spot, it can re-used for other visitors to the site.

As you can see, while there was some increased latency when switching to the High Replication datastore, the site latency is still very low. And the migration required no major code changes and no modification to the datastore structure between the two load tests.

For more information about the High Replication Datastore, see the Datastore documentation. The next scheduled maintenance period for the Master/Slave datastore is February 7th, 2011 from 5pm – 6pm PST. The High Replication datastore and Google Art Project will not need to be read-only during the by this period. On the High Replication Datastore, your application won’t need to be either.