Documents List API

 

There are a number of ways to add resources to your Google Documents List using the API. Most commonly, clients need to upload an existing resource, rather than create a new, empty one. Legacy clients may be doing this in an inefficient way. In this post, we’ll walk through why using resumable uploads makes your client more efficient.

The resumable upload process allows your client to send small segments of an upload over time, and confirm that each segment arrived intact. This has a number of advantages.

Resumable uploads have a customizable memory footprint on client systems

Since only one small segment of data is sent to the API at a time, clients can store less data in memory as they send data to the API. For example, consider a client uploading a PDF via a regular, non-resumable upload in a single request. The client might follow these steps:

  1. Open file pointer to PDF
  2. Pass file pointer and PDF to client library
  3. Client library starts request
  4. Client library reads 100,000 bytes and immediately sends 100,000 bytes
  5. Client library repeats until all bytes sent
  6. Client library returns response

But that 100,000 bytes isn’t a customizable value in most client libraries. In some environments, with limited memory, applications need to choose a custom chunk size that is either smaller or larger.

The resumable upload mechanism allows for a custom chunk size. That means that if your application only has 500KB of memory available, you can safely choose a chunk size of 256KB.

Resumable uploads are reliable even though a connection may not be

In the previous example, if any of the bytes fail to transmit, this non-resumable upload fails entirely. This often happens in mobile environments with unreliable connections. Uploading 99% of a file, failing, and restarting the entire upload creates a bad user experience. A better user experience is to resume and upload only the remaining 1%.

Resumable uploads support larger files

Traditional non-resumable uploads via HTTP have size limits depending on both the client and server systems. These limits are not applicable to resumable uploads with reasonable chunk sizes, as individual HTTP requests are sent for each chunk of a file. Since the Documents List API now supports file sizes up to 10GB, this is very important.

Resumable upload support is already in the client libraries for Google Data APIs

The Java, Python, Objective-C, and .NET Google Data API client libraries all include a mechanism by which you can initiate a resumable upload session. Examples of uploading a document with resumable upload using the client libraries is detailed in the documentation. Additionally, the new Documents List API Python client library now uses only the resumable upload mechanism. To use that version, make sure to follow these directions.

The Changes Feed in the Documents List API

We’re announcing the availability of the Changes feed in the Documents List API. This feed makes it easier to detect resources that have changed.

Currently, clients needing to sync resources between Google Docs and other systems or device often encounter a number of issues detecting changes to resources via the API. Clients typically query for all resources modified after a given date. This date is denoted by the app:edited field of a resource entry. However, this field is not updated in all cases the client may care about, for instance if a resource is shared. In addition, querying for all resources modified after a given date does not produce entries for resources that have been deleted. This leads to very complex implementations of change detection by clients. These complex implementations usually have race conditions, and require a large volume of data to be exchanged with the API.

The Changes feed simplifies this process by providing resource entries only for changed resources. If a resource occurs in the Changes feed at all, the occurrence indicates a change to the resource. Once all changes are consumed, clients can store an identifier of the last change consumed. This identifier is called a changestamp. Future queries to the Changes feed with a changestamp will only return changes occurring after the given changestamp.

To start using the Changes feed, make an authorized HTTP GET request to the following URI:

https://docs.google.com/feeds/default/private/changes

The response from the API includes a Google Data API feed of resources that have changed:


    xmlns:openSearch="http://a9.com/-/spec/opensearch/1.1/"
    xmlns:docs="http://schemas.google.com/docs/2007" 
    xmlns:gd="http://schemas.google.com/g/2005"
    gd:etag="W/"DEEMQ3w8eyt7ImA9WhZUGUo."">
  5635
        href="...?start-index=5636"/>
  
            term="http://schemas.google.com/docs/2007#change" 
        label="change"/>
    Project tasks
    ...
    
  
  ...

Fields shown in this example response are discussed in detail in the updated developer guide.

Google Apps at I/O 2011: A Lots of Excitement

It’s been only two short weeks since Google I/O 2011! There was fantastic energy at the event, and developers had their choice of over 100+ sessions on topics ranging from Google Apps to Android to App Engine to HTML5 to Cloud Robotics.


Here are the highlights from the Google Apps track:

Sessions

In the Google Apps track, we had 8 sessions on a variety of topics of interest to Google Apps Marketplace developers, systems integrators and customers alike. All of the sessions are available in HD on YouTube and we’ve also posted many of the session slides and notes.

Google Apps Marketplace:

  • Launch and Grow your Business App on the Google Apps Marketplace provided an intro to the Apps Marketplace, but most of the session was third-party developers telling the story of their businesses, demoing their integrations and providing guidance for other developers looking for success on the Marketplace. Teaser: 30% free->paid conversion rates from GQueues on the Google Apps Marketplace.
  • Apps Marketplace: Best Practices and Integrations covered a wealth of best practices for business app development and Google Apps integrations based on experience working with hundreds of developers building applications for the Google Apps Marketplace.

Google Apps Script:

  • Enterprise Workflow with Apps Script showed how Google Apps Script can be used to build complex workflows using simple server-side JavaScript code. The speakers built on several examples for document publishing approval, showing lots of code for how it’s done.
  • Developing Apps, Add-Ins and More stepped through building Add-Ins with deep integration into the Google Apps UI and full applications. The team announced the Apps Script GUI Builder to drag and drop UI components and full Apps Script APIs for Gmail and Google Docs.

Application APIs:

Solutions and Administration:

  • Developing Innovative Custom Business Solutions with Google Apps covered how web solution providers are driving us towards the goal of 100% web. Included many real-world examples from a variety of companies who are extending Google Apps using Apps Script, Google Sites, gadgets, Data APIs, App Engine, GWT and more.
  • Compliance and Security in the Cloud talked about the suite of APIs and tools available for Google Apps customers to handle policy compliance, audit, incident response and more. Very helpful session for IT administrators, CTOs and CIOs using Google Apps, with much of the session diving into several examples using real-world use cases.

Developer Sandbox

We had 24 fantastic companies in our Developer Sandbox this year, showcasing the applications they built for the Google Apps Marketplace and the services they provide Google Apps customers as system integrators or VARs. We were excited to see many of the companies talking about new integrations they have recently built with Google Apps.

Parties and Fun

The official After Hours event celebrated technical and artistic innovation and included robots, games and transforming vehicles in addition to a live performance from Jane’s Addiction. Many Google teams and companies attending I/O also threw plenty of great parties at nearby bars and restaurants.