Google API Talks – Android, KML,Google Maps,Gadgets/Mapplets

At the beginning of the April Silicon Valley GTUG meeting, several Googlers and external developers gave lightning talks about their latest and greatest. Watch Dick Wall talk about Android updates, Van Riper talk about his KML, Tom Brown display transit data visualization in Google Earth, Brian Hamlin show off animated weather KML, the Seero guys demonstrate their live video mapping site, Mano Marks introduce libKML, and Pamela Fox talk about Google Gadgets and Mapplets.

Memory Analysis for Android Applications

The Dalvik runtime may be garbage-collected, but that doesn’t mean you can ignore memory management. You should be especially mindful of memory usage on mobile devices, where memory is more constrained. In this article, we’re going to take a look at some of the memory profiling tools in the Android SDK that can help you trim your application’s memory usage.

Some memory usage problems are obvious. For example, if your app leaks memory every time the user touches the screen, it will probably trigger an OutOfMemoryError eventually and crash your app. Other problems are more subtle, and may just degrade the performance of both your app (as garbage collections are more frequent and take longer) and the entire system.

Tools of the trade

The Android SDK provides two main ways of profiling the memory usage of an app: the Allocation Tracker tab in DDMS, and heap dumps. The Allocation Tracker is useful when you want to get a sense of what kinds of allocation are happening over a given time period, but it doesn’t give you any information about the overall state of your application’s heap. For more information about the Allocation Tracker, see the article on Tracking Memory Allocations. The rest of this article will focus on heap dumps, which are a more powerful memory analysis tool.

A heap dump is a snapshot of an application’s heap, which is stored in a binary format called HPROF. Dalvik uses a format that is similar, but not identical, to the HPROF tool in Java. There are a few ways to generate a heap dump of a running Android app. One way is to use the Dump HPROF file button in DDMS. If you need to be more precise about when the dump is created, you can also create a heap dump programmatically by using theandroid.os.Debug.dumpHprofData() function.

To analyze a heap dump, you can use a standard tool like jhat or the Eclipse Memory Analyzer (MAT). However, first you’ll need to convert the .hprof file from the Dalvik format to the J2SE HPROF format. You can do this using the hprof-conv tool provided in the Android SDK. For example:
[php]hprof-conv dump.hprof converted-dump.hprof[/php]

Example: Debugging a memory leak

In the Dalvik runtime, the programmer doesn’t explicitly allocate and free memory, so you can’t really leak memory like you can in languages like C and C++. A “memory leak” in your code is when you keep a reference to an object that is no longer needed. Sometimes a single reference can prevent a large set of objects from being garbage collected.

Let’s walk through an example using the Honeycomb Gallery sample app from the Android SDK. It’s a simple photo gallery application that demonstrates how to use some of the new Honeycomb APIs. (To build and download the sample code, see the instructions.) We’re going to deliberately add a memory leak to this app in order to demonstrate how it could be debugged.

Imagine that we want to modify this app to pull images from the network. In order to make it more responsive, we might decide to implement a cache which holds recently-viewed images. We can do that by making a few small changes to ContentFragment.java. At the top of the class, let’s add a new static variable:
[php]private static HashMap<String,Bitmap> sBitmapCache = new HashMap<String,Bitmap>();[/php]
This is where we’ll cache the Bitmaps that we load. Now we can change the updateContentAndRecycleBitmap() method to check the cache before loading, and to add Bitmaps to the cache after they’re loaded.
[php]void updateContentAndRecycleBitmap(int category, int position) {
    if (mCurrentActionMode != null) {
        mCurrentActionMode.finish();
    }
 
    // Get the bitmap that needs to be drawn and update the ImageView.
 
    // Check if the Bitmap is already in the cache
    String bitmapId = "" + category + "." + position;
    mBitmap = sBitmapCache.get(bitmapId);
 
    if (mBitmap == null) {
        // It’s not in the cache, so load the Bitmap and add it to the cache.
        // DANGER! We add items to this cache without ever removing any.
        mBitmap = Directory.getCategory(category).getEntry(position)
                .getBitmap(getResources());
        sBitmapCache.put(bitmapId, mBitmap);
    }
    ((ImageView) getView().findViewById(R.id.image)).setImageBitmap(mBitmap);
}[/php]
I’ve deliberately introduced a memory leak here: we add Bitmaps to the cache without ever removing them. In a real app, we’d probably want to limit the size of the cache in some way.

Examining heap usage in DDMS

The Dalvik Debug Monitor Server (DDMS) is one of the primary Android debugging tools. DDMS is part of the ADT Eclipse plug-in, and a standalone version can also be found in the tools/ directory of the Android SDK. For more information on DDMS, see Using DDMS.

Let’s use DDMS to examine the heap usage of this app. You can start up DDMS in one of two ways:

  • from Eclipse: click Window > Open Perspective > Other… > DDMS
  • or from the command line: run ddms (or ./ddms on Mac/Linux) in the tools/ directory

Select the process com.example.android.hcgallery in the left pane, and then click the Show heap updates button in the toolbar. Then, switch to theVM Heap tab in DDMS. It shows some basic stats about our heap memory usage, updated after every GC. To see the first update, click the Cause GCbutton.

We can see that our live set (the Allocated column) is a little over 8MB. Now flip through the photos, and watch that number go up. Since there are only 13 photos in this app, the amount of memory we leak is bounded. In some ways, this is the worst kind of leak to have, because we never get anOutOfMemoryError indicating that we are leaking.

Creating a heap dump

Let’s use a heap dump to track down the problem. Click the Dump HPROF file button in the DDMS toolbar, choose where you want to save the file, and then run hprof-conv on it. In this example, I’ll be using the standalone version of MAT (version 1.0.1), available from the MAT download site.

If you’re running ADT (which includes a plug-in version of DDMS) and have MAT installed in Eclipse as well, clicking the “dump HPROF” button will automatically do the conversion (using hprof-conv) and open the converted hprof file into Eclipse (which will be opened by MAT).

Analyzing heap dumps using MAT

Start up MAT and load the converted HPROF file we just created. MAT is a powerful tool, and it’s beyond the scope of this article to explain all it’s features, so I’m just going to show you one way you can use it to detect a leak: the Histogram view. The Histogram view shows a list of classes sortable by the number of instances, the shallow heap (total amount of memory used by all instances), or the retained heap (total amount of memory kept alive by all instances, including other objects that they have references to).

If we sort by shallow heap, we can see that instances of byte[] are at the top. As of Android 3.0 (Honeycomb), the pixel data for Bitmap objects is stored in byte arrays (previously it was not stored in the Dalvik heap), and based on the size of these objects, it’s a safe bet that they are the backing memory for our leaked bitmaps.

Right-click on the byte[] class and select List Objects > with incoming references. This produces a list of all byte arrays in the heap, which we can sort based on Shallow Heap usage.

Pick one of the big objects, and drill down on it. This will show you the path from the root set to the object — the chain of references that keeps this object alive. Lo and behold, there’s our bitmap cache!

MAT can’t tell us for sure that this is a leak, because it doesn’t know whether these objects are needed or not — only the programmer can do that. In this case, the cache is using a large amount of memory relative to the rest of the application, so we might consider limiting the size of the cache.

Comparing heap dumps with MAT

When debugging memory leaks, sometimes it’s useful to compare the heap state at two different points in time. To do this, you’ll need to create two separate HPROF files (don’t forget to convert them using hprof-conv).

Here’s how you can compare two heap dumps in MAT (it’s a little complicated):

  1. Open the first HPROF file (using File > Open Heap Dump).
  2. Open the Histogram view.
  3. In the Navigation History view (use Window > Navigation History if it’s not visible), right click on histogram and select Add to Compare Basket.
  4. Open the second HPROF file and repeat steps 2 and 3.
  5. Switch to the Compare Basket view, and click Compare the Results (the red “!” icon in the top right corner of the view).

Open Street Maps –vs- Ordnance Survey

In this post I’m going to look at how OSM data compares in terms of quality, and what better benchmark to test it against than the Ordnance Survey (OS) – the historical and much-revered national executive mapping agency of Great Britain.

A UK company called ITO has produced a very useful OSM Analysis tool, that displays details of roads that are listed in the Ordnance Survey’s OS Locator dataset that are not present in the OSM dataset. The tool is provided as a service for OSM mappers, with the assumption being that the OS dataset is both complete and accurate, and therefore roads listed on the site are “missing” from the OSM dataset and need to be found and added.

Looking at the analysis for Norwich, there are a total of 38 roads (out of 1,619) of the OS Locator dataset for Norwich that are not present in the OSM dataset. This works out at a 97.65% match between OSM and OS, and ranks Norwich as 41st in terms of OSM completeness out of the listed UK areas. Being both a competitive person and proud of Norwich as my (adopted) home, I thought I’d set about tracking down those missing 38 roads and making the OSM map of Norwich 100% complete to the Ordnance Survey dataset.

But hang on, I thought that this post was about quality, not completeness? And there is the interesting thing…. it turns out that the “missing” roads are not due to incomplete data, but rather discrepancies in data quality. To explain, the list of roads currently shown on the OS Locator dataset but not on OSM is as follows:

  • ALKMAAR WAY
  • ATTHILL ROAD
  • BIRBECK ROAD
  • BLACK HORSE OPENING
  • BRIGHTY’S OPENING – minor error
  • BRUNEL WAY
  • BUMSTEDE COURT
  • CHARGE ROAD
  • CHURSTON CLOSE
  • CLOVELLY DRIVE
  • CLOVELLY DRIVE
  • CLYFFE COTTAGES
  • DRAYTON ROAD (A1067)
  • EBENEZER PLACE
  • EDEN CLOSE
  • EDWARD JORDELL PLAIN
  • GEORGE DANCE PLACE
  • GILDERS WAY
  • GUNTON LANE
  • HOOPER LANE
  • KETT’S HILL (B1140) – minor error
  • MIDDLETON’S LANE – minor error
  • NEWBRGIN ROAD
  • PLUMSTEAD ROAD
  • PRINCE ANDREW’S ROAD – minor error
  • QUEENS ROAD (A147)
  • RANDLE GREEN
  • ST FAITH’S ROAD – minor error
  • ST FAITH’S ROAD – minor error
  • ST FAITH’S ROAD – minor error
  • ST JAMES COURT
  • ST JOHN STREET
  • THE MONASTERY
  • THREE KING LANE
  • TRILITHORN CLOSE
  • UPPER ST GILES STREET
  • WALL LANE
  • WARRINGTON WAY

The first thing to note is that 7 of the errors are listed by the OSM Analysis tool itself as “minor” errors. In every case, the discrepancy is in the use of apostrophes in the road name. I’m not about to get into an argument on the use of English language as to which one is correct – but as far as I’m concerned if the only difference between the two datasets is in the use of an apostrophe, I’m happy to mark that down as not being relevant in terms of spatial data completeness/accuracy.

I was surprised to see some of the roads listed as apparently missing: Queen’s Road, for example, is a major part of the Norwich inner ring road – how could that be missing from OSM? The answer can be found by examining the map that shows where the “absent” Ordnance Survey record is supposed to lie:

image

Queen’s Road becomes Bracondale at a fairly non-descript location – it’s not a major intersection, it’s more of just a bend in the road. The OS Locator dataset contains one section of road that’s labelled “Queen’s Road”, which, in the OSM dataset, has been labelled “Bracondale” – essentially it’s just a discrepancy between exactly where the road changes name. Again, I’m happy to ignore that one.

Another road on the list that lies not far from me is “Birkbeck Road”. Or, at least, that’s what I thought it was called, and that’s what it’s called in OSM. But the OS dataset seems to think it’s called “Birbeck Road”. Could it be that I’ve been wrong? I went to check Google Maps:

image

Google Maps reckons it’s called “Birbeck Road”, so maybe the Ordnance Survey is right, and OSM is wrong. To double check, I went to Bing Maps:

image

Huh? So that’s two votes for “Birbeck”, and two for “Birkbeck”. I better carry on looking…

www.yell.com goes with “Birkbeck”, so that’s 3-2 to the OSM spelling.

image

But www.viamichelin.co.uk agrees with the Ordnance Survey. 3 all.

image

www.streetmap.co.uk agrees with OSM….

image

….until you zoom in, at which point it changes its mind and agrees with the Ordnance Survey!

image

And this is where Open Street Map really excels – it’s collaboratively created by people on the ground with local knowledge, and with the aid of my bicycle and a camera, I could demonstrate the ethos of Open Street Map and settle this dispute once and for all. So, I now proudly present….

image

Birkbeck Road. So it turns out that Open Street Maps is correct, and the Ordnance Survey is wrong (as are Google Maps, Via Michelin, and StreetMap.co.uk). Enthused by this, and still on my bike, I decided to check some of the other apparently missing roads. The following photograph explains the apparent missing EDWARD JORDELL PLAIN, which once again, OSM correctly has listed as Edward Jodrell plain.

image

Interestingly, once again Google Maps agrees with the Ordnance Survey, and gets it wrong.

image

Whereas Bing Maps agrees with Open Street Maps and gets the correct spelling:

image

Over the next week or so I’m hoping to work my way through the rest of the list, but I continue to be amazed by the data available through Open Street Maps. Not only does it contain more data than Bing or Google (in that it contains points of interest such as postboxes, bus stops, property boundaries, and any number of other amenities), but in many cases it has higher quality data than the Ordnance Survey, at least in terms of the very limited sample I’ve looked at here. In fact, whereas I originally set out to update Open Street Maps to match the data from the Ordnance Survey, I’m beginning to think that the folks at the Ordnance Survey ought to be looking at the OSM data very closely and checking their data against it. And did I mention that OSM is free…?