Gingerbread NDK Awesomeness

[This post is by Chris Pruett, an outward-facing Androider who focuses on the world of games. —Tim Bray]

We released the first version of the Native Development Kit, a development toolchain for building shared libraries in C or C++ that can be used in conjunction with Android applications written in the Java programming language, way back in July of 2009. Since that initial release we’ve steadily improved support for native code; key features such as OpenGL ES support, debugging capabilities, multiple ABI support, and access to bitmaps in native code have arrived with each NDK revision. The result has been pretty awesome: we’ve seen huge growth in certain categories of performance-critical applications, particularly 3D games.

These types of applications are often impractical via Dalvik due to execution speed requirements or, more commonly, because they are based on engines already developed in C or C++. Early on we noted a strong relationship between the awesomeness of the NDK and the awesomeness of the applications that it made possible; at the limit of this function is obviously infinite awesomeness (see graph, right).

With the latest version of the NDK we intend to further increase the awesomeness of your applications, this time by a pretty big margin. With NDK r5, we’re introducing new APIs that will allow you to do more from native code. In fact, with these new tools, applications targeted at Gingerbread or later can be implemented entirely in C++; you can now build an entire Android application without writing a single line of Java.

Of course, access to the regular Android API still requires Dalvik, and the VM is still present in native applications, operating behind the scenes. Should you need to do more than the NDK interfaces provide, you can always invoke Dalvik methods via JNI. But if you prefer to work exclusively in C++, the NDK r5 will let you build a main loop like this:

void android_main(struct android_app* state) {
    // Make sure glue isn't stripped.
    app_dummy();

    // loop waiting for stuff to do.
    while (1) {
        // Read all pending events.
        int ident;
        int events;
        struct android_poll_source* source;

        // Read events and draw a frame of animation.
        if ((ident = ALooper_pollAll(0, NULL, &events,
                (void**)&source)) >= 0) {
            // Process this event.
            if (source != NULL) {
                source->process(state, source);
            }
        }
        // draw a frame of animation
        bringTheAwesome();
    }
}

(For a fully working example, see the native-activity sample in NDK/samples/native-activity and the NativeActivity documentation.)

In addition to fully native applications, the latest NDK lets you play sound from native code (via the OpenSL ES API, an open standard managed by Khronos, which also oversees OpenGL ES), handle common application events (life cycle, touch and key events, as well as sensors), control windows directly (including direct access to the window’s pixel buffer), manage EGL contexts, and read assets directly out of APK files. The latest NDK also comes with a prebuilt version of STLport, making it easier to bring STL-reliant applications to Android. Finally, r5 adds backwards-compatible support for RTTI, C++ exceptions, wchar_t, and includes improved debugging tools. Clearly, this release represents a large positive ∆awesome.

We worked hard to increase the utility of the NDK for this release because you guys, the developers who are actually out there making the awesome applications, told us you needed it. This release is specifically designed to help game developers continue to rock; with Gingerbread and the NDK r5, it should now be very easy to bring games written entirely in C and C++ to Android with minimal modification. We expect the APIs exposed by r5 to also benefit a wide range of media applications; access to a native sound buffer and the ability to write directly to window surfaces makes it much easier for applications implementing their own audio and video codecs to achieve maximum performance. In short, this release addresses many of the requests we’ve received over the last year since the first version of the NDK was announced.

Saving Data Safely

With the advent of Gingerbread, we’re going to be running a series of posts in this space about the aspects of Android 2.3 that developers should care about. One thing that developers should care about more than anything else is not losing data. The rules are changing slightly as Gingerbread arrives, so I thought that would be a good starting point. I didn’t write this; I pulled it together from the contents of an email thread involving Android engineers Brad Fitzpatrick, Dianne Hackborn, Brian Swetland, and Chris Tate.

The question is: how do you make really sure your data’s been written to persistent storage? The answer involves a low-level system call named fsync(). Old C programmers like me mostly learned this the hard way back in the Bad Old Days; in 2008 at OSCON I immensely enjoyed Eat My Data: How Everybody Gets File IO Wrong by Stewart Smith; I’ve included a picture I took of one of his slides.

The reason this should be of concern to Android developers is that with 2.3, an increasing proportion of devices, notably including the Nexus S, are going to be moving from YAFFS to the ext4 filesystem, which buffers much more aggressively; thus you need to be more assertive about making sure your data gets to permanent storage when you want it to.

If you just use SharedPreferences or SQLite, you can relax, because we’ve made sure they Do The Right Thing about buffering. But if you have your own on-disk format, keep in mind that your data doesn’t actually consistently reach the flash chip when you write() it or even when you close() it. There are several layers of buffering between you and the hardware! And because of ext4 buffering policy, any POSIX guarantees that you thought you had before (but actually didn’t), you especially don’t have now.

Some Android devices are already running non-YAFFS filesystems, but as we brought up the Nexus S, buffering issues have actually bitten us a couple of times in framework code. When the Gingerbread source code becomes available, you’ll find lots of examples of how file I/O should be done.

To start with, for raw data consider using one of the synchronous modes of java.io.RandomAccessFile, which take care of calling fsync() for you in the appropriate way. If you can’t, you’ll want Java code that looks something like this.

     public static boolean sync(FileOutputStream stream) {
         try {
             if (stream != null) {
                 stream.getFD().sync();
             }
             return true;
         } catch (IOException e) {
         }
         return false;

In some applications, you might even want to check the return status of the close() call.

Now of course, there are two sides to this story. When you call fsync() and force the data onto storage, that can be slow; worse, unpredictably slow. So be sure to call it when you need it, but be careful not to call it carelessly.

Background reading if you want to know more:

Introducing Nexus S with Gingerbread

The very first Android phone hit the market in November 2008. Just over two years later, Android’s vision of openness has spurred the development of more than 100 different Android devices. Today, more than 200,000 Android devices are activated daily worldwide. The volume and variety of Android devices continues to surpass our wildest expectations—but we’re not slowing down.

Today, we’re pleased to introduce the latest version of the Android platform, Gingerbread, and unveil the next Android device from the Nexus line of mobile products—Nexus S. And for developers, the Gingerbread SDK/NDK is now available as well.

Nexus S is the lead device for the Gingerbread/Android 2.3 release; it’s the first Android device to ship with the new version of the Android platform. We co-developed this product with Samsung—ensuring tight integration of hardware and software to highlight the latest advancements of the Android platform. As part of the Nexus brand, Nexus S delivers what we call a “pure Google” experience: unlocked, unfiltered access to the best Google mobile services and the latest and greatest Android releases and updates.

Take a look at our backstory video for more on the vision behind this product and to understand why we think “a thousand heads are better than one”:

Nexus S is the first smartphone to feature a 4” Contour Display designed to fit comfortably in the palm of your hand and along the side of your face. It also features a 1GHz Hummingbird processor, front and rear facing cameras, 16GB of internal memory, and NFC (near field communication) hardware that lets you read information from NFC tags. NFC is a fast, versatile short-range wireless technology that can be embedded in all kinds of everyday objects like movie posters, stickers and t-shirts.

Gingerbread is the fastest version of Android yet, and it delivers a number of improvements, such as user interface refinements, NFC support, a new keyboard and text selection tool, Internet (VoIP/SIP) calling, improved copy/paste functionality and gyroscope sensor support.

Here’s a glimpse of the “magic” of Google on Nexus S:

You can find more Nexus S videos and information at google.com/nexus or follow @GoogleNexus on Twitter for the latest updates. After December 16, Nexus S can be purchased (unlocked or with a T-Mobile service plan) online and in-store from all Best Buy and Best Buy Mobile stores in the U.S. and after December 20 at Carphone Warehouse and Best Buy retailers in the U.K.

We’ll be open-sourcing Gingerbread in the coming weeks and look forward to new contributions from the Android ecosystem in the months ahead.