David Janes' Code Weblog

April 9, 2010

Useful Android Command Line Commands

android · David Janes · 9:13 am ·

This post is basically an FYI for me.

Running an Android emulator

$ emulator -avd david_6 &

or if you’re getting Out of Disk Space errors

$ emulator -avd david_6 -wipe-data &

Seeing the console output

$ adb logcat

Seeing what AVDs are available

$ android &

Djolt-obj: Django-like templates for Objective-C

code fragments,djolt,iphone · David Janes · 7:51 am ·

My company, Discover Anywhere Mobile, has open sourced a software component that many iPhone developers may be interested in: Djolt (Django-like Templates) for Objective C. The objective of this project is to be able to take NSObjects – particularly NSDictionary, NSArray, NSString and NSNumber – and be able to feed them to a text template, Django style, and get text output. This will be fantastic for working with APIs that deliver data via JSON or XML.

It’s dependent onĀ RegexKit library for now, though if the iPhone OS 4 NSRegularExpression is powerful enough we may transition to that.

The code is being tested against the Django template test cases, with the goal of having as complete as possible (given different language idioms) compatibility. We only cover a small subset of Django yet – the stuff we need – but if you’re interested in working on the project, please by all means join in.

Code

NSDictionary* d = [[NSData dataWithContentsOfFile:@"test.json"] yajl_JSON];

DjoltTemplate* t = [[DjoltTemplate alloc] initWithTemplateFile:@"test.djolt"];
NSString* r = [t render:d];

test.json

{
 "title" : "Test 1",
 "items" : [
  {
   "href" : "http://www.example.com/1",
   "title" : "Example 1"
  },
  {
   "href" : "http://www.example.com/2",
   "title" : "Example 2 & Example \"2.5\""
  },
  {
   "href" : "http://www.example.com/3",
   "title" : "Example 3"
  }
 ]
}

test.djolt

<html>
<head>{{ title }}</head>
<body>
{% if fred or items or bleck %}
<ul>
{% for itemd in items %}
<li><a href='{{ itemd.href }}'>{{ itemd.title|safe }}</a> ({{ forloop.counter }})</li>
{% endfor %}
</ul>
{% endif %}
<p>
{{ nothing|default:"something" }}
</p>
</body>
</html>

Output

<html>
<head>Test 1</head>
<body>

<ul>

<li><a href='http://www.example.com/1'>Example 1</a> (1)</li>

<li><a href='http://www.example.com/2'>Example 2 & Example "2.5"</a> (2)</li>

<li><a href='http://www.example.com/3'>Example 3</a> (3)</li>

</ul>

<p>
something
</p>
</body>
</html>

April 3, 2010

Optimizing JSON database loading on an iPod Touch

iphone · David Janes · 3:13 pm ·

This is the story of how we sped up loading of our application’s database on a 1 generation iPod Touch from 19.28 seconds to 8.75 seconds. We test all our applications of the oldest hardware in use out there to get a sense of what our “floor” performance will be – generally if it’s bearable there, it’ll be better to awesome on a newer iPhone.

The Battle Plan

Our current database schema/code is called the “B” database. It loads from a JSON file (actually several, but only one counts) which is a list of dictionary. Each dictionary represents a location, listing, event, coupon or whatever in our application.

To speed up loading times, we decided to create a “C” database with the following characteristics:

  • still load from a JSON file
  • empty values would be removed from the database
  • default values – e.g. “country_name” : “Canada”, where the country is always Canada – would be removed
  • we would “lazy” load most of the data
    • the database would only contain the ID for each item, the lat/lon and a few other essential items
    • a separate JSON file for each location would be made also, containing all the data
    • because we well encapsulate access to our data objects (no peeking inside of the dictionary that holds all the data!), when the lazy data was needed we’d load it on the spot

We ended up only doing the the first two of these options. We’re likely to do the lazy loading in the future, but because this would require implementing a proper search (because search looks at every record) we decided to defer to another release. Our testing shows that this will shave about another 25% of load time, which at this point wasn’t worth the trouble but may be later when we start getting into 2500+ items to load.

Initially we planned also to do the following, but after learning that most of the work was in object creation and not in data loading, we decided not to pursue this

  • well known keys would be replaced by a single letter (i.e. “title” with “T”)
  • hash keys, represented as a 32 byte hex string, would be replaced by 20 character base64 strings

Because it turns out that object allocation and messing around with dictionaries is relatively expensive, and loading strings into memory not so much, we probably will never do these.

Performance Numbers

All times in seconds, measured from the start of loading

B database starting point
  • loading data from disk: 0.41
  • finishing converting JSON data to NSObjects: 11.34
  • finished wrapping NSObjects into Location objects: 19.28

We learned here that we have two obvious problems: converting JSON and creating the Location objects.

Use C database

The C database JSON file is about 20% of the size of the B database.

  • loading data from disk: 0.08
  • finishing converting JSON data to NSObjects: 5.56
  • finished wrapping NSObjects into Location objects: 13.30

Already we’ve seen a big improvement of almost 6 seconds.

Turn off slideshow

Here we just turned off the slideshow to see what would happen

  • loading data from disk:0.05
  • finishing converting JSON data to NSObjects: 4.42
  • finished wrapping NSObjects into Location objects: 11.57

Almost two seconds saved! Our new code now doesn’t start the animation (or in fact, any other non-critical background task) until the DB is loaded. For the most we can say that 4.42s is the floor for loading a database from disk using JSON – it really isn’t going to get smaller than this.

Slightly optimized B database, better Location constructor

There’s a lot of steps I’ve left out here – I just turned off the Location “init” function altogether, found that things worked at almost the speed shown below. Looking at the code, I discovered a totally unnecessary dictionary copy, a couple of computations that could be done lazily, and some unnecessary initializations. After that was all done, we’ve got the the Location creating step down to about 1.1 seconds – originally it was almost 8!

  • loading data from disk: 0.40
  • finishing converting JSON data to NSObjects: 7.6
  • finished wrapping NSObjects into Location objects: 8.76

Note that we we lost 3 seconds (4.42 to 7.6) going back to the B database. However, this saved us another long programming and testing stint, so we’ll let this ride for now. However, we know that we can get this down to about 25% of the original speed when we need to.

Conclusions

  • Look for things inside of loops – the costs add up quickly
  • Object creation isn’t free, and object copying is expensive
  • Don’t make assumptions whether you’re CPU bound or IO bound – test and find out
  • Where you’re CPU bound – don’t do other stuff (e.g. animation cuteness) that’s competing for the CPU!
  • Where you’re IO bound – make things smaller (duh)
  • we’re likely to do some experimenting with SQLdb in the future, but given that loading isn’t really the problem, it probably won’t make much difference
  • CoreData didn’t exist when we started this application, so we’re not even looking at that
  • On an iPhone 3GS the complete database loads in about 3.5 seconds, which is sweet enough

Powered by WordPress

Switch to our mobile site