David Janes' Code Weblog

March 1, 2010

Key Preserving YAJL

code fragments, iphone · David Janes · 8:35 am ·

Of all the JSON parsing libraries for the iPhone, the one I like the most is YAJL-obj, which is based on the “event driven/SAX style” YAJL. When I first started creating iPhone applications, I used a lot of Plists; I’ve now switched this all over to JSON not only because it’s more standards-y, but also because I can use the same data sets with my Python/Django and Android applications.

The JSON data I’m working with looks something like a dump of a SQL Table: lots of rows using the same column names — i.e. dictionary keys – over and over. If you’re loading a couple of thousand rows of data, each with 20 or or 30 columns/keys, the shear size of the key strings can add up — even though really there’s only a couple of dozen keys in total really in use. All these duplications waste a lot of memory – which is especially tight on a 8Gb iPod Touch – so I decided it might help out to actually reuse keys in my JSON loaders.

Here’s the changes I made to YAJL:

YAJLDocument.h

@interface YAJLDocument : NSObject  {
...
    NSMutableDictionary *keyStore_;
}
@property(nonatomic,retain) NSMutableDictionary *keyStore_;

YAJLDocument.m

- (id)initWithParserOptions:(YAJLParserOptions)parserOptions {
	if ((self = [super init])) {
...
		keyStore_ = [[NSMutableDictionary alloc] initWithCapacity:16];
	}
	return self;
}

- (void)parser:(YAJLParser *)parser didMapKey:(NSString *)key {
#if 1 // DPJ 2010-03-01
	key_ = [keyStore_ objectForKey:key];
	if (!key_) {
		[keyStore_ setObject:key forKey:key];
		key_ = key;
	}
#else
	key_ = key;
#endif
	[keyStack_ addObject:key_]; // Push
}

To rebuild your library:

  • select build target Device – 3.0 | Release | Combine Libs
  • the output file is Project-IPhone/build/libYAJLIPhone-*.zip
  • just add all the things in that ZIP file to your iPhone project and you’re good to go

Final thoughts:

  • whether this happens or not should be controlled by the YAJLParserOptions
  • I’m probably going keep modifying this code, adding a version with a “scrubbing delegate” that allows dictionaries to be scrubbed on the spot of data not needed in the application

2 comments »

  1. I wonder if having a category on NSString for this would be more useful?

    Then you could just do something like:
    key_ = [key internedString];
    to get the canonical copy of the key. And you could use it anywhere you thought you might have duplicate strings. :)

    Later,
    Blake.

  2. David Janes · 2010-03-01 14:41

    It’s an excellent idea, but since I’m compiling this in a library and have to make sure I reduce the external dependencies, I think.

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress