Friday, August 29, 2008


Our tech reviewer just finished the last chapter, so now the chapters are off to the formatter and then the copy editor. It's amazing how involved the commercial publishing process is. Having only written for magazines and online publishers before, this is a whole new ball o' wax for me. That's not a bad thing, but it does give me a new appreciation for just how many people contribute to those books you see on the shelves of the book store. There may be one or two names on the cover, but likely at least eight people (and possibly many more) were actively involved in shaping the content of the book, not to mention the layout, production, printing and all the other tasks, which likely drives the number up into the dozens.

If anyone's interested in becoming a committer on the SQLite Persistent Objects project, drop me a line at jeff underscore lamarche at mac dot com.

Monday, August 25, 2008

SQLitePersistentObjects now in Google Code Repository

Check it out - you can always get the latest and greatest from here:

There aren't any substantive changes since the initial release - I'm starting to get chapters back now fast and furious and have to get as much done as I can before I leave on vacation. Anyone who is interested in contributing to the project, just drop me a line.

Tuesday, August 19, 2008

SQLite Persistent Objects

Wouldn't it be nice if your Objective-C data objects just knew how to save and load themselves? It'd be nice if you could just call "save" and trust that your object would save itself properly somewhere, and that when you wanted to load it back in, you could just call a class method to retrieve the object or objects you wanted?

Well, now you can. I've been talking about this project for a while, but it got put on hold for the book. I have now finished the first draft of the code. This is very much a beta release, but I am interested in feedback from people, so am releasing it. In the next release, I will provide more sample code and unit tests, and other niceties, but for this release, you just get the source code files and a little bit of information about how to use them.

What does it do?

It lets you create Objective-C classes that know how to persist themselves into a SQLite database. Not only that, but it completely hides the implementation details from you - you do not need to create the database, create the tables, or do anything else except work with your ojbects. you simply subclass SQLitePersistentObject and create Objective-C 2.0 properties for every data element you want persisted. When you create an instance of this object and send it the save message, it will get saved into the database.

How does it work

Every subclass of SQLitePersistentObject gets its own table in the database. Every property that's not a collection class (NSDictionary, NSArray, NSSet or mutable variants) will get persisted into a column in the database. Properties that are pointers to other objects that are also subclasses of SQLitePersistentObject will get stored as a reference to the right row in that object's corresponding table. Collection classes gets stored as child tables, and are capable of storing either a foreign key reference (when the object they hold is a subclass of SQLitePersistentObject) or in a field on the child table.

Can all properties be stored?

No. But most can. This currently does not support properties that are c-strings, void pointers, structs, or unions. All scalars (ints, floats, etc) get saved into appropriate fields. When it comes to Cocoa objects, any class that conforms to NSCoding can be stored in a column. It is also possible to provide support for specific classes by adding a category on the class you wish to support that tells the system how to store and retrieve that object from a column's data. There are provided categories for NSDate, NSString, NSData, NSMutableData, NSNumber, and (of course) NSObject. Creating new ones to let other objects be persisted is relatively easy - just look at one of the included categories and implement the same methods. The methods are documented in NSObject-SQLitePersistence.h.

Classes that don't have direct support (the ones listed above or any that you add), will use NSObject's persistence mechanism, which archives the object into a BLOB using an NSKeyedArchiver. This is inefficient for some objects because you can't search or compare on these fields, but at least most object can be persisted. Some classes like NSImage, this method actually works quite well and there's probably no reason to add a specific category.

How do I use it?

Add all the files from the zip file to your project, link in libsqlite3.dylib. Then, declare data objects like this:

#import <foundation/foundation.h>
#import "SQLitePersistentObject.h"

@interface PersistablePerson : SQLitePersistentObject {
NSString *lastName;
NSString *firstName;
@property (nonatomic, retain) NSString * lastName;
@property (nonatomic, retain) NSString * firstName;

Once you've done that, you can just create your objects as usual:

PersistablePerson *person = [[PersistablePerson alloc] init];
person.firstName = @"Joe";
person.lastName = @"Smith";

Now, you can save it to the database (which will be created if necessary) like so:

[person save];

Loading it back in is almost as easy. Any persistable object gets dynamic class methods added to it to allow you to search. So, we could retrieve all the PersistablePerson objects that had a last name of "Smith" like so:

NSArray *people = [PersistablePerson findByLastName:@"Smith"]

Or, you could specify the exact criteria like this:

PeristablePerson *joeSmith = [PersistablePerson findFirstByCriteria:@"WHERE last_name = 'Smith' AND first_name = 'Joe'];

Notice that the camel case word divisions (marked by capital letters) got changed into an underscore, so if you want to use findByCriteria: or findFirstByCriteria: you have to make sure you get the column names correct. I plan on adding additional dynamic class methods to allow searching based on multiple criteria, but for now, if you want to search on more than one field you have to manually specify the criteria. Don't worry, it's not hard.

Can I create Indexes?

Yep. Just need to override a class method

+(NSArray *)indices;

You should return an NSArray of NSArrays. Each array contained in the returned array represents one index, and should contain the name of the properties to build the index on. Use the property names - although, in some cases, the names are changed, this method should return the actual property names, not the database column names. Here is an example implementation of indices

+(NSArray *)indices
NSArray *index1 = [NSArray arrayWithObject:@"lastName"];
NSArray *index2 = [NSArray arrayWithObjects:@"lastName", @"firstName", nil];
NSArray *index3 = [NSArray arrayWithObjects:@"age", @"lastName", @"firstName", nil];
return [NSArray arrayWithObjects:index1, index2, index3, nil];

Assuming the strings passed back all describe valid properties, the table that holds this class' data will have three indices. Easy enough, right?

What happens if I load the same object in multiple times?

These classes maintain a map of all persistable objects in memory. If you load one that's already in memory, it returns a reference to that instead of going back to the database. If you need to be able to make a copy of the object, you'll have to implement NSCopying, as SQLitePersistableObject does not currently conform to NSCopying, though it probably will before I consider this "done".

How is this Licensed

It's a very liberal license. You can use it however you want, without attribution, in any software commercial or otherwise. You are not required to contribute back your changes (although they are certainly welcome), nor are you required to distribute your changed version. The only restriction that I place on this is that if you distribute the source code, modified or unmodified, that you leave my original comments, copyright notice, and contact information, and ask that you comment your changes.

Is it fully functional?

Close to it. As I said earlier, I want to add more robust methods for finding objects. I also haven't implemented rollback. Because this maintains a memory map, it's actually difficult right now to go back to the way an object was when it was last saved because you have to make sure that every object that has a reference to that object releases it, make sure that it is gone, then re-load it from the database. Rollback is high on my priority list. Because this is an early release, there are bound to be bugs and I'm sure you can come up with enhancements that could make it better.

Does it use private or undocumented libraries?

No. It does not. It does make extensive use of the Objective-C runtime, but that is fully documented and open to developers. There is also nothing in here covered by the NDA. It was developed under Cocoa for OS X, but since it only uses foundation objects, it should, in theory, work just as well on the iPhone. Because of the NDA, I'm not actually going to say it works on the iPhone... I can neither confirm or deny that it works on said platform. But it should.

So, Where is it Already?

You can get it right here.

You should check out the file SQLitePersistentObject.h, which contains fairly extensive documentation in headerdoc format.

Monday, August 18, 2008

Dynamically adding class objects

Okay, folks, the first draft of the book is done, and I have some breathing room until the chapters start coming back from the tech reviewer and copy editor. Mostly, I'm using that time to catch up on sleep and house jobs and spend some time with the kids, but I've also been working on an idea I mentioned earlier about creating a way for NSObjects to automatically persist themselves into a SQLite database. I just finished my first draft of the code to do that, and once I've done some more thorough testing, I'll do a blog post about it.

Today, I'm going to post one of the little gotchas that bit me while writing that code, and which is really, really hard to figure out from Apple's documentation: how to dynamically add a class method at runtime. What I did in my persistence classes was to create class methods to help you retrieve objects. So, if you wanted to retrieve all the objects in the database where the name property equaled "John", you could do this

NSArray *johns = [Person findByName:@"John"];

Anybody who has worked with Ruby's excellent ActiveRecord implementation will recognize this pattern as the one used there. The way to implement this is to override resolveClassMethod: (you would use resolveInstanceMethod: to do the same thing for instance methods, but that is documented well and easy to implement so I won't discuss it here).

The resolveClassMethod: documentation refers you to resolveInstanceMethod: for an example implementation, which looks like this:

+ (BOOL) resolveInstanceMethod:(SEL)aSEL
if (aSEL == @selector(resolveThisMethodDynamically))
class_addMethod([self class], aSEL, (IMP) dynamicMethodIMP, "v@:");
return YES;
return [super resolveInstanceMethod:aSel];

If you use code like that in your resolveClassMethod:, however, you'll get an unrecognized selector error at runtime. Why? Because that code adds a method to the class object, which means you added an instance method to that class. That's not what you want. What you want is to add a method to your class' metaclass object instance, which means going to the runtime to get the Class object that represents the class' metaclass object (say that five times fast!). So, the example above, when turned into a class method override, would look like this:

+ (BOOL) resolveClassMethod:(SEL)aSEL
if (aSEL == @selector(resolveThisMethodDynamically))
Class selfMetaClass = objc_getMetaClass([[self className] UTF8String]);
class_addMethod([selfMetaClass, aSEL, (IMP) dynamicMethodIMP, "v@:");
return YES;
return [super resolveClassMethod aSEL];

And, in the immortal words of Bugs Bunny: Viola! You've added a class method to your class at runtime. Of course, you have to create the function dynamicMethodIMP in order for this to work and do all the other assorted steps necessary to make dynamic methods work, but this isn't a tutorial on dynamic method creation, just a solution to one gotcha you may encounter while doing it.

Friday, August 8, 2008

Book's Almost Done

I just finished Chapter 16. I'll give it another read-over in the morning then it will go off to my writing partner for his review, then we'll bat it back and forth a few times until we're both happy with it. There are only two more chapters left, and one of them should be tiny.

The book has ended up being a lot longer than we first envisioned it. Our original concept was a very basic, very beginner-level book. What we ended up writing was almost a soup-to-nuts book that will end up being over 500 pages including front and back matter. The current manuscript is at 470 pages with two chapters left, and that doesn't include the index, TOC, preface.

I'm a little bummed, we ended up cutting a chapter that I would have liked to write. It would have made the book too long and it didn't really belong in a book with "Beginning" in the title, but I thought it was important and kinda wish we had the luxury to include it.

Looks like we missed our deadline by probably ten days. Not bad considering it's over 150 pages longer than our contractual requirement. We probably could have made the deadline, but I think the quality would have suffered, and since the NDA is still in effect, we can't deliver the manuscript anyway, so I think we did the right thing by taking an extra week or two.

I'm actually really, really happy with the book. Dave's a great writer and our tech reviewer is phenomenal, so having the two of them keeping me honest can't help but make me look good.

Man, am I tired. I haven't had a day off in two months, and I haven't had a day shorter than 13 hours in six weeks. Once I send the last chapter to Dave, I'm going to jump for joy, and then sleep for a week.