BlackDog Foundry

Overview

This service is designed to assist you in keeping your application reference (or operational) data up-to-date. For example, consider the case where you have developed an application that stores train timetable information in an sqlite database on your iphone. This information would be correct at the time of application publication, however, over time it would quickly become out of date. The replication service that BlackDog offers is to store a copy of the “master” data on the BlackDog server (which you can update at any time), and the devices connect to the BlackDog server and replicate down only the changed records.

Many developers write their own data replication behaviour, but it is complex and error-prone. The BlackDog Replication service allows you to concentrate on developing your application’s behaviour and core functionality, not dealing with challenges like replication.

This tutorial describes the steps you will need to perform in order to integrate the sqlite replication behaviour into your application. It assumes that you are have previously used XCode to develop iphone applications.

The overriding premise of the BlackDog replication behaviour is that you, as the application developer, can develop your application with only a small amount of consideration to the replication that occurs under the covers. This means that you should be able to develop your application reading from the database with no BlackDog functionality, and then add in the BlackDog replication once your application is working.

Looking at a typical iphone development project, it would normally involve the following high-level sequence of steps:

• Create initial sqlite database that contains some data
• Copy that database into your project’s resource folder
• Build your application that reads from that database using sqlite APIs
• Finalise the contents of the sqlite database
• Perform final testing
• Ship the application to Apple for approval

Contrast that to the workflow when using the BlackDog replication libraries (new steps in bold):

• Create initial sqlite database that contains some data
• Copy that database into your project’s resource folder
• Build your application that reads from that database using sqlite APIs
• Finalise the contents of the sqlite database
• Upload the contents of the sqlite database to the BlackDog server
• Add a few lines of code to invoke the BlackDog replication service
• Perform final testing
• Ship the application to Apple for approval

Downloading BlackDog Libraries

The first thing you will need to download is the latest BlackDog replication libraries. The BlackDog libraries come in a ZIP file that contains a number of libraries, but the two that you will need to use the replication functionality are:

• bdcommon
• bdrepl

When you unzip that file, you will find the following file hierarchy:

You will note that there are libraries provided for both Release and Debug compilation options, as well as the device and simulator runtime engines. The Headers directory contains the Objective-C header files that define the BlackDog functionality. Unzip this file into a directory of your choosing. For the purposes of this tutorial, I assume that the zip file was unzipped into /Users/xxxx/BlackDog

Setting up your XCode Environment

As all developers like to manage their development environment settings, there is no single canonical way to set up your environment. However, there are several key objectives that must be achieved to link in the BlackDog replication libraries.

• Include the Headers directory
• Link the replication libraries
• Add some pre-requisite frameworks

One possible way of achieving each of these is described below.

Include the Headers directory

Right-click on your Target, and choose Get Info. On the Build tab, type “header search” in the filter field and enter /Users/xxxx/BlackDog/Headers into the “Header Search Paths” setting.

Link the replication libraries

In the Target Info Build tab, filter for “other linker flags” and enter -all_load -lbdcommon -lbdrepl in the “Other Linker Flags” setting.

In the Target Info Build tab, filter for “library search” and add /Users/xxxx/BlackDog/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME) in the “Library Search Paths” setting.

Note that the use of the $(CONFIGURATION) and $(EFFECTIVE_PLATFORM_NAME) variables are a clever technique that allows for dynamically picking up the correct BlackDog runtime library depending on whether you are running on the device or simulator, or in Debug/Release mode.

Add some pre-requisite frameworks

In the Target Info General tab, add the following frameworks/dynamic libraries:

• CFNetwork.framework
• libxml2.dylib
• libsqlite3.0.dylib

Calling the Libraries

Once your XCode environment has been setup correctly, you can now start calling the Replication libraries. The functionality that is available is defined in the BlackDogReplication.h file in the Headers folder. The API is very simple:

@interface BlackDogReplication : NSObject {	
}
 
/**
 * Launches a synchronous replication.  If you require an asynchronous
 * invocation, use a separate background thread to run this function.
 * 
 * Parameters:
 *   delegate - The delegate that provides the input parameters and responds
 *              to the callback functions as replication occurs
 *   fullReplication - Should we get a full replication, or a delta.  In nearly
 *                     all cases, this value will be false as you would normally
 *                     only want to replicate changes since your last update.
 *                     However, there may be cases where you want to completely
 *                     refresh the contents of the table.  In those rare cases,
 *                     pass true in this parameter.
 */
+(void)replicate:(id<BlackDogReplicationDelegate>)delegate fullReplication:(BOOL)fullReplication;
@end

There is a single static method defined on the BlackDogReplication class that takes your delegate and a boolean indicating whether you want a full replication, or an incremental replication. You will nearly always want an incremental so you should pass NO as the parameter. You will need to define a delegate class that can be used, and it will need to implement the methods that are defined in the BlackDogReplicationDelegate class and its parent class (BlackDogDelegate). The simplest example of a delegate class looks like:

@implementation DemoDelegate
 
-(NSString *)appKey {
	return @"cd79fbe3ae6d41ba9daad1acf45b5bff";
}
-(NSString *)secretKey {
	return @"3a1248037bc74fd2aa512f4e8feabf90";
}
-(NSString *)sqliteDatabasePath {
	NSString *docDirectory = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
	return [docDirectory stringByAppendingPathComponent:@"database.sqlite"];
}
-(NSString *)tableName {
	return @"mytable";
}
-(int)initialChangeId {
	return 1;
}
@end

An example invocation of the API looks like:

DemoDelegate *delegate = [[DemoDelegate alloc] init];
[BlackDogReplication replicate:delegate fullReplication:NO];
[delegate release];

The above code connects to the BlackDog server and replicates any changes since your device last replicated data from the master server.

Updating your data

There are two ways to update your master data:

1. Uploading a complete sqlite database via the BlackDog website (non-incremental)
2. Invoking a web service (XML-based) APIs to send incremental updates to the BlackDog server

Some important notes

• When you initially upload an sqlite database to the BlackDog website, it performs a number of validations, and creates a snapshot of the data with a changeID of 1.
• Every time changes get made to the table and uploaded to the BlackDog website, the changed records (including deletions) are flagged with the next ChangeID.
• The client libraries know what the latest ChangeID is for the data on the device, so when it performs a replication, it only fetches the rows that have been changed since that ChangeID.
• If your replicated tables do not make use of a primary key column, the BlackDog replication service is unable to provide incremental replication. That is, the whole table will be replicated even if only one row has changed.
• If you delete a replicated table from the BlackDog server, the data is gone forever. You can upload a fresh copy of the table, but it will be given a ChangeID of 1 and your devices are very likely to have a ChangeID of, say, 23. In this case the client will download a complete set of the data again.

Caveats for uploading complete sqlite databases

• If you upload a complete sqlite database via the website, it is treated as non-incremental. That is, when devices replicate they will receive a complete data refresh. If you need incremental changes to be incremented, you need to use the BlackDog web services which will be described in a later tutorial.
• If the uploaded sqlite database contains multiple tables, all tables will be registered for replication (if there are tables in the database that you do not need for replication, you can delete the reference to them on the BlackDog website.) Tables that already exist on the BlackDog server will be updated with a new ChangeID; new tables will be initialised with a ChangeID of 1.