This is a repost of an article I published on the Realmac Software blog.

Since the introduction of sandboxing on OS X, there has been much discussion about whether an application is a good fit for the sandbox. The truth is, if your users care about security they will want your application to be sandboxed. The Mac App Store also now requires sandboxing so it is a pretty clear message that application developers should adopt it.

Keeping this in mind, in this article we will discuss how to implement shared user defaults between multiple sandboxed applications.

User defaults in a Cocoa application

To store user defaults, Cocoa applications often rely on the NSUserDefaults class from the Foundation framework. NSUserDefaults offers a nice key-value API to store and retrieve preferences. Internally NSUserDefaults uses CFPreferences from the CoreFoundation framework that takes care of the actual storage of the preferences plist on disk. Even though it’s a CFPreferences implementation details, the plist is written to the preferences directory under ~/Library/Preferences/. When an application creates an instance of NSUserDefaults, the content of this plist is retrieved from disk and is kept synchronised when values are updated by the application.

NSUserDefaults also offers a handy method addSuiteNamed: which allows an application A to mount the preferences of another application B in a read-only mode. Internally this method looks for the plist file for application B based on its bundle identifier inside the Preferences folder and adds it to the set of User Defaults visible to application A.

Taking this into account, we will now see how application sandboxing slightly disrupted this model.

The model before the sandbox

If you don’t know much about sandboxing (or need to refresh your memory about it) I strongly recommend that you read the App Sandbox Design Guide and watch sessions 203 and 204 from WWDC 2011 and session 700 from WWDC 2012 on Apple’s website.

Simply put, before the introduction of sandboxing on OS X, any application was free to access any user data without restriction. Users were still prompted for their password in order to access privileged resources but essentially every application on the system had full access to all resources in user land, leading to plenty of security concerns.

Not sandboxed applications

The sandbox

A sandboxed application however cannot access resources outside of its own sandbox located under ~/Library/Containers/codesign-identifier but has unrestricted access to items inside it. In the App Sandbox Design Guide Apple defines it as following:

App Sandbox is an access control technology provided in OS X, enforced at the kernel level. Its strategy is twofold:

  1. App Sandbox enables you to describe how your app interacts with the system. The system then grants your app the access it needs to get its job done, and no more.
  2. App Sandbox allows the user to transparently grant your app additional access by way of Open and Save dialogs, drag and drop, and other familiar user interactions.

The second point is key since it essentially means that no resource outside of the sandbox can be granted access without specific user interaction.

Sandboxed applications

User defaults and the sandbox

In the case of preferences, this means that whereas non-sandboxed applications have full read-write access to the shared Preferences folder under ~/Library/Preferences/, sandboxed applications have their own version of this folder inside their sandbox with unrestricted access only to content created inside the scope of the application itself.

What this means is that APIs such as addSuiteNamed: on NSUserDefaults simply do not work with the sandbox. The NSUserDefaults class reference indeed states:

Sandbox Considerations

A sandboxed app cannot access or modify the preferences for any other app. (For example, if you add another app's domain using the addSuiteNamed: method, you do not gain access to that app's preferences.)

Attempting to access or modify with another app's preferences does not result in an error, but when you do, OS X actually reads and writes files located within your app's container, rather than the actual preference files for the other application.

Despite these considerations Apple made available two temporary exception entitlements to read and write shared preferences: com.apple.security.temporary-exception.shared-preference.read-only and com.apple.security.temporary-exception.shared-preference.read-write. However, because of their temporary nature we decided not to use them in any new development and our solution doesn’t rely on them.

In short, we cannot use the default implementation of NSUserDefaults to write and read user preferences that need to be shared between multiple applications.

Looking for a workaround, let’s first assume an application A that depends on a helper application B. Both applications are bundled together (with the helper application B likely to be located in the Login Items directory inside the main application A’s bundle) and communicate by some means of IPC (NSXPCConnection being the perfect fit for this job). By simply using NSUserDefaults, preferences created by each application will be stored in their respective sandbox and won’t be accessible to another sandboxed application.

Sandboxing: a fairy tale

When sandboxing was initially introduced in OS X 10.7 Lion, the only way sandboxed applications could share data was with some sort of IPC. Access to that data was however not persistent across launches. With 10.7.3, security-scoped bookmarks were introduced and defined in the App Sandbox Design Guide as:

Starting in OS X v10.7.3, you can retain access to file-system resources by employing a security mechanism, known as security-scoped bookmarks, that preserves user intent. An app-scoped bookmark provides your sandboxed app with persistent access to a user-specified file or folder.

An application could thus be provided with persistent access to a file living in another’s application sandbox. The security-scoped bookmark data would still need to be received via some form of IPC and stored locally to support resolving and accessing the location across launches.

OS X 10.7.4 saw the introduction of a much acclaimed feature in application sandboxing called Application Groups. From the App Sandbox Design Guide:

In addition to per-app containers, beginning in OS X v10.7.4, an application can use entitlements to request access to a shared container that is common to multiple applications produced by the same development team. This container is intended for content that is not user-facing, such as shared caches or databases.

Since OS X 10.7.4, applications defining a common application group entitlement can now access a shared container located at ~/Library/Group Containers/application-group-identifier/. Applications in this group have unrestricted access to any resource inside the shared container.

Sandboxed applications with application group

This feature made adoption of sandboxing much easier for applications relying on a helper process that shared common resources with the main application.

A proposal

So, with this in mind, if we could store a shared preference plist in the group container, multiple applications in the application group could access the same set of user defaults.

NSUserDefaults unfortunately doesn’t support setting a custom preferences location. Luckily, using the powerful file coordination API, we should be able to build a custom user defaults implementation that reads and writes to a shared file in the application group container.

NSFileCoordinator was introduced with OS X 10.7 Lion and its purpose can be summarised as:

The NSFileCoordinator class coordinates the reading and writing of files and directories among multiple processes and objects in the same process.

In short, NSFileCoordinator takes away the heavy lifting of coordinating file access between multiple processes and ensures that applications in the application group can safely update the file by acquiring mutually exclusive access to the file before writing, to avoid stomping on another process’ changes or corrupting the file.

By also adopting the NSFilePresenter protocol we can appropriately be informed of change to the file made by another process:

The NSFilePresenter protocol should be implemented by objects that allow the user to view or edit the content of files or directories. You use file presenters in conjunction with an NSFileCoordinator object to coordinate access to a file or directory among the objects of your application and between your application and other processes. When changes to an item occur, the system notifies objects that adopt this protocol and gives them a chance to respond appropriately.

By having our custom implementation conform to the NSFilePresenter protocol and reading and writing to disk being done using NSFileCoordinator, applications in the application group can safely keep their view of the preferences file up to date.

NSUserDefaults offers a very handy and well-known interface and ideally we would like our custom implementation to match it as much as possible. The best way to ensure this is to simply subclass NSUserDefaults and extend it with our features set.

Using file coordination will also allow us to post NSUserDefaultsDidChangeNotification notification for updates coming from another process. We will also enhance the change notifications by providing a user info dictionary containing the updated default name and value.

The implementation

The whole implementation is provided as a framework and is hosted along with a sample application on GitHub. Disclaimer: This framework is a proof of concept and we aren’t using it in any of our applications (yet!). Bear in mind that we might have overlooked some bugs and you should use it with caution.

So let’s wait no longer and take a look at the code.

Application group identifier

As mentioned above, the preferences plist file will be saved to the application group directory so our class will need to know the application group identifier to resolve the container’s URL. The easiest way to achieve that is by adding a custom initialiser that becomes our designated initialiser.

- (id)initWithApplicationGroupIdentifier:(NSString *)applicationGroupIdentifier;

Note that we also support applicationGroupIdentifier being nil in which case we retrieve the default group container identifier from the application entitlements as such:

SecTaskRef task = SecTaskCreateFromSelf(kCFAllocatorDefault);

CFTypeRef applicationGroupIdentifiers = SecTaskCopyValueForEntitlement(task, CFSTR("com.apple.security.application-groups"), NULL);
if (applicationGroupIdentifiers == NULL || CFGetTypeID(applicationGroupIdentifiers) != 	CFArrayGetTypeID() || CFArrayGetCount(applicationGroupIdentifiers) == 0) {
	return nil;
}

CFTypeRef firstApplicationGroupIdentifier = CFArrayGetValueAtIndex(applicationGroupIdentifiers, 0);
if (CFGetTypeID(firstApplicationGroupIdentifier) != CFStringGetTypeID()) {
	return nil;
}

NSString *applicationGroupIdentifier = (__bridge NSString *)firstApplicationGroupIdentifier;
CFRelease(task);

return applicationGroupIdentifier;

This simply creates the security task for the code making the call and retrieves the value for the com.apple.security.application-groups entitlement.

In case no application group identifier can be retrieved at runtime we will simply throw an exception.

File coordination

Since we want our user default class to be a file presenter we need to conform to the NSFilePresenter protocol and register it with NSFileCoordinator so that it can receive change notifications when the file is modified. This is simply done by calling the class method addFilePresenter: on NSFileCoordinator in our initialiser as such:

[NSFileCoordinator addFilePresenter:self];

In order to conform to NSFilePresenter we are required to override the following two properties:

@property (readonly) NSURL *presentedItemURL;
@property (readonly) NSOperationQueue *presentedItemOperationQueue;

presentedItemURL being the location of the file on disk and presentedItemOperationQueue a private queue that we create in the initialiser on which the file presented methods will be invoked.

We also implement presentedItemDidChange so that we are notified when the presented file is updated by another process using file coordination.

Internal data structure

As previously stated, the data will be stored in a plist file on disk. However, for fast update and retrieval we want an in-memory representation of the actual content that will act as a scratch pad before the data is synchronised with the content of the file on disk.

To achieve this we will simply use two dictionaries, one immutable that will model the data as last seen on disk and a mutable one where we will store keys and values that have been updated since the last synchronisation. Synchronisation will then only become a matter of taking a new picture of the content on disk, compute a diff between the previously seen data on disk, apply our local updates and write the updated content back to disk.

It is important to note that we apply a key-by-key overwriting merge policy where the latest local change overwrites the change on disk in case of conflict, the merge occurring by individual property rather than at a whole. This is similar to what NSMergeByPropertyObjectTrumpMergePolicy does on NSManagedObjectContext.

Accessors

At this point, it is worth mentioning another interesting class: NSUserDefaultsController.

NSUserDefaultsController is a Cocoa bindings compatible controller class. Properties of the shared instance of this class can be bound to user interface elements to access and modify values stored in NSUserDefaults.

NSUserDefaultsController is a very handy controller used in many places in a Cocoa application and for this very reason, we would like our subclass to support it. To ensure that, we need to make sure that our class is KVO compliant for its user defaults keys (i.e. we need to ensure that KVO notifications are posted when the value for a default key is mutated).

NSUserDefaults provides an extensive set of accessors that eventually all funnel through two key methods wrapping some type checking around them. Since we want to keep the type checking behaviour provided by the parent class we only have to override these two methods.

- (void)setObject:(id)object forKey:(NSString *)defaultName;
- (id)objectForKey:(NSString *)defaultName;

Based on what we discussed in the previous paragraph about the data structures, let’s analyse the actual implementation of each method.

- (void)setObject:(id)object forKey:(NSString *)defaultName
{
	[self _lock:[self accessorLock] criticalSection:^ {
		[self willChangeValueForKey:defaultName];
		[[self updatedUserDefaultsDictionary] setObject:object forKey:defaultName];
		[self didChangeValueForKey:defaultName];
	}];
}

The setter is pretty basic, it simply adds the object in the updates dictionary, nicely wrapped up between KVO change calls to ensure key-value observer compliance.

Note that we use a handy helper method to lock a critical section.

- (void)_lock:(id <NSLocking>)lock criticalSection:(void (^)(void))criticalSection
{
	[lock lock];
	criticalSection();
	[lock unlock];
}

Note that accessorLock is an NSRecursiveLock property that we use to protect access to the internal data structures.

The reason for using a lock rather than a “lockless” approach such as a serial queue is so that we can ensure that the KVO notifications are posting on the mutating thread while ensuring atomicity of the mutation and notification messages.

The reason for using a recursive lock is to avoid deadlocks that might be caused by an observer invoking the getter on the very same mutating thread upon receiving a KVO notification.

One final step in this method, omitted for brevity, would be the posting of an NSNotification with an appropriately populated user info dictionary acknowledging the default update.

- (id)objectForKey:(NSString *)defaultName
{
	__block id object = nil;

	[self _lock:[self accessorLock] criticalSection:^ {
		object = [self updatedUserDefaultsDictionary][defaultName];
		if (object != nil) {
			return;
		}
		object = [self userDefaultsDictionary][defaultName];
		if (object != nil) {
			return;
		}
		object = [self registeredUserDefaultsDictionary][defaultName];
	}];

	return object;
}

The getter is also pretty traditional, we first check whether we have an updated value for the given default, otherwise we return the value that was on disk last time we attempted synchronisation. Similarly as in the setter, we make sure that access to our local data structures are correctly wrapped into a lock and unlock calls.

Synchronising

One key aspect of the implementation regards the synchronisation of the user defaults. As we mentioned above we will use file coordination to ensure read and write to the file on disk are done in a coordinated manner. We will also appropriately use a lock to protect access to our internal data structures. Ideally we would like synchronisation to happen every time a default is updated and when the client intends to.

NSUserDefaults API declares a blocking - (BOOL)synchronise; method that does the synchronisation inline. While we want to respect the blocking contract of the API in our implementation we will probably want a non-blocking variant to be invoked “in the background” when opportune. Since updates might be triggered when background synchronising we also need to ensure that the actual mutation/merge and accompanying KVO notifications are posted on the main queue since this is very likely the environment observers such at the user defaults controller expect to receive them.

Synchronisation can then be achieved by executing the following steps:

  1. Get a copy of the locally updated default keys and values.
  2. Get an up-to-date copy of the content dictionary from the file on disk.
  3. Compute the diff between the content dictionary as currently on disk and the version we previously cached.
  4. Apply local changes to the newly retrieved content dictionary.
  5. Safely write the resulting dictionary to disk.
  6. Set the local cache to the resulting dictionary, clean up the locally updated defaults and notify observers of any update.

Note that steps 1 and 6 will have to protect access to the local data structures by appropriately locking and steps 2, 3, 4 and 5 will have to be performed inside a coordinated write on the file coordinator.

Coordinated reading and writing with NSFileCoordinator

In order to ensure that access and updates to the file on disk are coordinated between multiple processes we use an instance of NSFileCoordinator.

NSFileCoordinator *fileCoordinator = [[NSFileCoordinator alloc] initWithFilePresenter:self];

[fileCoordinator coordinateWritingItemAtURL:[self userDefaultsDictionaryLocation] options:NSFileCoordinatorWritingForMerging error:NULL byAccessor:^ (NSURL *userDefaultsDictionaryLocation) {
	NSData *onDiskUserDefaultsData = [NSData dataWithContentsOfURL:userDefaultsDictionaryLocation];
	NSDictionary *onDiskUserDefaultsDictionary = (onDiskUserDefaultsData != nil) ? [NSPropertyListSerialization propertyListWithData:onDiskUserDefaultsData options:NSPropertyListImmutable format:NULL error:NULL] : nil;

	NSDictionary *userDefaultsDictionary = [self _dictionary:onDiskUserDefaultsDictionary byApplyingChanges:userDefaultsUpdatesDictionary];

	NSData *userDefaultsDictionaryData = [NSPropertyListSerialization dataWithPropertyList:userDefaultsDictionary format:NSPropertyListXMLFormat_v1_0 options:(NSPropertyListWriteOptions)0 error:NULL];
	[userDefaultsDictionaryData writeToURL:userDefaultsDictionaryLocation options:NSDataWritingAtomic error:NULL];
}];

Nothing fancy here, we just get the current content from disk as an NSData, convert it to a dictionary by using NSPropertyListSerialization, update it with the local changes, encode it back to NSData and eventually write it back to disk atomically.

Note that this coordinated writing will trigger the presentedItemDidChange method on file presenters in other processes to be triggered.

Coalescing synchronisation calls

Since we will attempt synchronisation after each time a user default is set, we want to try to coalesce synchronising operations. A nice API to achieve such a thing is NSNotificationQueue where notifications are enqueued and can be coalesced based on their name or sender. Unfortunately NSNotificationQueue is very much related to NSRunLoop and one simply cannot rely on the notifications being received if the environment in which they are posted doesn’t have a runloop running (for a more thorough explanation I recommend reading the warning in the Foundation release notes).

In order to achieve a similar behaviour we will use a block-stealing operation.

@interface RMCoalescingOperation

- (id)initWithBlock:(void (^)(void))block;

@property (copy, nonatomic) void (^block)(void);

@end

@implementation RMCoalescingOperation

(…)

- (BOOL)replaceBlock:(void (^)(void))block
{
	@synchronized (self) {
		if ([self block] == nil) {
			return NO;
		}
		[self setBlock:block];
	}
	return YES;
}

- (void)main
{
	void (^block)(void) = nil;

	@synchronized (self) {
		block = [self block];
		[self setBlock:nil];
	}

	block();
}

@end

This operation subclass, similarly to NSBlockOperation, is initialised with a block that is executed in the main method. It additionally supports replacing the block, if it is yet to be sent main the body of the operation can be swapped for a new block. This comes in handy when deciding whether to create a new synchronisation operation or reuse one currently sitting in the queue.

RMCoalescingOperation *lastSynchronizationOperation = [self lastSynchronizationOperation];

void (^synchronizationBlock)(void) = ^ {
	[self synchronize];
};

if ([lastSynchronizationOperation replaceBlock:synchronizationBlock]) {
	return;
}

RMCoalescingOperation *synchronizationOperation = [RMCoalescingOperation coalescingOperationWithBlock:synchronizationBlock];
[self setLastSynchronizationOperation:synchronizationOperation];
[self synchronizationQueue] addOperation:synchronizationOperation];

Conclusion

With help of the application group entitlement and the fantastic NSFileCoordinator we were able to implement an NSUserDefaults subclass that supports sharing preferences between multiple applications with a nice notifying API.

If you have any comments, you can contact me on Twitter, I’m @DamienDeVille.