In my last article I mentioned that CFMessagePortCreateLocal() was not usable on iOS, quoting the documentation as evidence. However, I should have known better since I had recently experimented (and succeeded, more about this soon) with registering a Mach port name with the Bootstrap Server on iOS by using XPC. And, as pointed out by Ari Weinstein, it looks like one can indeed use CFMessagePortCreateLocal() on iOS!

There are a few things at play here so let’s first review the API documentation.

Documentation deep dive

The documentation for CFMessagePort on the iOS Developer Library states the following:

Special Considerations

This method is not available on iOS 7 and later—it will return NULL and log a sandbox violation in syslog. See Concurrency Programming Guide for possible replacement technologies.

This sounds like a pretty clear statement and it sure feels like one should stay away for this API on iOS even though its header is fully available. So why do I believe this is actually a red-herring?

Well, while this consideration makes sense on iOS 7, things have changed quite a bit since the introduction of iOS 8. Starting with iOS 8, an iOS application looks a lot more like an OS X sandboxed application than it used to. The main reason for this, as previously discussed, are application groups.

There is not a lot of documentation regarding sandboxing rules for iOS but luckily the Mac Developer Library has a bunch. Since the technology behind sandboxing on both platforms is pretty identical we can get a good understanding about how things work on iOS by reading the OS X one.

To step back a little, let’s discuss what being sandboxed means for an application. In short, it means no access to the file system outside of the sandbox, no sharing of ports with other processes, no ingoing or outgoing connection, etc… Obviously, since applications wouldn’t do much if all these rules were enforced, Apple provide some entitlements that can restore targeted capabilities to the sandboxed target.

Among these, the com.apple.security.application-groups entitlement was introduced with 10.8 (well technically with 10.7.4 but things didn’t work too well back then) and iOS 8. As previously discussed, application groups give access to a shared container directory on disk and shared user defaults for applications in the same group.

But the extra capabilities provided by application groups are not limited to the file system. In particular, the section about the Application Group Container Directory in the Mac Sandbox Design Guide also has some very interesting information about IPC:

Note: Applications that are members of an application group also gain the ability to share Mach and POSIX semaphores and to use certain other IPC mechanisms in conjunction with other group members. See IPC and POSIX Semaphores and Shared Memory for more details.

Looking at the IPC and Posix Semaphores and Shared Memory section we learn that:

Normally, sandboxed apps cannot use Mach IPC, POSIX semaphores and shared memory, or UNIX domain sockets (usefully). However, by specifying an entitlement that requests membership in an application group, an app can use these technologies to communicate with other members of that application group.

We’ve already covered the UNIX domain sockets but the Mach IPC statement is definitely intriguing and luckily there’s some good news a few paragraphs below:

Any semaphore or Mach port that you wish to access within a sandboxed app must be named according to a special convention:

• Mach port names must begin with the application group identifier, followed by a period (.), followed by a name of your choosing.

Boom!

So it sure looks like multiple applications in the same application group can send Mach messages through a Mach port communication channel, assuming the port name is carefully chosen to start with the application group identifier.

Remember that these docs are for OS X but I don’t see any reason why it wouldn’t work on iOS, despite what the CFMessagePort docs have to say.

OK, that was quite a lot of theory. Let’s take a look at the code and see if these assumptions actually hold in practice.

In practice

Let’s try to use CFMessagePortCreateLocal() to create a port with a dynamically created name (i.e. not prefixed by the application group identifier) and confirmed that it fails.

CFDataRef message_callback(CFMessagePortRef local, SInt32 msgid, CFDataRef data, void *info) {
    return NULL;
}

void create_port(void) {
    CFStringRef port_name = (__bridge CFStringRef)[[NSUUID UUID] UUIDString];
    CFMessagePortRef port = CFMessagePortCreateLocal(kCFAllocatorDefault, port_name, &message_callback, NULL, NULL);
}

Note that you will have to run this on a device since the iOS Simulator doesn’t seem to respect any sandboxing rule.

As expected, the call to CFMessagePortCreateLocal() returns NULL and the following error is printed to the console.

*** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x450f, name = 'E030B15B-3EC6-4A21-A415-668AD0CB1B52'
See /usr/include/servers/bootstrap_defs.h for the error codes.

(lldb) po port
<nil>

Looking at the bootstrap_defs.h header (which is really just a symbolic link to /usr/include/bootstrap.h) as suggested by the error message, we can see that the 1100 error code is BOOTSTRAP_NOT_PRIVILEGED.

So that’s confirmed, we definitely cannot register random dynamic port name in our sandboxed iOS application.

But if we change the port name to be prefixed with the application group identifier (such as com.ddeville.app.group.port) a port is returned and no error is printed to the console, as previously inferred from the docs.

We did also confirm that a local port can indeed be created on iOS when the port name is prefixed by the application group identifier. However, what does make that prefixed name so special and why does CFMessagePortCreateLocal() fail to create a port for names that do not match this requirement?

In order to figure this out we first need to discuss Mach bootstrap registration.

Mach bootstrap registration

Mach ports are used everywhere on OS X. Due to the fact that Apple treats the Mach layer as the very bottom layer in the kernel – with the BSD layer and other subsequent components being implemented on top of it – many operations on OS X eventually translate to Mach ports being involved down the stack.

Mach ports 101

In my opinion, the Kernel Programming Guide has the clearest documentation about Mach ports.

A port is an endpoint of a unidirectional communication channel between a client who requests a service and a server who provides the service. If a reply is to be provided to such a service request, a second port must be used. This is comparable to a (unidirectional) pipe in UNIX parlance.

and

Tasks have permissions to access ports in certain ways (send, receive, send-once); these are called port rights. A port can be accessed only via a right. Ports are often used to grant clients access to objects within Mach. Having the right to send to the object’s IPC port denotes the right to manipulate the object in prescribed ways. As such, port right ownership is the fundamental security mechanism within Mach. Having a right to an object is to have a capability to access or manipulate that object.

and

Traditionally in Mach, the communication channel denoted by a port was always a queue of messages.

and

Ports and port rights do not have systemwide names that allow arbitrary ports or rights to be manipulated directly. Ports can be manipulated by a task only if the task has a port right in its port namespace. A port right is specified by a port name, an integer index into a 32-bit port namespace. Each task has associated with it a single port namespace.

and finally

Tasks acquire port rights when another task explicitly inserts them into its namespace, when they receive rights in messages, by creating objects that return a right to the object, and via Mach calls for certain special ports (mach_thread_self, mach_task_self, and mach_reply_port.)

(Note: in Mach, a task is basically what you would call a process in the BSD world).

Phew! That’s a lot of information. To summarize, a port is a communication channel that lets multiple tasks send messages to each other, assuming that they have the appropriate right to do so and that the port right, specified by the port name, is available in the task namespace.

In user space, this basically translates to mach_port_t, mach_port_name_t and calls to mach_msg(). Since using Mach ports directly can be pretty hard, Core Foundation (CFMachPort and CFMessagePort) and Cocoa (NSMachPort and NSPortMessage) wrappers were created.

Bootstrap registration

When it is created, a Mach task is given a set of special ports including one called the bootstrap port whose purpose is to send messages to the bootstrap server.

Let’s discuss what is the bootstrap server (for more information about this section – and really about the whole article – make sure to read chapter 9 of Amit Singh’s Mac OS X Internals: A Systems Approach).

The Bootstrap Server

In short, the bootstrap server allows tasks to publish ports that other tasks on the same machine can send messages to. It achieves this by managing a list of name-port bindings. The bootstrap server’s functionality is provided by the bootstrap task, whose program encapsulation nowadays is the launchd program.

The reason why a bootstrap server is necessary is because Mach port namespaces are local to tasks. The bootstrap server allows service names and associated ports to be registered and looked up, across tasks.

Registration

In the pre-launchd days (before Mac OS X 10.4 Tiger), one would register a port name by means of the bootstrap_register() function:

kern_return_t
bootstrap_register(mach_port_t bootstrap_port,
                   name_t      service_name,
                   mach_port_t service_port);

The server side of the connection would thus register a name for the port it will read from. With this call, the bootstrap server would provide send rights for the bound port to the client.

On the client side, the bootstrap_look_up() function can be used to retrieve send rights for the service port of the service specified by the service name. Obviously, the service must have been previously registered under this name by the server.

kern_return_t
bootstrap_look_up(mach_port_t     bootstrap_port,
                  name_service_t  service_name,
                  mach_port_t    *service_port);

The register_service() function in the helper application source for mDNSResponder (Rest In Peace) provides a nice demonstration of this technique.

However, the bootstrap_register() function was deprecated with Mac OS X 10.5 Leopard and Apple now recommends to use launchd instead. I won’t go into the details of this decision here (there was a great discussion about it on the darwin-dev mailing list a while ago) but Apple was essentially trying to encourage a launch-on-demand pattern with launchd and this API just didn’t fit with it.

Since using a launchd service or submitting a job via the ServiceManagement is not always appropriate (or possible), there are Cocoa and Core Foundation APIs that take care of registering the name with the bootstrap server by means of an SPI: bootstrap_register2(). These are NSMachBootstrapServer and CFMessagePort.

Since Core Foundation is open source, one can check the implementation of CFMessagePortCreateLocal() and double check that the port name is indeed being registered. It’s also easy to disassemble -[NSMachBootstrapServer registerPort:name:] and realize that it’s essentially wrapping bootstrap_register2(). Remember that NSMachBootstrapServer is only available on OS X so it’s not actually useful to this discussion but it’s still worth keeping in mind.

The circle is now complete.

CFMessagePortCreateLocal and the magic name prefix

Now that we understand the process of registering the port name with the bootstrap server we can look into why using the application group identifier as a prefix for the port name magically works.

By calling into CFMessagePortCreateLocal() with a random name that doesn’t meet the sandbox criteria and setting a symbolic breakpoint on the function we can step through the instructions and find out where it fails.

We can quickly individuate the point of failure (remember that CFMessagePort.c is open source):

kern_return_t ret = bootstrap_register2(bs, (char *)utfname, mp, perPID ? BOOTSTRAP_PER_PID_SERVICE : 0);
if (ret != KERN_SUCCESS) {
    CFLog(kCFLogLevelDebug, CFSTR("*** CFMessagePort: bootstrap_register(): failed %d (0x%x) '%s', port = 0x%x, name = '%s'\nSee /usr/include/servers/bootstrap_defs.h for the error codes."), ret, ret, bootstrap_strerror(ret), mp, utfname);
    CFMachPortInvalidate(native);
    CFRelease(native);
    CFAllocatorDeallocate(kCFAllocatorSystemDefault, utfname);
    CFRelease(memory);
    return NULL;
}

It looks like bootstrap_register2() itself is failing leading CFMessagePortCreateLocal() to print the error and return NULL.

bootstrap_register2() probably ends up being implemented somewhere between launchd and the kernel so we can take a look at the launchd source to try and figure out why it would fail. launchd was not open sourced as part of 10.10 but the 10.9.5 source will do (remember, the source between iOS and OS X will likely be extremely similar if not identical and application groups were introduced on OS X 10.8).

I was not entirely sure where to look for but job_mig_register2 in core.c looks like a good candidate.

These few lines in particular are definitely interesting:

bool per_pid_service = flags & BOOTSTRAP_PER_PID_SERVICE;
#if HAVE_SANDBOX
    if (unlikely(sandbox_check(ldc->pid, "mach-register", per_pid_service ? SANDBOX_FILTER_LOCAL_NAME : SANDBOX_FILTER_GLOBAL_NAME, servicename) > 0)) {
        return BOOTSTRAP_NOT_PRIVILEGED;
    }
#endif

Once again, I had no idea where that sandbox_check() function was implemented so I poked around the included headers to see if anything jump to my eyes. sandbox.h definitely seemed promising but the version in /usr/include/sandbox.h doesn’t declare the function. After some more poking around /usr and disassembling a few libraries I found the implementation in /usr/lib/system/libsystem_sandbox.dylib!

sandbox_check() is pretty lame and is basically a proxy into sandbox_check_common(). The latter does the actual work of checking whether the process requesting the mach-register action can use the provided service name. We could spend another article going through the disassembly of the function so let’s just assume that it does a few checks based on the entitlements of the process and returns whether the service name is allowed or not. In our case, it’s obvious that the function checks whether the service name is prefixed with the application group identifier retrieved from the process entitlements and denies it if it doesn’t.

It’s worth noting that it would take Apple a simple change to the sandbox check to disallow that API on iOS. However, given how many APIs rely on the current mach-register behavior (XPC being one of them) I don’t see this happen any time soon.

Using Mach messages on iOS

Now that we’ve uncovered the conditions under which the API worked, let’s see how one would use it to do IPC on iOS.

Creating the ports

Similarly to Berkeley sockets, we will have a process acting as the server and another one acting as the client. The server will be in charge of registering the port name by creating a local port while the client will simply connect to it by creating a remote port for the same port name. Ordering is important since the remote port creation will fail if the server hasn’t had a chance to register the name yet.

As shown above, the server would create the port by doing:

CFDataRef callback(CFMessagePortRef local, SInt32 msgid, CFDataRef data, void *info)
{
    return NULL;
}

void create_port(void)
{
    CFStringRef port_name = CFSTR("com.ddeville.myapp.group.port");
    CFMessagePortRef port = CFMessagePortCreateLocal(kCFAllocatorDefault, port_name, &callback, NULL, NULL);
    CFMessagePortSetDispatchQueue(port, dispatch_get_main_queue());
}

We schedule the message callbacks to happen on the main queue so that we don’t need to setup a runloop source for the callbacks and manually having to run the runloop while waiting for a reply to a message.

Similarly, on the client side we create the remote message port:

void create_port(void)
{
    CFStringRef port_name = CFSTR("com.ddeville.myapp.group.port");
    CFMessagePortRef port = CFMessagePortCreateRemote(kCFAllocatorDefault, port_name));
}

Since the port creation will fail if the server hasn’t registered the local port yet, an appropriate solution would be to retry every few seconds until it succeeds.

Sending messages

It is important to note that the connection is somewhat unidirectional. While the client can send messages to the server, the server can only reply to the messages synchronously when they are received (you have probably noted that the client doesn’t have a way to set up a message callback).

In the simple case (that is where no reply is expected), the client can send a message by doing:

void send_message(void)
{
    SInt32 messageIdentifier = 1;
    CFDataRef messageData = (__bridge CFDataRef)[@"Hello server!" dataUsingEncoding:NSUTF8StringEncoding];
    CFMessagePortSendRequest(port, messageIdentifier, messageData, 1000, 0, NULL, NULL);
}

As you can see, any data can be sent in the message so LLBSDMessaging could be re-implemented on top of Mach messages. The message identifier integer is also a nice API to distinguish between message types.

Upon sending, on the server side, the callback function will be invoked and the message identifier and data passed through. Nice!

Replying to a message

As previously noted, the server can optionally reply to the message by returning some data synchronously in the callback function. For it to work client side, we need to slightly change the way we send the message.

// On the client
void send_message(void)
{
    SInt32 messageIdentifier = 1;
    CFDataRef messageData = (__bridge CFDataRef)[@"Hello server!" dataUsingEncoding:NSUTF8StringEncoding];

    CFDataRef response = NULL;
    SInt32 status = CFMessagePortSendRequest(port, messageIdentifier, messageData, 1000, 1000, kCFRunLoopDefaultMode, &response);
}

// On the server
CFDataRef callback(CFMessagePortRef local, SInt32 msgid, CFDataRef data, void *info)
{
    return (__bridge CFDataRef)[@"Hello client!" dataUsingEncoding:NSUTF8StringEncoding];
}

Upon return, if no error has happened (you can check the returned status integer) the response reference will point to the data that was sent back by the server.

It’s important to note that CFMessagePortSendRequest() will run the runloop in the specified mode (here kCFRunLoopDefaultMode) thus blocking until the response comes through. We can assume that IPC is pretty fast but the server might still be taking some time to reply. This is where the timeout becomes important: using an appropriate timeout will prevent a thread from being blocked for too long. It’s also probably not a great idea to block the main thread but should you use a background thread remember that it will need to have a serviced runloop ( threads created by a dispatch queue do not have one for example). Another option could be to provide a custom mode on the main thread but be extremely cautious if you need to do this.

Bidirectional communication

As mentioned above, while the server can reply to messages sent by the client, it cannot initiate a new message.

A way to workaround this issue would be to create another pair or ports where the current client act as the registrar. Upon the initial connection from the server, the client would register an additional local port with a new name and send the name to the server. Upon receiving, it would create a remote port matching that name.

This solution is slightly more complicated than the bidirectional-by-nature one provided by Berkeley sockets but it should work as expected. Also, most server-client architectures don’t actually require the server to ever initiate a request since it almost always acts as a response provider.

Conclusion

We were able to register a Mach port with the bootstrap server by creating a service name prefixed by the application group identifier and using CFMessagePortCreateLocal() to take care of the registration. With this set up, it was trivial to send messages between applications.

It wouldn’t be too hard to rewrite LLBSDConnection on top of Mach ports rather than Berkeley sockets. It would also likely be slightly faster. But I guess this post was already long enough so we’ll leave this for another time :-)