[Previous] [Contents] [Index] [Next]

Resource Managers

Introduction

To give QNX Neutrino a great degree of flexibility, to minimize the runtime memory requirements of the final system, and to cope with the wide variety of devices that may be found in a custom embedded system, the OS allows user-written processes to act as resource managers that can be started and stopped dynamically.

Resource managers are typically responsible for presenting an interface to various types of devices. This may involve managing actual hardware devices (like serial ports, parallel ports, network cards, and disk drives) or virtual devices (like /dev/null, a network filesystem, and pseudo-ttys).

In other operating systems, this functionality is traditionally associated with device drivers. But unlike device drivers, resource managers don't require any special arrangements with the kernel. In fact, a resource manager looks just like any other user-level program.

What is a resource manager?

Since QNX Neutrino is a distributed, microkernel OS with virtually all nonkernel functionality provided by user-installable programs, a clean and well-defined interface is required between client programs and resource managers. All resource manager functions are documented; there's no "magic" or private interface between the kernel and a resource manager.

In fact, a resource manager is basically a user-level server program that accepts messages from other programs and, optionally, communicates with hardware. Again, the power and flexibility of our native IPC services allow the resource manager to be decoupled from the OS.

The binding between the resource manager and the client programs that use the associated resource is done through a flexible mechanism called pathname space mapping.

In pathname space mapping, an association is made between a pathname and a resource manager. The resource manager sets up this pathname space mapping by informing the process manager that it is the one responsible for handling requests at (or below, in the case of filesystems), a certain mountpoint. This allows the process manager to associate services (i.e. functions provided by resource managers) with pathnames.

For example, a serial port may be managed by a resource manager called devc-ser*, but the actual resource may be called /dev/ser1 in the pathname space. Therefore, when a program requests serial port services, it typically does so by opening a serial port -- in this case /dev/ser1.

Why write a resource manager?

Here are a few reasons why you may want to write a resource manager:

The types of resource managers

Depending on how much work you want to do yourself in order to present a proper POSIX filesystem to the client, you can break resource managers into two types:

Device resource managers

Device resource managers create only single-file entries in the filesystem, each of which is registered with the process manager. Each name usually represents a single device. These resource managers typically rely on the resource-manager library to do most of the work in presenting a POSIX device to the user.

For example, a serial port driver registers names such as /dev/ser1 and /dev/ser2. When the user does ls -l /dev, the library does the necessary handling to respond to the resulting _IO_STAT messages with the proper information. The person who writes the serial port driver can concentrate instead on the details of managing the serial port hardware.

Filesystem resource managers

Filesystem resource managers register a mountpoint with the process manager. A mountpoint is the portion of the path that's registered with the process manager. The remaining parts of the path are managed by the filesystem resource manager. For example, when a filesystem resource manager attaches a mountpoint at /mount, and the path /mount/home/thomasf is examined:

/mount/
Identifies the mountpoint that's managed by the process manager.
home/thomasf
Identifies the remaining part that's to be managed by the filesystem resource manager.

Here are some examples of using filesystem resource managers:

Communication via native IPC

Once a resource manager has established its pathname prefix, it will receive messages whenever any client program tries to do an open(), read(), write(), etc. on that pathname. For example, after devc-ser* has taken over the pathname /dev/ser1, and a client program executes: 

fd = open ("/dev/ser1", O_RDONLY);

the client's C library will construct an io_open message, which it then sends to the devc-ser* resource manager via IPC.

Some time later, when the client program executes: 

read (fd, buf, BUFSIZ);

the client's C library constructs an io_read message, which is then sent to the resource manager.

A key point is that all communications between the client program and the resource manager are done through native IPC messaging. This allows for a number of unique features:

Note: All QNX Neutrino device drivers and filesystems are implemented as resource managers. This means that everything that a "native" QNX Neutrino device driver or filesystem can do, a user-written resource manager can do as well.

Consider FTP filesystems, for instance. Here a resource manager would take over a portion of the pathname space (e.g. /ftp) and allow users to cd into FTP sites to get files. For example, cd /ftp/rtfm.mit.edu/pub, would connect to the FTP site rtfm.mit.edu and change directory to /pub. After that point, the user could open, edit, or copy files.

Application-specific filesystems would be another example of a user-written resource manager. Given an application that makes extensive use of disk-based files, a custom tailored filesystem can be written that works with that application and delivers superior performance.

The possibilities for custom resource managers are limited only by the application developer's imagination.

Resource manager architecture

Here is the heart of a resource manager: 

initialize the dispatch interface
register the pathname with the process manager
DO forever
    receive a message
    SWITCH on the type of message
        CASE io_open:
            perform io_open processing
            ENDCASE
        CASE io_read:
            perform io_read processing
            ENDCASE
        CASE io_write:
            perform io_write processing
            ENDCASE
        .   // etc. handle all other messages
        .   // that may occur, performing
        .   // processing as appropriate
    ENDSWITCH
ENDDO

The architecture contains three parts:

  1. A channel is created so that client programs can connect to the resource manager to send it messages.
  2. The pathname (or pathnames) that the resource manager is going to be responsible for is registered with the process manager, so that it can resolve open requests for that particular pathname to this resource manager.
  3. Messages are received and processed.

This message-processing structure (the switch/case, above) is required for each and every resource manager. However, we provide a set of convenient library functions to handle this functionality (and other key functionality as well).

Message types

Architecturally, there are two categories of messages that a resource manager will receive:

A connect message is issued by the client to perform an operation based on a pathname (e.g. an io_open message). This may involve performing operations such as permission checks (does the client have the correct permission to open this device?) and setting up a context for that request.

An I/O message is one that relies upon this context (created between the client and the resource manager) to perform subsequent processing of I/O messages (e.g. io_read).

There are good reasons for this design. It would be inefficient to pass the full pathname for each and every read() request, for example. The io_open handler can also perform tasks that we want done only once (e.g. permission checks), rather than with each I/O message. Also, when the read() has read 4096 bytes from a disk file, there may be another 20 megabytes still waiting to be read. Therefore, the read() function would need to have some context information telling it the position within the file it's reading from.

The resource manager shared library

In a custom embedded system, part of the design effort may be spent writing a resource manager, because there may not be an off-the-shelf driver available for the custom hardware component in the system.

Our resource manager shared library makes this task relatively simple.

Automatic default message handling

If there are functions that the resource manager doesn't want to handle for some reason (e.g. a digital-to-analog converter doesn't support a function such as lseek(), or the software doesn't require it), the shared library will conveniently supply default actions.

There are two levels of default actions:

For more information on default actions, see the section on "Second-level default message handling" in this chapter.

open(), dup(), and close()

Another convenient service that the resource manager shared library provides is the automatic handling of dup() messages.

Suppose that the client program executed code that eventually ended up performing: 

fd = open ("/dev/device", O_RDONLY);
...
fd2 = dup (fd);
...
fd3 = dup (fd);
...
close (fd3);
...
close (fd2);
...
close (fd);

The client would generate an io_open message for the first open(), and then two io_dup messages for the two dup() calls. Then, when the client executed the close() calls, three io_close messages would be generated.

Since the dup() functions generate duplicates of the file descriptors, new context information should not be allocated for each one. When the io_close messages arrive, because no new context has been allocated for each dup(), no release of the memory by each io_close message should occur either! (If it did, the first close would wipe out the context.)

The resource manager shared library provides default handlers that keep track of the open(), dup(), and close() messages and perform work only for the last close (i.e. the third io_close message in the example above).

Multiple thread handling

One of the salient features of QNX Neutrino is the ability to use threads. By using multiple threads, a resource manager can be structured so that several threads are waiting for messages and then simultaneously handling them.

This thread management is another convenient function provided by the resource manager shared library. Besides keeping track of both the number of threads created and the number of threads waiting, the library also takes care of maintaining the optimal number of threads.

Dispatch functions

The OS provides a set of dispatch_* functions that:

For more information, see the chapter on Writing a Resource Manager in the Programmer's Guide.

Combine messages

In order to conserve network bandwidth and to provide support for atomic operations, the OS supports combine messages. A combine message is constructed by the client's C library and consists of a number of I/O and/or connect messages packaged together into one.

For example, the function readblock() allows a thread to atomically perform an lseek() and read() operation. This is done in the client library by combining the io_lseek and io_read messages into one. When the resource manager shared library receives the message, it will process both the io_lseek and io_read messages, effectively making that readblock() function behave atomically.

Combine messages are also useful for the stat() function. A stat() call can be implemented in the client's library as an open(), fstat(), and close(). Instead of generating three separate messages (one for each of the component functions), the library puts them together into one contiguous combine message. This boosts performance, especially over a networked connection, and also simplifies the resource manager, which doesn't need a connect function to handle stat().

The resource manager shared library takes care of the issues associated with breaking out the individual components of the combine message and passing them to the various handler functions supplied. Again, this minimizes the effort associated with writing a resource manager.

Second-level default message handling

Since a large number of the messages received by a resource manager deal with a common set of attributes, the OS provides another level of default handling. This second level, called the iofunc_*() shared library, allows a resource manager to handle functions like stat(), chmod(), chown(), lseek(), etc. automatically, without the programmer having to write additional code. As an added benefit, these iofunc_*() default handlers implement the POSIX semantics for the messages, again offloading work from the programmer.

Three main structures need to be considered:

Three data structures of a resource manager

A resource manager is responsible for three data structures.

The first data structure, the context, has already been discussed (see the section on "Message types"). It holds data used on a per-open basis, such as the current position into a file (the lseek() offset).

Since a resource manager may be responsible for more than one device (e.g. devc-ser* may be responsible for /dev/ser1, /dev/ser2, /dev/ser3, etc.), the attributes structure holds data on a per-device basis. The attributes structure contains such items as the user and group ID of the owner of the device, the last modification time, etc.

For filesystem (block I/O device) managers, one more structure is used. This is the mount structure, which contains data items that are global to the entire mount device.

When a number of client programs have opened various devices on a particular resource, the data structures may look like this:

Context blocks, attributes structures, and a mount structure

Multiple clients opening various devices.

The iofunc_*() default functions operate on the assumption that the programmer has used the default definitions for the context block and the attributes structures. This is a safe assumption for two reasons:

  1. The default context and attribute structures contain sufficient information for most applications.
  2. If the default structures don't hold enough information, they can be encapsulated within the structures that you've defined.

By definition, the default structures must be the first members of their respective superstructures, allowing clean and simple access to the requisite base members by the iofunc_*() default functions:

Encapsulated and encapsulating structures

Encapsulation.

The library contains iofunc_*() default handlers for these client functions:

chmod()
chown()
close()
devctl()
fpathconf()
fseek()
fstat()
lock()
lseek()
mmap()
open()
pathconf()
stat()
utime()

Summary

By supporting pathname space mapping, having a well-defined interface to resource managers, and providing a set of libraries for common resource manager functions, QNX Neutrino offers the developer unprecedented flexibility and simplicity in developing "drivers" for new hardware -- a critical feature for many embedded systems.

[Previous] [Contents] [Index] [Next]