IPC Topics

  • Pipes

  • Named Pipes / FIFOs

  • Signals

  • Shared memory

  • Memory mapped files

  • Locking in shared memory

  • Files

  • Domain sockets

  • Doors

  • TCP/IP

IPC Performance Hierarchy

  • Fastest - no kernel interaction needed, except to setup:
    • Shared memory

    • Locking in shared memory

  • Very fast - virtual memory manager involved
    • Memory mapped files

  • Fast - system calls required - i.e. context switches
    • Pipes, FIFOs / Named Pipes

    • Signals

    • Domain sockets

  • Medium / Slow - FS or network involved
    • Files

    • TCP/UDP sockets

Pipes

  • Pipes are the oldest UNIX IPC mechanism aside from files.

  • Pipes are half-duplex - data only flows in one direction

  • Pipes can only be used between processes that have a common parent. Pipes are created by a parent process and then inherited across a fork() call.

  • Passing data from one process to another through a pipe involves at least two context switches. A third one is necessary for control to return to the writing process.

  • Pipes in general are very fast. Sending data at rates of 100s to 1000s of MB/s is possible.

Pipes

  • Pipes have two file descriptors
    • One file descriptor is read-only

    • One file descriptor is write-only

  • The Linux call to create a pipe is:
    • int pipe(int pipefd[2]);

    • pipefd[0] is the read-only half

    • pipefd[1] is the write-only half

  • The Windows call to create a pipe is:
    • BOOL CreatePipe(HANDLE readHandle, HANDLE writeHandle, LPSECURITY_ATTRIBUTES attr, DWORD nSize);

    • readHandle and writeHandle are the read/write file handles

    • attr allows processes that run under more than one user account to manage security of individual pipes

    • nSize is the suggested buffer size for the pipe

Pipes - Linux

 1#include <stdio.h>
 2#include <unistd.h>
 3#include <fcntl.h>
 4#include <stdlib.h>
 5
 6int main(int argc, char* argv[]) {
 7
 8        int pipes[2];
 9        pipe(pipes);
10
11        int inputPartOfPipe = pipes[0];
12        int outputPartOfPipe = pipes[1];
13
14        int pid = fork();
15
16        if(pid > 0) {   //parent process
17                dup2(inputPartOfPipe, 0); // redirect STDIN
18                close(outputPartOfPipe);  // close unused half of pipe
19                int value;
20                scanf("%d\n", &value);
21                printf("child sent value = %d\n", value);
22        } else if(pid == 0) {   //child process
23                dup2(outputPartOfPipe, 1); // redirect STDOUT
24                close(inputPartOfPipe);     // close unused half of pipe
25                printf("%d\n", 5000);
26        } else {
27                printf("fork failed!\n");
28        }
29
30        //don't worry about closing remaining pipes,
31        //process exit does this for us
32
33        return 0;
34}

Pipes - Context Switches

Pipes and Context Switching

Named Pipes / FIFOs

Named pipes are the same as regular pipes except:

  • Named pipes live in the filesystem namespace

  • Named pipes can have a different lifespan than individual processes

  • Since named pipes are files, you can apply more advanced permissions to them

Example usage of a regular pipe in Linux

Example:

$ cat file | gzip -c9 > file.gz

Example usage of a named pipe in Linux:

$ mkfifo pipe_file
$ gzip -c9 < pipe_file > file.gz
$ cat file > pipe_file
$ rm -f pipe_file

Named Pipes - Common Usages

  • Named pipes are commonly used for single machine client / server applications

  • In Windows, named pipes can ride on top of TCP/IP and be used for intra-machine IPC.

  • To improve performance of applications that are written to only work with files and not with normal pipes.
    • Does not work if the program uses random I/O

    • A program that is written to read a large file from disk can instead read from a named pipe. The named pipe’s data can in turn be produced by another program.

    • A program that can only work with files from the local disk can be made to work with a file on the web through a named pipe and a command that can write a URL to standard output

    • This pattern can be used to add encryption, file compression, and other extensions to programs that do not already have them.

Named Pipes - Atomic Reads / Writes

What happens if two processes write to a named pipe at the same time?

  • If the size of each write(…) call is <= PIPE_BUF, the writes will be atomic and in order.

  • If the size of each write(…) call is > PIPE_BUF, then the individual writes will be broken up and interleaved with other simultaneous callers.

  • So, a good rule of thumb, is to make sure that what you write to a named pipe is less than PIPE_BUF in size. Otherwise, you need to guarantee that you are the only one writing to the pipe file

Signals

  • Signals are software interrupts / events

  • Provide a way for handling asynchronous events

  • Modern UNIX systems have > 30 different signals defined

  • Signals have a concept of disposition or action
    • Ignore the signal: works for all signals except SIGKILL, SIGSTOP, and SIGEMT

    • Catch the signal: program registers with the kernel a function to handle a signal.

    • Default: allows the signal to perform its default action. Every signal has a default action

List of Important Signals

List of Important Signals

Signal

Description

SIGABRT

abnormal termination, generated by abort() function. Default terminates and core dumps the process.

SIGALRM

generated when a timer expires. Set by alarm() function.

SIGBUS

Indicates a hardware fault. Often memory protection faults. Default terminates the application.

SIGCHLD

Sent to a parent process when a child process terminates.

SIGCONT

Sent to a stopped process when it is continued.

SIGEMT

General hardware fault

SIGFPE

Arithmetic exception: divide by 0, floating point overflow, etc…

SIGHUP

Terminal is disconnected

SIGILL

The process has executed an illegal instruction (like trying to disable interrupts)

SIGINT

Generated by terminal when we press Ctrl-C

SIGAIO

Generated when an asynchronous I/O event occurs.

SIGKILL

Can’t be caught or ignored. Default action is to kill a process.

SIGPIPE

Generated when you write to a pipe where the reader has been terminated.

SIGSEGV

Segmentation violation

SIGSTOP

Sent before a process is put into the stop state.

Handling Signals - Example

 1static void handler(int);
 2
 3int main(int argv, char* argv[]) {
 4  signal(SIGUSR1, handler);
 5  signal(SIGINT, handler);
 6  while(1) { pause(); }
 7}
 8
 9static void handler(int signalNum) {
10  if(signalNum == SIGUSR1) {
11    printf("received SIGUSR1\n");
12  } else if(signalNum == SIGINT) {
13    printf("received SIGHUP\n");
14  }
15}

Sending Signals - Example

Signals can be sent to a running process using kill:

$ firefox &
[1] 5050
$ kill -USR1 5050        - sends signal SIGUSR to firefox
$ kill 5050              - sends signal SIGTERM to firefox

Signals - Interrupted System Calls

  • Signals will “wake up” blocking calls to “slow” system calls

  • In old UNIX, this was any blocking system call.

  • In modern UNIX, the following are “slow” system calls meaning that they can block for ever in theory:
    • Reads from pipes, terminal devices, and network devices

    • Writes to pipes, terminal devices, and network devices if the data cannot be accepted immediately.

    • Opens of files that block until some condition occurs (such as a serial device or modem connecting).

    • All calls to the pause() function, pause() waits until signals are caught.

  • This means, if you are catching signals, you will have to check errno for EINTR (interrupted by signal) error if these functions return in an error state. If they do, you must retry your operation

  • In newer UNIX systems, if signals are triggered with a SA_RESTART flag passed to sigaction(…), system calls would be automatically restarted after signal handling completes.

Signals - Reentrant Functions

  • When handling a signal, the main execution of the process is suspended. The interrupted state is not something that can be examined or otherwise determined.

  • It is possible that the interrupted state could be inside of a malloc() or free() call. If we call malloc() or free() in the signal handler, we could corrupt the allocated or free list.

  • So, in signal handlers we can only call reentrant functions.

  • Reentrant functions have the following properties:
    • Do not call malloc() or free()

    • Do not refer to mutable static data structures

    • Rule of thumb is to call functions that are either system calls or low level wrappers of system calls are safe. Other functions should not be used unless you are certain of their safety.

Signals - Reentrant Functions

abort       access       alarm      chdir       chmod        chown
close       creat        dup        dup2        execle       execve
exit        fcntl        fork       fstat       getgid       getuid
kill        link         longjmp    lseek       mkdir        mkfifo
open        pathconf     pause      pipe        read         rename
rmdir       setgid       setsid     setuid      sigaction    sigaddset
sigdelset   sigemptyset  signal     sigpending  sigsuspend   sleep
stat        sysconf      time       times       umask        uname
unlink      utime        wait       waitpid     write

Signals - Sending In Code

  • Sending a signal to another process:
    • int kill(pid_t pid, int signo);

    • pid > 0, signal is sent to process with id = pid

    • pid = 0, signal sent to all process whose group ID is the same as the sender of the signal. Will not send to init or swapper daemons.

    • pid < 0, signal is sent to all processes whose process group ID equals the absolute value of pid for which the sender has permission to send the signal.

    • pid = -1, undefined.

  • Sending a signal to your own process:
    • int raise(int signo);

  • Sending SIGALRM (terminates process by default):
    • int alarm(unsigned int seconds);

  • Waiting for signals:
    • int pause(void);

Example - implementing sleep() with alarm

 1    void sig_alrm(int signo) 
 2    {
 3       /* ... */
 4    }
 5
 6    unsigned int sleepFor(int numSeconds) {
 7       signal(SIGALRM, sig_alrm);    //set signal handler
 8       alarm(numSeconds);            //set alarm for n-seconds
 9       pause();                      //wait for signal
10       return alarm(0);              //turn off alarm
11       handler;
12    }

Example - “better” sleep implementation

 1jmpbuf env_alrm;
 2
 3void sig_alrm(int signo) {
 4    longjmp(env_alrm, 1);
 5}
 6
 7unsinged int sleepFor(int numSeconds) {
 8    signal(SIGALRM, sig_alrm);
 9    if(setjmp(env_alrm) == 0) {
10        alarm(nsecs);
11        pause();
12    }
13    return alarm(0);
14}

Demonstrating the sleepFor problem

 1void sig_int(int signo) {
 2    volatile int j = 0;
 3    printf("sig_int enter\n");
 4    for(int i = 0; i < 10000000; i++) {
 5        j += i * i;
 6    }
 7    printf("sig_int done\n");
 8}
 9
10int main(int argc, char* argv[]) {
11    signal(SIGINT, sig_int);
12    unsigned int unslept = sleepFor(1);
13    printf("sleepFor returned: %u\n", unslept);
14    return 0;
15}
16

History of sleep and why signals are scary.

  • These two sleep implementations are similar to historical implementations of the sleep() call in UNIX

  • Both of these demonstrate that implementations of signal handlers need to be handled with great care.

  • You must remember the following between signal and pause: Between the signal registration and the pause call, you may receive the signal. Pause could block forever because of this

  • You must remember the following in signal handlers:
    • You may be interrupting execution of the main program or another signal handler.

    • Avoid modifying global variables

  • Avoid non-reentrant functions
    • Be careful of stack modifying functions like setjmp or longjmp

    • If you need this behavior, look towards sigsetjmp or siglongjmp

    • Be careful of what signals your handler generates

Other uses for alarm(…)

  • Aside from using alarm() to implement a sleep() call

  • alarm() can be used to put an upper time limit on slow or blocking operations.

  • If we want to ensure that we don’t wait on a call to read from a named pipe forever, we can setup an alarm call.

Memory Mapped Files

  • Memory mapped files are one of two ways to share memory regions between applications.

  • First, there are some virtual memory concepts that we need to introduce:
    • Virtual address - the address that the program sees for an object in memory

    • Physical address - the real address (if it exists) of an object in physical memory

    • Physical and virtual addresses are almost never the same.

    • Backing store - the non physical memory location that backs a physical memory object.

  • All physical memory objects have backing stores
    • text segments - executable and library files

    • data, stack segments - swap files

    • memory mapped regions - memory mapped files

  • We will dig deeper into virtual memory in a later lecture, but these concepts will be sufficient to proceed

Memory Mapped Files

  • The technique used in memory mapped files is simple but very powerful.

  • Fundamentally, when you memory map a file with the mmap() function, you are declaring a region of virtual memory to be backed by a file.

  • This means, when you write to this mapped region, the values will appear (to other programs) immediately in the backing file. If the backing file is updated, the values in that file will (to the running program) appear immediately in memory.

  • When two programs map a file into virtual memory, the virtual addresses will most likely differ between the programs.

  • Two programs that have the same region of a memory mapped file mapped to memory will be able to communicate with each other by reading and writing values to that memory region

Memory Mapped Files

Memory Mapped Files - Virtual Addresses

  • There are two instances in virtual memory where position independent code (the use of relative memory addressing) must be used.

  • Shared libraries - because the data / bss segments are shared among processes and mapped to different virtual addresses.

  • Memory mapped files - because the memory mapped file is mapped to different virtual addresses in different processes.

  • To implement position independent code, techniques similar to -fpic used in GCC must be used by code that you write to work with memory mapped regions.

  • Rules for writing position independent code in memory mapped regions:
    • Do not pass pointers or structures that make use of pointers. Pointers use virtual addresses.

    • Objects in memory mapped regions should not refer to objects outside of memory mapped regions.

Absolute Memory Addressing

 1typedef struct {
 2    char[50] Brand;
 3    char[2] TractionRating;
 4    char[2] SpeedRating;
 5} Wheel;
 6
 7typedef struct {
 8    Wheel*[4] Wheels;
 9    char* Make;
10    char* Model;
11} Car;
12
13Car* car = (Car*)malloc(sizeof(Car));
14car.Make = (char*)malloc(sizeof(char)*50);
15car.Model = (char*)malloc(sizeof(char)*50);
16car.Wheels[0] = (Wheel*)malloc(sizeof(Wheel));
17car.Wheels[1] = (Wheel*)malloc(sizeof(Wheel));
18car.Wheels[2] = (Wheel*)malloc(sizeof(Wheel));
19car.Wheels[3] = (Wheel*)malloc(sizeof(Wheel));
20
21car.Wheels[1]->Brand

Here, the location of each of the wheel objects is stored with an absolute reference from the Car. If this is shared in a memory mapped file, a call to car.Wheels[0] will not resolve to a correct address.

Relative Memory Addressing

 1typedef struct {
 2    char[50] Brand;
 3    char[2] TractionRating;
 4    char[2] SpeedRating;
 5} Wheel;
 6
 7typedef struct {
 8    Wheel[4] Wheels;
 9    char[50] Make;
10    char[50] Model;
11} Car;
12
13Car* car = (Car*)malloc(sizeof(Car));

Here, we are using relative addressing. The entire Car struct is allocated in a contiguous memory region. To access each wheel, you need only use an offset from the beginning of the Car struct.

Linux / Minix - Use of mmap()

General Algorithm:

  1. Process A creates file on disk

  2. Process A seeks a length of N

  3. Process A writes a 0 to the file

  4. Process A calls mmap on the file to map it to memory

  5. Process A copies its data structures into the mapped region

  6. Process B calls mmap on the file to map it to memory

  7. Process A and B proceed using the mapped region to cooperate

mmap - simple example

 1int main(int argc, char* argv[]) {
 2    const char *data = "Hello World";
 3    const int dummyValue = 0;
 4    int dataSize = sizeof(char) * strlen(data) + sizeof(char);
 5
 6    int fd = open("shared.dat", O_CREAT|O_TRUNC|O_RDWR, 0666);
 7    lseek(fd, dataSize, SEEK_SET);
 8    write(fd, (char*)&dummyValue, sizeof(char));
 9    
10    void* map = mmap(NULL,dataSize,PROT_READ|PROT_WRITE,MAP_SHARED,fd,0);
11
12    memcpy(map, data, dataSize);
13
14    getchar();
15
16    munmap(map, dataSize);
17    close(fd);
18
19    return 0;
20}
21

mmap - simple example

 1int main(argc, char* argv[]) {
 2    struct stat fileStat;
 3    stat("shared.dat", &fileStat);
 4    int fileSize = fileStat.st_size;
 5
 6    int fd = open("shared.dat", O_RDWR);
 7
 8    void* map = mmap(NULL, fileSize, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
 9
10    char* data = (char*)calloc(1, fileSize);
11
12    memcpy(data, map, fileSize);
13
14    printf("%s\n", data);
15
16    free(data);
17    munmap(map);
18    close(fd);
19
20    return 0;
21}
22

Memory Mapped Files - Atomicity

  • When we explored pipes, we learned that writing to a pipe (below a certain limit) is an atomic operation. We get this guarantee because the OS kernel guarantees this by guaranteeing that system calls are atomic.

  • The only system calls involved in memory mapped files are mmap and munmap. These are only for initializing and cleaning up mapped memory regions.

  • Reading and writing mapped memory regions does not involve system calls and carries no guarantee of atomicity. It is similar to multiple threads writing to the same heap.

  • To achieve atomicity in memory mapped regions, you must use:
    • Mutexes

    • Semaphores

    • Monitors / Condition Variables

  • The implementation that you use must also use relative addressing internally or otherwise be able to be configured for it.

  • The fact that memory mapped regions do not guarantee atomicity and do not use context switches is also why memory mapped regions perform so well.

  • In general, the fewer the guarantees a mechanism provides, the faster it is.

Memory Mapped Files - Bounded Buffer

 1#include <semaphore.h>
 2
 3const int MessageQueueSize = (5);
 4const int MaxMessageSize = (20);
 5
 6class Message {
 7public:
 8  ~Message();
 9  void EnqueueMessage(const char *msg);
10  char* DequeueMessage();
11  static Message *CopyToMemoryMappedFile(int fd);
12  static Message *GetFromMemoryMappedFile(int fd);
13static void ReleaseFile(Message *msg, int fd);
14private:
15  Message();
16  sem_t _lock;
17  sem_t _empty;
18  sem_t _full;
19  int _current;
20  char _messages[MessageQueueSize][MaxMessageSize];
21};
22

Memory Mapped Files - Bounded Buffer

 1Message::Message() {
 2  sem_init(&_lock,  1, 1);                                //passing 1 as the 2nd parameter allows the
 3  sem_init(&_empty, 1, 0);                              //semaphore work in a memory mapped region
 4  sem_init(&_full,  1, MessageQueueSize);
 5  _current = 0;
 6}
 7Message::~Message() { }
 8Message *Message::CopyToMemoryMappedFile(int fd) {
 9  int datasize = sizeof(Message);
10  printf("message size = %d\n", datasize);
11  if(lseek(fd, sizeof(Message), SEEK_SET) == (-1)) {
12     fprintf(stderr, "error in lseek\n");
13  }
14  int dummyVal = 0;
15  if(write(fd, (char*)&dummyVal, sizeof(char)) == (-1)) {
16     fprintf(stderr, "error in write\n");
17  }
18  void *map = mmap(NULL, sizeof(Message), (PROT_READ|PROT_WRITE), MAP_SHARED, fd, 0);
19  if(map == (void*)(-1)) {
20     fprintf(stderr, "mmap() returned -1\n");
21  }
22  Message *msg = new Message();
23  memcpy(map, (void*)msg, sizeof(Message));
24  delete msg;
25  return (Message*)map;
26}
27

Memory Mapped Files - Bounded Buffer

 1Message *Message::GetFromMemoryMappedFile(int fd) {
 2  void *map = mmap(
 3     NULL, sizeof(Message), 
 4     (PROT_READ|PROT_WRITE), MAP_SHARED, fd, 0);
 5  if(map == (void*)(-1)) {
 6     fprintf(stderr, "mmap() returned -1\n");
 7  }
 8  Message* msg = (Message*)map;
 9  return msg;
10}
11
12void Message::ReleaseFile(Message *msg, int fd) {
13  if(munmap((void*)msg, sizeof(Message)) == (-1)) {
14     fprintf(stderr, "munmap() failed\n");
15  }
16}
17

Memory Mapped Files - Bounded Buffer

 1void Message::EnqueueMessage(const char *msg) {
 2  sem_wait(&_full);
 3  sem_wait(&_lock);
 4  _current += 1;
 5  bzero(&_messages[_current], MaxMessageSize*sizeof(char));
 6  memcpy(&_messages[_current], msg, strlen(msg)*sizeof(char));
 7  sem_post(&_lock);
 8  sem_post(&_empty);
 9}
10
11char* Message::DequeueMessage() {
12  char *msg = new char[MaxMessageSize];
13  sem_wait(&_empty);
14  sem_wait(&_lock);
15  memcpy(msg, &_messages[_current], MaxMessageSize*sizeof(char));
16  _current -= 1;
17  sem_post(&_lock);
18  sem_post(&_full);
19  return msg;
20}
21

Memory Mapped Files - Bounded Buffer

Producer Code

 1int main(int argc, char* argv[]) {
 2  const char *sharedFileName = "shared.dat";
 3  const mode_t mode = 0666;
 4  const int openFlags = (O_CREAT | O_TRUNC | O_RDWR);
 5  int fd = open(sharedFileName, openFlags, mode);
 6
 7  if(fd == (-1)) {
 8     printf("open returned (-1)\n");
 9     return (-1);
10  }
11
12  Message* msg = Message::CopyToMemoryMappedFile(fd);
13
14  for(int i = 0; i < 100; i++) {
15     char message[10];
16     sprintf(message, "%d\n", i);
17     msg->EnqueueMessage(&message[0]);
18     printf("enqueued %d\n", i);
19  }
20  printf("message queue written\n");
21  getchar();
22  Message::ReleaseFile(msg, fd);
23  close(fd);
24}

Memory Mapped Files - Bounded Buffer

Consumer Code

 1int main(int argc, char* argv[]) {
 2  const char *sharedFileName = "shared.dat";
 3  const mode_t mode = 0666;
 4  const int openFlags = (O_RDWR);
 5  int fd = open(sharedFileName, openFlags, mode);
 6
 7  if(fd == (-1)) {
 8     printf("open returned (-1)\n");
 9     return (-1);
10  }
11
12  Message* msg = Message::GetFromMemoryMappedFile(fd);
13
14  int count = 0;
15
16  while(1) {
17     char *message = msg->DequeueMessage();
18     printf("%d: %s", ++count, message);
19     fflush(stdout);
20  }
21
22  Message::ReleaseFile(msg, fd);
23
24  close(fd);
25}

Memory Mapped Files - Fast I/O

Memory mapped I/O is faster because it avoids a copy from user mode to kernel mode

Normal User - Kernel write(…) call algorithm:

  1. user app - write(fd, user_buf, len);

  2. user app - context switch into OS (software interrupt)

  3. kernel mode - allocate space in file, check security, etc…

  4. kernel mode - copy user_buf to FS buffer cache

  5. kernel mode - context switch into user app (interrupt return)

  6. at some later time, kernel commits buffer cache to disk

Normal User - Kernel mmap(…) write algorithm:

  1. user app - copy values to mapped region

  2. kernel mode - MMU triggers page fault (hardware interrupt)

  3. kernel mode - writes page to backing store

  4. kernel mode - context switch to user app (interrupt return)

Memory Mapped Files - Fast I/O

  • The read algorithm for regular read vs. mmap read is very similar to the write algorithm and also avoids a kernel to user mode copy

  • On systems with large address spaces (64-bit), memory mapped I/O can be very advantageous. - For example, a database server can memory map an entire database that is several TB in size into memory.

  • Since most VM systems user a very efficient LRU algorithm and have a lot of I/O scheduling data, memory mapping large files is amongst the fastest approaches.

  • Other advanced approaches include scatter/gather or vectored I/O.

  • Memory mapped I/O carries the greatest advantage when the structure of the file maps well into the domain model. This means that no serialization / deserialization is needed.

Files - IPC

  • Files are the oldest and most generally used form of IPC

  • Almost every resource in a modern operating system is accessible through a file based contract (open, read, write, seek, close)

  • File based IPC is available in almost every operating system.

  • File based IPC, in modern times is best considered in a few patterns:
    • State persistence - beyond the lifetime of a program

    • Exposing current state - during program execution

    • Queues / Spooling folders - mail daemons, printers, execution queues

    • Resource state expression - lock files, availability, etc…

Files - Exposing Current State

/proc filesystems are important ways for OS designers to expose system information without inventing new system calls.

  • This is a key advantage for kernel module developers and for device driver developers

  • Also provides a great way to debug kernel changes

List of /proc-style Filesystems

Filesystem

Description

Filesystem

Description

/proc/vmstat

virtual memory stats and configuration

/proc/cpuinfo

individual CPU information

/proc/<PID>

individual process information

/proc/loadavg

moving average of ready process load

Spool Folders

  • Spool folders are most commonly used for:
    • E-mail daemons (Postfix, Exchange, etc…)

    • Printer managers (CUPS, lpr, etc…)

    • Job managers (CRON, etc…)

  • Spool folders are folders reserved for a single process that’s single task is to monitor the folder and process each new file created in the folder. Each file in the folder represents a task to complete.

  • Spool folders, like other persisted queueing systems are resilient to failure. If the processing daemon crashes, it can be restarted without losing the task list.

  • Often, spool folders are processed by some defined order. One typical order is to process files alphabetically.

  • If the jobs are not meant to be repeated, files are typically moved to another folder upon completion or deleted.

Spool Folders - Cron

Cron typically maintains a few spool folders under /etc:

  • /etc/cron.daily

  • /etc/cron.hourly

  • /etc/cron.monthly

  • /etc/cron.weekly

Cron will, at the correct time, enter each of these folders and execute every executable file in each of these folders.

Spool Folders - CUPS

  • CUPS is a printer daemon for UNIX operating systems

  • The spool folder for CUPS is typically /var/spool/cups

  • In the spool folder there are two types of files:
    • Data files - The object that is being printed
      • Named d00001-001, d00001-002, d00002-001,….

    • Control files - A file that represents a set of data files
      • Named c00001, c00002, ….

  • The use of this file naming convention exposes a domain model of sorts that the CUPS system respects in its queue.

  • CUPS has a series of programs that manage the creation of files in the spooling folder and the processing of files once in the spooling folder.

Lock Files

  • A useful aspect of all system calls is that they are atomic operations. This means that some system calls can be used to perform test-and-set like operations and be the basis of locking systems.

  • A common example is the use of files to make sure only one instance of a software program is running at one time.
    • For example, you would not want to HTTP daemons running on the same port.

    • To prevent this, you could create a lock file like /var/lock/http_80.

    • When the HTTP daemon starts up, it can check very early on for a /var/lock/http_## for the port it is configured on before proceeding. When the daemon shuts down it can delete the file

  • The creation of and deletion of files is an atomic operation. So, if two processes both try at the same time to create the same file, only one will succeed.

Doors

  • Doors pretty much only exist on Solaris and nowhere else

  • Even so, they are an interesting concept.

  • Doors basically allow a process to expose one or more functions through one or more files on the filesystem. It is basically a file system based RPC mechanism

Server Code

 1void server(void* cookie, char* argp, size_t arg_size, door_desc_t* dp, uint_t n_desc) {
 2
 3    /*server code goes here*/
 4
 5}
 6
 7int doorfd = door_create(server, 0, 0);
 8int fd = creat("/tmp/door", 0666);
 9fdetach("/tmp/door");
10fattach(doorfd,"/tmp/door");
11pause();
12

Client Code

 1typdef struct myArg {
 2    int id
 3} myArg_t;
 4
 5door_arg_t d_arg;
 6int doorfd = open("/tmp/door", O_RDONLY);
 7myArg_t* arg = (myArg_t*)malloc(sizeof(myArg_t));
 8arg->id = 12;
 9d_arg.data_ptr = (char*)arg;
10d_arg.data_size = sizeof(*arg);
11d_arg.desc_ptr = NULL;
12d_arg.desc_num = 0;
13door_call(doorfd, &d_arg);

Domain Sockets

  • Domain sockets serve the same purpose as named pipes, except that domain sockets are:
    • Full - duplex

    • Can have many clients to one server

    • Support datagram and streaming modes

  • Domain sockets use functions that are very similar to what is used with internet sockets:
    • accept, bind, listen, connect, socket

  • The setup for domain sockets can be a bit complex, so the best way to explain is through a simple example.

Domain Sockets - Example

 1int server_listen(const char *fileName) {
 2  unlink(fileName);
 3  int socket_fd = socket(PF_UNIX, SOCK_STREAM, 0);
 4  struct sockaddr_un address;
 5  memset(&address, 0, sizeof(struct sockaddr_un));
 6  address.sun_family = AF_UNIX;
 7  sprintf(address.sun_path, fileName);
 8
 9  bind(socket_fd, (struct sockaddr*)&address, sizeof(struct sockaddr_un));
10  listen(socket_fd, 5);
11  int connection_fd;
12  socklen_t address_length;
13  while((connection_fd = accept(socket_fd, (struct sockaddr*)&address, &address_length)) > (-1)) {
14     int child = fork();
15     if(child == 0) {
16        return connection_handler(connection_fd);
17     } else {
18        close(connection_fd);
19     } 
20  }
21
22  close(socket_fd);
23  unlink(fileName);
24  return 0;
25}

Domain Sockets - Example

 1int connection_handler(int socket_fd) {
 2  char buff[256];
 3  int nBytes = read(socket_fd, buff, 256);
 4  buff[nBytes] = 0;
 5  printf("message from client: %s\n", buff);
 6  nBytes = snprintf(buff, 256, "hello from server");
 7  write(socket_fd, buff, nBytes);
 8
 9  close(socket_fd);
10  return 0;
11}
12

Domain Sockets - Example

 1int client_connect(const char* fileName) {
 2  int socket_fd = socket(PF_UNIX, SOCK_STREAM, 0);
 3
 4  struct sockaddr_un address;
 5  memset(&address, 0, sizeof(struct sockaddr_un));
 6  address.sun_family = AF_UNIX;
 7  sprintf(address.sun_path, fileName);
 8
 9  connect(socket_fd, (struct sockaddr*)&address, sizeof(struct sockaddr_un));
10  char buffer[256];
11  int nBytes = snprintf(buffer, 256, "hello from a client");
12  write(socket_fd, buffer, nBytes);
13
14  nBytes = read(socket_fd, buffer, 256);
15  buffer[nBytes] = 0;
16
17  printf("message from server: %s\n", buffer);
18
19  close(socket_fd);
20
21  return 0;
22}
23