Creating a System Call in Fedora 9 kernel 2.6.26 (i686)

The procedure to create a system call varies in different versions of Linux. I will provide a simple step by step guide on adding a system call to the Fedora 9 i686 kernel.

Getting Started

The system calls we are implementing are actually compiled into the Linux kernel, so you must have the Linux source tree installed on your system. Furthermore, compiling the kernel can take a good amount of time (depending on the system and kernel options to it can take hours), so give yourself time.

Notes

When folders are described with source/ at the start of the path, it represents the highest level of the Linux source tree. You should have folders such as arch, configs, include, and init in this folder.

Creating the System Call Source

We will be creating a simple hello world system call. Create a file mysyscall.c and fill it with the following code.

#include <linux/kernel.h>

#include <linux/linkage.h>

asmlinkage long sys_mysyscall(int i){

printk(KERN_DEBUG "Hello World! The number was %d\n", i);

}

Adding the System Call

1. Firstly, navigate to the root directory of your Linux source. Create a folder source/mysyscall. Next, inside of your newly created folder, add a source file mysyscall.c that contains all of your system call source code.

2. Next, create a make file within this directory. This make file only needs to contain one line that says what to compile.

Create source/mysyscall/Makefile (keep the capital ‘M’)

obj-y := mysyscall.o

3. Next we need to modify the kernel’s make file. This is located in the root folder of your kernel’s source. Find the line that adds a bunch of folders to the core-y variable and add the folder name that the system call source is in. (mysyscall in this example)

In source/Makefile, the line to change looks something like this:

core-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/

It should be changed to something like this:

core-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/ mysyscall/

4. Now, navigate to the source/arch/x86/kernel folder. You will need to add your system call to the system call table. This is done in the file syscall_table_32.S. At the end of the file you add an entry for the new system call. Make sure to prefix your entry with “sys_”.

It should look something like this:

.long sys_mysyscall

You should notice the numbers commented out to the right of some of the entries. They tell you what number the system call is. Keep track of which number your new system call is.

5. Next navigate to the source/include/asm-x86 folder and edit unistd_32.h. Add a #define line for the new system call. Use the number that you noted in the last step.

You should add a line similar to this:

#define __NR_mysyscall 327

6. You are now ready to recompile your kernel.

Using the System Call

1. Next to use the system call we need to create a header file that exposes the new call. Create a file, mysyscall.h which will be imported into any program that wants to use the new system call.

This header file should contain the following:

#include "unistd.h"

#define __NR_mysyscall 327

long mysyscall(int i)

{

return syscall(__NR_mysyscall, i);

}

2. Write a program that includes your newly created header file, and then call the mysyscall (int i) function you have implemented.

Adding a Variable Shared Between a System Call and a Kernel Module

1. Using External Variables

In order to use a variable that has been declared in another file, the keyword extern must be used.

#include <linux/kernel.h>

extern int myVar;

int MyModuleFunction()

{

/* Has access to myVar which is defined in a different kernel file*/

}

2. Making Variables Externally Visible

However, variables are not visible to other files by default. For a variable declared in a system call to be visible to a kernel module it must be exported. To export a variable you must use the macro EXPORT_SYMBOL. You can export functions as well using this same macro.

#include <linux/kernel.h>

int myVar;

EXPORT_SYMBOL(myVar);

static int MySystemCall()

{

/* Has access to myVar */

}