Writing epics drivers (and Device Support)

Writing EPICS Drivers

• (and Device Support)

Contents

• Introduction

• A simple example driver

• Tips and tricks

• Example device support

• Multi-threading

• I/O Intr

Introduction

Required knowledge about EPICS

• Records and fields

• Standard records (ai, ao, mbbi, stringout, …)
• Probably specialized records (motor, …)

• Record processing

• Scan methods (periodic, I/O Intr, …)
• Links that cause processing (PP links, CP links, forward links)

• Channel Access clients (medm, caget, …)

• These are topics of a basic EPICS training

• See also: IOC Application Developper's Guide.

Required knowledge about programming

• C

• Variable flavours (global, local, static, volatile)
• struct, union, array [], typedef, enum
• Memory allocation (malloc, calloc) and pointers
• Pointers to functions
• Macros (#define) and conditional compilation (#ifdef)

• Data structures

• Integer representations (2's complement, hex, byte order)
• Bit fields (masking, shifting, …)
• Linked lists

Important knowledge about hardware I/O

• I/O registers

• Side effects of reading and writing
• Fifo registers

• Busses

• VME address spaces A16, A24, A32
• Memory mapped access
• Out-of-order execution / pipelining

• Interrupts

• Interrupt levels and vectors
• Interrupt handlers

Driver and Device support

What is an EPICS driver?

• Software interface between EPICS records and hardware (or 3rd party software)

• Usually split in 2 parts for better maintainability

• Device support
• Interfaces to records
• Does not care about hardware
• Files: devXXX.c, XXX.dbd
• Driver
• Does low-level hardware (register) access
• Does not care about records
• Files: drvXXX.c, drvXXX.h, (sometimes XXX.dbd)

Layers: Record - Device Support – Driver - Hardware

Splitting functionality: device support vs. driver

• Device support is the "glue" between record and driver

• Parses INP or OUT link
• Reads / writes / initializes record fields
• Calls driver API functions

• Driver does the real work

• Initializes the hardware
• Creates work thread if necessary
• Register access
• Interrupt handling
• Resource handling (semaphores, etc)

A simple example driver

In this chapter

• Example hardware "myDac"

• Interface (API) in drvMyDac.h

• Implementation in drvMyDac.c

• Configuration function
• open() function
• I/O funtions
• Report function

• Register to EPICS with drvMyDac.dbd

• Registering variables
• Registering functions

Example: 8 channel DAC

• The card is a VME board in A16 address space

• One DAC "card" has 8 "signals".

• Each signal is 16 bit (0x0000 = -10 V, 0xFFFF = +10 V)

• The 8 signals are registers at offsets 0x10, 0x12, 0x14, 0x16

• It is possible to read back the current register setting.

• The card does not use interrupts.

• The card has an identifier at offset 0x00

• String: "MYDAC"+nullbyte

How to start?

• Decide what the driver should be able to do

• Define one function for each functionality of the hardware

• Don't think about records at this time
• Think of "cards" or "boards", "signals" and functionality

• Model a "card" as a structure

• It contains hardware register address, internal states, etc…
• Define a function which returns a pointer to this structure (like open() for a file)
• Other functions take this "handle" as an argument

• Define a configuration function to initialize one "card"

What to put into the header file

• This file defines the interface to the driver

• All public (API) driver functions
• Maybe error codes if used as return values
• Typedefs for used function parameters

• This file does not contain implementation details

• No card structure details
• No internal functions (e.g. interrupt handler)
• No macros etc. used only internally in driver
• No register layout

Example drvMyDac.h file

What to avoid

• Do not use void* for driver handle.

• This allows the compiler to warn about wrong arguments.

• Do not define the card handle structure in the header file.

• This is an implementation detail and not of interest to the user.

• Do not list internal functions in the header file.

• Interrupt handlers etc. are implementation details.

• Do not define the register layout in the header file.

• You don't want anyone else but the driver to access registers.

Implementing the driver (part 1)

• Define "card" structure.

• Contains at least register base address.
• May contain thread ID, semaphores, etc.
• Build a container of "cards" (linked list).

• Define macros for register access.

• Mark all registers volatile.
• This prevents the compiler from "optimizing" and reordering register access.

• Avoid global variables whenever possible.

• At least use static for any private global variable.
• Prefix every non-static global variable with driver name.

Includes, card structure and macros in devMyDac.c

What to avoid

• Do not use a structure to map registers.

• Structures are compiler dependent.
• Compiler may insert pad bytes to align elements.
• Compiler may reorder elements to "optimize" access.
• Use macros to calculate address offset.

• Do not use compiler dependent data types for registers.

• Not all compilers use 2 bytes for unsigned short.
• Use compiler independent types like epicsUInt16 instead.

• Do not use global variables to store any card details.

• You may have more than one card. Make a list of structures.

Implementing the driver (part 2)

• Configuration function

• Configure one and only one card per function call.
• Give each card an individual id (number or string).
• Do not use auto-numbering of cards.
• This avoids problems when you have to remove a card (temporarily)

• open() function

• Return a card handle from a card id

• Input/Output functions

• Report function

• (Interrupt handler)

What to do in the configuration function

• Check parameters for sanity.

• Calculate local (memory mapped) address of registers.

• Check if the hardware is installed.

• Create "card" structure.

• Allocate memory.
• Fill in values.
• Put into linked list.

• Give good error messages if anything goes wrong.

• Don't forget driver name and parameters in error message.
• Write error messages to stderr, not to stdout.

Example configuration function in drvMyDac.c

Configuration function continued

Configuration function continued

Configuration function finished

What to avoid

• Do not use other parameter types than int or char*.

• double does not work on vxWorks shell on PPC architecture
• Other types are not supported by iocsh.

• Do not use too many parameters.

• vxWorks supports only 10 parameters for shell functions.

• Do not crash if card is absent.

• Check card before use.

• Do not give meaningless messages like "driver init failed".

• The user needs information. What failed where and why?
• Provide driver/function name, failing parameter and error reason.

Hardware registers

• A hardware register is not just a variable!

• Write or read access may have side effects on the hardware.
• Reading may get a different value than the last written one.
• FIFO registers provide different values every time they are read, giving sequential access to an array through a scalar.
• Reading or writing in pieces (e.g. low and high 16bit word to a 32bit register) may be invalid, may have unexpected effects, or may require a certain order.

• Always use volatile to access registers

• This tells the compiler not to try "optimization" on hardware registers.

What to do in the API functions

• Open

• Check card id (number or string)
• Find card in list of configured cards.
• Return pointer to card structure (handle) or NULL.

• I/O functions

• Check handle for validity.
• Read or write registers.
• Return error code on failure or 0 on success.
• No need to print error messages here (device support should do that)
• Put in switchable debug messages.

Example API functions

Example API functions continued

What to avoid

• Do not translate card id to structure in each API function call.

• Get card handle once and use it in all other API functions.

• Do not use card when configuration failed.

• When configuration fails return NULL from open() call.
• Check for NULL in all other functions.

• Do not assume anything about records.

• Do not use function names like write_ao().
• The driver only cares about the features of the hardware.
• Records are the business of device support.

Reporting hardware and driver status

• Write a report function.

• Print driver and register information to stdout.
• Provide multiple levels of detail.
• In lowest level (0) print only one line per card.
• In higher levels print more details about configuration, registers, etc.

• Register report function to EPICS.

• Create a driver support structure.
• Fill in a pointer to report function.
• Export driver support structure to EPICS.

Example report function

Report function call

• The dbior shell function calls driver report functions.

• Example:

• dbior Driver: myDac card 1 @0xfbff1000
• dbior "myDac",1 Driver: myDac card 1 @0xfbff1000 DAC 0 @0xfbff1010 = 0x0000 = -10.0000 V DAC 1 @0xfbff1012 = 0x0000 = -10.0000 V DAC 2 @0xfbff1014 = 0x0000 = -10.0000 V DAC 3 @0xfbff1016 = 0x0000 = -10.0000 V DAC 4 @0xfbff1018 = 0x0000 = -10.0000 V DAC 5 @0xfbff101a = 0x0000 = -10.0000 V DAC 6 @0xfbff101c = 0x0000 = -10.0000 V DAC 7 @0xfbff101e = 0x0000 = -10.0000 V

Exporting variables and functions to EPICS

• VxWorks shell can access C functions and variables directly

• Other architectures must run iocsh

• Shell must know about functions, variables, driver support
• Export variables and driver support from C
• epicsExportAddress (int, myDacDebug);
• epicsExportAddress (drvet, myDac);
• Much more complicated for functions
• Write parameter description
• Write wrapper function
• Write registrar function
• Very ugly and error-prone

Wrapper and registrar for shell functions

Importing variables and functions to EPICS

• Make exported C variables and functions known to EPICS

• Write MyDac.dbd file with references to exported entities

• Driver support structure
• driver(myDac)
• Variables
• variable(myDacDebug, int)
• Registrar
• registrar(myDacRegister)
• Coming soon: Device support
• device(…)

Tips and Tricks

• From Dirk's Code Kitchen

A bit more safety / paranoia

• Even if card != NULL, it may be invalid

• E.g. user calls myDacGet() with cardnumber instead of handle.

• Accessing wrong hardware address is bad.

• A cheap way to check the card handle is a "magic number"

• Add magic number to card structure.
• Insert magic number when card is configured.
• Check magic number in every API function.

• A good magic number is CRC checksum of driver name

• echo -n myDac | cksum 2191717791 5

Example usage of magic numbers

Simulation Mode

• EPICS does not support VME on Unix (or Windows)

• devRegisterAddress() fails on softioc
• devReadProbe() fails on softioc

• Implement "simulation mode" on Unix for driver test

• Work on allocated memory instead of registers #ifdef UNIX /* UNIX has no VME. Use a simulation for tests */ #include #define devRegisterAddress(name, type, addr, size, pbase) \ ((*(pbase)=memalign (0x10000, size))? \ strncpy ((void*)*(pbase), "MYDAC", size), 0 : S_dev_noMemory) #define devReadProbe (size, ptr, buff) \ (memcpy (buff, (void*)ptr, size), 0) #endif

Example (synchronous) device support

In this chapter

• Device support structure

• Record initialization

• Parse INP/OUT link
• Connect to driver
• Fill record private data structure
• Initialize record from hardware

• Read or write (record processing)

• Transfer data between record and driver

• Linear scaling

How to start?

• Decide what record types to support

• Write one device support for each record type
• Find out what record expects in device support
• See record reference manual / record source code
• Unfortunately no header file defines the device support

• Choose synchronous or asynchronous support

• If driver never blocks: synchronous
• e.g. register access
• If driver may block or driver has callback functions: asynchronous
• e.g. field bus access

• Maybe "I/O Intr" support if possible

The device support structure

• One device support structure for each supported record type.

• Contains pointers to device support functions.

• Depends on record type.

• Usually: struct { long number; /* of functions below */ DEVSUPFUN report; DEVSUPFUN init; DEVSUPFUN init_record; DEVSUPFUN get_ioint_info; DEVSUPFUN read_or_write; }
• Additional functions for some record types (see record reference manual)

Typical device support functions

• report (can be NULL)

• Report function similar to driver report function, but per record type

• init (can be NULL)

• Initialization of device support per record type

• init_record

• Initialization of device support per record

• get_ioint_info (can be NULL)

• For records scanned in "I/O Intr" mode

• read or write (depending on record type)

• Actual I/O during record processing

Example: synchronous ao support for myDac

• Device support structure of ao:

• Additional function special_linconv.
• struct { long number; /* must be 6 */ DEVSUPFUN report; /* can be NULL */ DEVSUPFUN init; /* can be NULL */ DEVSUPFUN init_record; DEVSUPFUN get_ioint_info; /* can be NULL */ DEVSUPFUN write; DEVSUPFUN special_linconv; }

• Implement 3 functions

• long myDacInitRecordAo(aoRecord *record)
• long myDacWriteAo(aoRecord *record)
• long myDacSpecialLinconvAo(aoRecord *record, int after)

• Store record private data in record->dpvt.

Analog out device support (includes, private data)

Analog out device support (device support structure)

Importing device support to EPICS

• Make exported device supports known to EPICS

• Add one line for each supported record type to MyDac.dbd: device (ao, VME_IO, myDacAo, "MyDac")

Analog out device support (init_record part 1)

Analog out device support (init_record part 2)

Analog out device support (write)

Analog out device support (special_linconv)

• Only required for ao and ai records

• ao record calculates RVAL = (VAL - EOFF) / ESLO ai records calculates VAL = RVAL * ESLO + EOFF

• User provides:

• EGUL (should map to minimal raw value, e.g. 0x0000)
• EGUF (should map to maximal raw value, e.g. 0xFFFF)

I/O Intr

What is I/O Intr?

• It is a record scanning mode.

• The record is scanned whenever the driver has new data.

• Its an easy way to implement fast (>10 Hz) or irregular scanning.

• Can be triggered from driver thread or from interrupt level.

How to set up I/O Intr scanning?

• Create one IOSCANPVT structure (from dbScan.h) for each source of “new data events” (e.g. interrupt) of the driver.

• Implement get_ioint_info() in device support

• The record calls this function whenever SCAN is set to “I/O Intr”.
• The function should get the IOSCANPVT from the driver.
• It calls scanIoInit(IOSCANPVT *) to register with the “new data event”

• The driver calls scanIoRequest(IOSCANPVT *) whenever it has new data.

• The record processes and reads the value.

Differences to normal scanning

• Normally the record asks the driver to start I/O.

• Here the driver does I/O first, then processes the record.

• The driver should store the data where the record can find it.

• Many record can be triggered from one event source.

• E.g. 32 bi records bound to an digital I/O card.
• This may increase performance when hardware access is costly.

Asynchronous Device Support

About asynchronous support

• When to use?

• If hardware access is slow or may block.
• Examples: Fields busses (serial, GPIB, …)

• What is the problem?

• Record processing must not block.

• Solution: Asynchronous device support

• Driver starts a “work thread” that can block.
• Device support starts driver action with non-blocking function.
• Driver calls back when I/O is complete.

Asynchronous read or write function in detail

• The read or write function calls driver to start I/O.

• Then it sets the PACT (processing active) field and returns.

• The record now knows that I/O is still in progress and pauses.

• The IOC continues with other records.

• Driver thread requests to process record again when ready.

• Read or write function is called a second time with PACT=1.

• The function transfers values from driver to record and returns.

• Record processing completes (forward link, monitors, etc).

Example asynchronous ai read function

Asynchronous finish function

Multi-threading issues

Threads used in EPICS

• Many parts of the EPICS software work in parallel. (In vxWorks, threads are called tasks.)

• E.g. each SCAN type runs in a separate thread
• High priority thread for ".1 second" scanning
• Low priority thread for "10 second" scanning
• Lowest priority for "Passive" scanning as the result of a caput.
• Additional threads for callbacks, timeouts, channel access, …

• Many threads may execute the same function at the same time

• E.g. two records with the same driver and different scan rates.

• The CPU can switch from one thread to another at any time.

The re-entrancy problem

• Functions must be re-entrant.

• Bad example: char* numToString (int number) { static char buffer[20]; sprintf (buffer, "%d", number); return buffer; }

• What's bad?

• Thread 1 calls numToString(12345).
• Some time after sprintf() the CPU switches to thread 2.
• Thread 2 calls numToString(42).
• Some time after sprintf() the CPU switches to thread 1.
• Thread 1 uses the function result and reads "42"

Making code re-entrant

• Never return a pointer to static memory.

• Never call such a function.

• Nobody would do that? System functions that do:

• char *ether_ntoa (const struct ether_addr *addr)
• char *asctime (const struct tm *tm)
• struct tm *localtime (const time_t *timep)
• char *strerror (int errnum)

• Use functions where the caller provides the buffer

• char *ether_ntoa_r (const struct ether_addr *addr, char *buf)
• char *asctime_r (const struct tm *tm, char *buf)
• struct tm *localtime_r (const time_t *timep, struct tm *result)
• int strerror_r (int errnum, char *buf, size_t n)

Non-atomic operations on global resources

• Non-atomic operations may be interrupted by an other thread.

• If the other thread accesses the same global resource it may get inconsistent data.

• Example:

What are global resources?

• Global variables

• Static variables

• Heap objects

• Hardware registers

• Files

• Directories

• Sockets

• Anything for that you have only a pointer or handle

What are non-atomic operations?

• read-modify-write:

• if (p == NULL) p = …
• if (! file_exists(filename)) { fopen (filename, "w"); … }
• flags |= 1;
• reg &= mask;
• counter++;

• sequential read/write:

• strcpy (globalstring, s);
• globalstruct.a = a; globalstruct.b = b;
• addressregister = adr; val = valueregister;
• element->next = previous->next; previous->next = element;

What is thread-safe?

• Local variables

• Everything on the stack is thread specific.

• errno

• Even though it looks like a global variable, it is thread specific.

• Single threaded context

• startup script
• functions called from iocInit
• driver and record initialization

• Resources used only by one thread

• Resources used only in interrupt handler

How to make non-atomic operations safe?

• Disable interrupts.

• Only interrupts can cause unexpected thread switch.
• This is very brute.
• Do this only for VERY SHORT times.

• Use mutual exclusion semaphores.

• Use operating system independent wrapper (epicsMutex).
• Be careful to prevent deadlocks: Two different threads must never take two different semaphores in reverse order.
• Lock resources as short as possible.