A generic bus/device design

This document is to describe a new interface on how all devices will be described and probed by the system. It will reqruie that the machine dependant code include a processor bus, but other than that, nothing else should be processor dependant, if it is, it's broken and needs to be fixed and/or improved.

Log of Changes
Resource System
Bus/Device System
Device API to Bus/Device
How to use the system
1. Announcing a Device Driver
2. Adding Bus code

Log of Changes

A list of design changes and the times they happened:

yymmddhh: Brief description of change
98032021: I have decided against a hierarchical layout in favor of a graph layout for the devices. This will provide MUCH better DMA support among other things. This means that if you need to transfer data from one device to another, you will tell the dma system, and it will set up the necessary mappings if possible to do the transfer for you. This will also easily allow a bt848 card to dump to a video card behind a pci-pci bridge.
98030517: Add struct bus as a new data structure to better sepearte the functions of generic bus code and bridge chip functions. struct bus will contain pointers to routines that need to be specific to each bridge chip. It will also contain a pointer to struct genbus for the generic bus routines.
98012416: Add handler to struct genbus to provide universal code for each bus implementation.
97122417: Add Resource API section, also mentions the use of NetBSD's bus_space for mem/io.
97121801: Add document of new union id to struct resource so that the same structure is used both internally and in static declarations. Add comment about flags in struct device. Add of TOC to the document, it's getting a bit large, and it's nice to know exactly what the document contains.
97121118: Minor mods, add a void *key to the allocate resource routine so when we disconnect a device providing resources, we can disconnect/notify the devices using the resource. Add name to the genbus structure. Modify registerresource to take a structresource to pass in the information instead of name/num pair.
97102400: Change the probe/attach/detach routins in struct gendevice into a single handler that will take an argument of what the bus code would like the device to do.
97101819: Prevent the resource module from know about the internals of the device module. Add key and address to struct device for an address bus. Add addresstoname to struct genbus to decode address. Remove mapresource from struct genbus as it was pointless.
97101801: Changed bus in struct gendevice to a char * and added a function to set up the intrmask genericly.
97092816: Created a new struct, struct genbus that contains the routines to manipulate bus resources. Moved the resource alloacation from struct gendevice to this new struct. Ability to add resources to a bus for other devices to use. A prime example is irqs and dma channels on the isa bus.
97092805: Creation of this document, and first annoncement

Resource System

There will be no hardcoded resources. Resources will be described in a driver by a string that will be decoded by the bus code. When the bus requires that a new type of resource be created, it will call the function newresource to obtain a resource id for it. The resource can by anything from memory, to scsi id's. There is a compainion function, claimresource, that is used to actually mark a resource region as allocated. This function is to only be called by the parent bus of the resource. Normal resource allocation goes through the parent bus of the device.

int newresource(char *name, int start, int len);

This function will declare a new resource with the name provided. It will have a range of valid values from start up to start + len. The value returned will the the resource id that is used in calls to claimresource.

int claimresource(struct resource res, void key);

This function is used to mark the resource in res as used. The possible return values are:

RES_SUCCESS: Allocation was successful.
RES_FAIL: Allocation failed.

This function is only to be called by the bus/device that created the resource with newresource. If it is not, then there will be no way to use the resource allocated. Make sure you use the member allocate_res that is part of struct genbus when allocating a bus resource for device usage.

struct resource

{
	union {
		char	*name;
		int	num;
	} id;
	int	start;
	int	len;
	int	flags;
	void	*data;
};

This structure defines the resource that are or are going to be allocate. This structure is passed to allocate_res in struct genbus of your parent bus. The start and len members define the begining and length of the resource. The flags member is a bit set that contains different options. The current list of options are:

RES_ACTIVE: Resource is currently allocated and in use.
RES_ONLYACTIVE: Resource is only used when device is active (i.e. opened).
RES_WAITFORFREE: When allocating, wait for the resource to be free.
RES_SHARED: Resource is shared
RES_ISNUM: If this flag, the data in the id field is actually the id number and not name. This is so that in static declarations, we can define the universal ascii name for it, then when the device gets touched by the bus/device code, it will be replaced with the proper code.

The data member is the value returned by allocate_res which is used to access the resource.

Resource API

The resource system allows many different types of resources to be registered for other device use. Resources are also very diverse things, so there is no easy way to declare a completely common interface without introducing extra overhead in the system. This means that the API for each resource will have to be declared ahead of time (if this is to be used by other publicly available device drivers).

A listing of the ascii names and the section for more information:

Mem/IO API

The two most commonly used resources are mem and io. Their ascii string representations will be "mem" and "io". To be compatible, we are going to use NetBSD's mem/io bus abstraction. Take a look at NetBSD's bus_space(9) manual page for more information.

Not Yet Defined

This is a section for those ascii names that will be defined, but haven't yet.

Bus/Device System

Devices can be both a bus and a device. There really is no different between the two as they use the same generic resources. The only difference is that a bus will provide resource mappings to subdevices which are able to allocate and use them.

The struct device will be allocated and destroyed by the bus code. It will be initalized in preperation before being passed to the device for use when probing and attaching. This will include setting info to the appropriate struct gendevice for this particular device.

struct device

{
	int	unit;
	union {
		char	*name;
		struct	gendevice *gdev;
	} un;
	struct	device *bus;
	int	flags;
	int	address;
	void	*key
	void	*data;
	int	num_res;
	struct	resource *resources;
};

This containes the basic information on a device. There will be one for each device/bus that is connected to the system. The memebers are:

unit

unit number for the device driver

un

either the ascii name of the proper driver or a pointer to struct gendevice, depending upon the flag DEV_ISGDEV, contains functions/data specific to each driver, this will be NULL if there is no driver currently assigned and this is an address bus

bus

pointer to struct device, parent bus that device is attached to

flags

current flags are:

DEV_ACTIVE: device is active
DEV_SUSPENDED: device is currently in the suspended state
DEV_PROBED: this device has already been probed, do not probe again

A friend of mine comments that the above should actually be a finite state machine. Right now they are a seperate set of flags. Comments on this would be greately apricated.

address

This contains the address of the device on it's parent bus. This can be converted to a string using addresstoname in struct genbus. This is only useful on an address bus.

key

Contains the key that identifies this device. This is only useful on an address bus.

data

device specific data

num_res

number of resources in member resources

resources

list of resources used by device (struct resource)

struct gendevice

{
	char	*name;
	char	*bus;
	int	(*handler)(struct device *dev, int event);
	intrmask_t	(*spllevel)();
};

This contains the information that should be driver specific. They contain function to probe, attach, and detach the drive. They also contain resource handling functions when the device is also a bus. The members are:

name: ASCII representation of the driver name (usually prepended to unit number for messages)
bus: ASCII representation of the bus that this driver belongs to. NULL means that the other routines support multiple bus types, and will be called for all bus types
handler: field various events that will be delivered to a device.
spllevel: function used to set the spl level this driver needs

int handler(struct device *dev, int event);

This function will handle different events that the device can receive. First there are a few flags that make up the event:

DVE_MANDATORY: If event & DVE_MANDATORY is true, then this even must be handled. If this event isn't implemented yet, it must return EINVAL. (Should this be DVE_OPTIONAL instead?)
DVE_EVENT: This really isn't a flag, it's the mask of bits that make up the event. Must be used before comparision of possible events which are listed below.

List of possible events:

DVE_PROBE: This will check to make sure that the device exists on the specific type of bus. If the bus type was provided in struct gendevice then the bus of struct device does not need to be checked. (Should this be allowed to probe other possible places? Should we add a flag exesive probe that will probe internal defaults/possibilities?)
DVE_ATTACH: Initalize internal data structures and the device for use.
DVE_DETACH_NORM: Normal detach. Clean up memory, deinitalize device, and deinitalize other parts of the kernel. A return value of 0 means that it was successful and the device is ready for removal. A return value of non-zero means that the driver was unable to detach the device, and that removal should be rejected.
DVE_DETACH_FORCE: Device is still in machine, but do everything possible to prepare this device for removal. This function should only fail (return non-zero result) if there is a major problem, such as a hardware jam.
DVE_DETACH_GONE: The device has already departed. Access to resources may fail. The driver should just clean up memory and other possible uses of the device. Return value will be ignored, and dev may be deallocated or reused after this fuction returns.
DVE_SUSPEND: Suspend this device. It should save the current state of the device and power it off. If this fails, there is no assurances that when a DVE_RESUME event is delivered that the device will be in the same state. The driver must internally handle this.
DVE_RESUME: Resume the device to previous state before DVE_SUSPEND was delievered. If this fails, it will deliver a DVE_GONE event to the device after the function returns.

struct bus

{
	union {
		char *name;
		struct	genbus *bus;
	} un;
	int	(*allocate_res)(struct device *dev, struct resource *res);
	void	(*deallocate_res)(struct device *dev, struct resource *res);
	int	(*mapresourcename)(struct device *dev, char *name);
	int	(*registerresource)(struct device *dev, struct resource *res);

This struct contains the functions that are used to manage the resources of the bus. They allow you to obtain resource identifiers for your own use, along with presenting/adding resources to the bus that other can use.

Right now these need to under go a design change. There should be a standard way to add a resource (simply passing struct resource and struct device to it), and that will do the right thing, just add it to the list (in the allocate upon open), or actually call the bus specific code to allocate the resource. There should be a function that will take a string, start, length, and a struct device which will allocate the struct resource and call above mentioned function.

allocate_res: allocate res for use by a device attached to this bus, see above for dev, res is the pointer to the struct resource you are allocating.
deallocate_res: deallocate a struct resource previously allocated with allocate_res
mapresourcename: returns the global resource id from a name that is bus specific
registerresource: adds a resource to the bus that a device provides (such as dma and irqs). You pass the ascii representation of the resource in the data field, and the resulting gloab resource id will be filled in.

struct genbus

{
	char	*name;
	int	(*handler)(struct device *dev, int event);
	char	*(*addresstoname)(int address);
		/* this needs work to inform bus of callbacks to allocate the
		   resource */
};

This structure contains the generic bus code. The handler should be called to be notified when events happen. I still need to define the events, but there shouldn't be much more than notification of an interrupt as the generic code should handle detecting device arrival/removal by processing the interrupt.

handler: handle the bus specific parts of the code such as finding out what devices are attached
addresstoname: returns a printable address of this device. This should take another argument to tell what is exactly wanted in the return value.

int (handler)(struct device dev, int event);

This is the routine that will handle all the bus specific probing of devices attached to the bus. The bus code will use this routine to obtain the addresses of devices on the bus (if there are any). More will come later.

int allocate_res(struct device dev, struct resource res);

This will allocate res for use by dev. The data memeber of res will be filled in with the appropriate data to access the device. This function's behavior is not defined if RES_ACTIVE in the resource's flags is set. The function will set RES_ACTIVE upon successful completion. The parameter res will not be changed unless the alloaction was successful. A return value of 0 means success and 1 upon failure.

void deallocate_res(struct device dev, struct resource res);

This function will deallocate a resource when it is no longer needed. The behavior is not defined if the RES_ACTIVE flag isn't set. The RES_ACTIVE flag will be cleared upon return of this device.

int mapresourcename(struct device dev, char name);

This function is used to map a resource name to a global resource id. It is probably best to provide an enum of all possible resources on the bus side, and use a char *resnames[] to provide the mapping. This will allow new devices to register resources that the whole bus can use without having to modify the bus specific code.

int registerresource(struct device dev, char name, int num);

This will register a resource named name for use by other devices on the same bus. We still need to provide a way to do callbacks to the device to init resource mappings, but that isn't hard to extend. There should also be a way for the device providing resources to detach and all other devices that use this resource be removed also. This currently isn't implemented, but probably can be done though detaching devices that require use of the resource, and making sure all devices that have the resource only when active are closed. Once these conditions are met, then the device can be detached. This could be provided by replacing the detach stub fuction with a bus provided one.

Device API to Bus/Device

There are a number of things that the device will want be be able to communicate back to the Bus/Device code that is asynchronous to the normal events that the Bus/Device code delievers to the device. This will usually be things like, I'm going to be detached now. Sending events from the device to the bus/code doesn't have that many uses, but will be included from completeness.

One really important part is to have the device act as a bridge to another bus type. When a device is actually a bridge to another bus, it will have to be able to field the requests for resources, and if neccessary forward them to the parent bus to obtain machine mappings. There will be some resources that the Bus/Device code provides that makes handling resources much simpler, such as extent mapping, to automaticly prevent resource collissions.

int createextent(struct device *dev, struct extent **ptr, char *name, int flags);

This will create a new extent associated with dev. This association isn't current tracked, but this could be useful in the future (such as garbage collection). If the routine fails, the data pointed to by ptr is undefined.

Once you have successfully allocated an extent, you can use the functions, extent_alloc_subregion, extent_alloc_region and extent_free to manipulate the extent. For more information on usage of these functions, see the NetBSD manpages.

int destroyextent(struct device *dev, sturct extent **ptr);

This will deallocate the extent that was allocated with createextent. If the routine fails, the data pointed to by ptr will remain unchanged.

How to use the system

At the base of the whole system will be what we call the processor bus. This bus is the parent of ALL hardware that exists on the system. Also, any hardware access will actually have to happen at this level, hardware on other busses won't be able to see these read/writes unless there is a bridge chip that will convert the processor bus signals to that other bus. In most cases the only types of accesses that are allowed will be memory and io port accesses, but there are a few other resources that buses provide such as interrupts and dma. These other resources are controled through devices on the bus, and as such, are required to be devices also.

A bus is not required to have memory and io port resources to be managed by this system. Things such as the SCSI bus do not have memory writes support, but can use the bus/device code to be able to both mange SCSI id's and to prove a way to map a driver to a device without complex code of their own.

In the future I would also like to move the cdev and bdev along with other internal systems over to this system. This will allow you to be able to use one utility to view almost any aspect of the system.

Announcing a Device Driver

Before the bus/device code is able to use a device driver, it must be told about it. The code should look like:

static struct gendevice devicename {
	/* declare the data here */
};
DEVICEDRIVER(devicename);

This will simply inform the bus/device code that the driver has been loaded and is ready to serve. It will be implemented as a wrapper around DECLARE_MODULE that use a generic device driver handler that will handle unload events appropriately.

Adding Bus Code

To add a new type of bus to the pool of available bus types, you would code something like:

static struct genbus busname {
	/* declare the data here */
};
BUSCODE(busname);

This will then create a new module bus_busname, which will refere to the data in the struct. It will allow devices that are bridges, to grab a pointer to the genbus struct.

There is one bus that is always required before any devices can be probed. That is the nexus bus. This is the code that defines how the processor bus is split to support the devices (and possibly other busses) that hang of it. In some cases, the processor bus IS another bus (such as the i386 and the isa bus). They should be implemented by creating a dummy device that acts as a bridge to the isa bus.

A generic bus/device design

Table of Contents

int newresource(char *name, int start, int len);

int claimresource(struct resource *res, void *key);

int handler(struct device *dev, int event);

int (*handler)(struct device *dev, int event);

int allocate_res(struct device *dev, struct resource *res);

void deallocate_res(struct device *dev, struct resource *res);

int mapresourcename(struct device *dev, char *name);

int registerresource(struct device *dev, char *name, int num);