Linux PCI Driver Model

PCI Understanding:

• The Peripheral Component Interconnect Bus (PCI) today is present in a wide variety of microcomputers ranging from Intel-based PC architectures to DEC-Alpha-based work-stations.
• The CPU communicates with the PCI subsystem using a special chipset known as PCI-Bridge.
• PCI-Bridge is an intelligent controller, that handles all necessary tasks to transfer data from or to the CPU or the Memory Subsystem.
• On PCI the addresses and data are transferred as separate chunks over the bus because all bus lines can be used either as address or as data-lines.
• Each bus device has its own 265byte space of Memory for configuration  purposes that can be accessed through the CONFIG_ADDRESS and the CONFIG_DATA registers.
• All devices that are known to Linux you will see at /proc/pci.
• Device resources (I/O addresses, IRQ lines) automatically assigned at boot time, either by the BIOS or by Linux itself (if configured). 
• To identify a certain device while driver writing you will at least have to know the vendor-id and the device-id that is statically stored in the device configuration block.
• Driver writers normally need to know only the base address of the device and the IRQ line that the device is using.
• PCI device configuration information is Little-Endian. Remember that in your drivers. 
• lspci enumerate all the devices.

          00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)

          00:19.0 Ethernet controller: Intel Corporation 82579V Gigabit Network Connection (rev 05)

          |     |   |

          |     |   |_Function Number

          |     |_PCI Device Number

          |_PCI Bus Number

• lspci -tv

          -[0000:00]-+-00.0  Intel Corporation 2nd Generation Core Processor Family DRAM Controller

                          +-01.0-[01]--+-00.0  nVidia Corporation GF108 [GeForce GT 430]

                          |            \-00.1  nVidia Corporation GF108 High Definition Audio Controller

             |        |       |        |   

             |        |       |        |_ PCI Bus-1

             |        |       |_ PCI Bridge

             |        |_ PCI Bus-0

             |_ PCI Domain

• Device configuration can be displayed with lspci -x
• Standard information found in PCI configurations:
• Offset 0: Vendor Id
• Offset 2: Device Id
• Offset 10: Class Id (network, display, bridge...)
• Offsets 16 to 39: Base Address Registers (BAR) 0 to 5
• Offset 44: Subvendor Id
• Offset 46: Subdevice Id
• Offsets 64 and up: up to the device manufacturer
• Kernel sources: these offsets are defined in include/linux/pci_regs.h

• PCI Device Initialization steps:
• Enable the device.
• Request I/O port and I/O memory resources.
• Set the DMA Mask for both coherent and streaming DMA.
• Allocate and Initialize shared coherent data.
• Initialize device Registers.
• Register IRQ handler.
• Register to other subsystems (network, video ..)
• Enable DMA or Processing Engines.
• Before touching any device registers, the driver should first execute pci_enable_device(). This will:
• Wake up the device if it was in suspended state.
• Allocate I/O and memory regions of the device, If not already done by the BIOS.
• Assign IRQ to the device, If not already done by the BIOS 
• Enable DMA by calling pci_set_master(). This will:
• Enable DMA by setting the bus master bit in the PCI_COMMAND register. The device will then be able to act as a master on the address bus.
• Fix the latency timer value if it's set to something bogus by the BIOS.
• This enables the PCI_COMMAND bit for Memory ­ Write Invalidate.
• This also ensures that the cache line size register is set correctly.
• Accessing configuration registers:
• Reading:
• int pci_read_config_byte(struct pci_dev *dev, int where, u8 *val);
• int pci_read_config_word(struct pci_dev *dev, int where, u16 *val);
• int pci_read_config_dword(struct pci_dev *dev, int where, u32 *val);
• Writing:
• int pci_write_config_byte(struct pci_dev *dev,  int where, u8 val);
• int pci_write_config_word(struct pci_dev *dev,  int where, u16 val);
• int pci_write_config_dword(struct pci_dev *dev,  int where, u32 val);
• Accessing I/O registers and memory:
• Each PCI device can have up to 6 I/O or memory regions, described in BAR0 to BAR5.
• Access the base address of the I/O region:
• long iobase = pci_resource_start (pdev, bar);
• Access the I/O region size:
• long iosize = pci_resource_len (pdev, bar);
• Reserve the I/O region:
• pci_request_region(pdev, bar, “my driver”);
• Use pci_dma_set_mask() to declare any device with more (or less) than 32­bit bus master capability
• In particular, must be done by drivers for PCI­X and PCIe compliant devices, which use 64 bit DMA.
• If the device can directly address "consistent memory" in System RAM above 4G physical address, register this by calling pci_set_consistent_dma_mask().
• You can allocate your cache consistent buffers if you plan to use such buffers. 
• If needed by the device Set some “capability” fields and do some vendor specific initialization or reset Example: clear pending interrupts.
• Register interrupt handlers:
• Need to call request_irq() with the IRQF_SHARED flag, because all PCI IRQ lines can be shared.
• Registration also enables interrupts, so at this point
• Make sure that the device is fully initialized and ready to service interrupts.
• Make sure that the device doesn't have any pending interrupt before calling request_irq(). 
• Where you actually call request_irq() can actually depend on the type of device and the subsystem it could be part of (network, video, storage...). 
• Your driver will then have to register to this subsystem.

• PCI Device Shutdown: 
• Disable the generation of new interrupts. If you don't, the system will get spurious interrupts, and will eventually disable the IRQ line. Bad for other devices on this line!
• Release the IRQ.
• Stop all DMA activity. Needed to be done after IRQs are disabled (could start new DMAs)
• Release DMA buffers: streaming first and then consistent ones.
• Unregister from other subsystems
• Unmap I/O memory and ports with io_unmap().
• Disable the device with pci_disable_device().
• Unregister I/O memory and ports. If you don't, you won't be able to reload the driver.