Next Previous Contents

7. Debugging tips and programming information

7.1 Submitting useful problem reports

The best way to submit reports is to use the online pcmcia-cs forums or the bug tracker at SourceForge. That way, other people can see current problems (and fixes or workarounds, if available). Here are some things that should be included in all bug reports:

All the PCMCIA modules and the cardmgr daemon send status messages to the system log. These will usually end up somewhere like /var/log/messages or /var/log/daemon.log. These files should be the first place to look when tracking down a problem. When submitting a bug report, always include the relevant contents of these files. If you are having trouble finding your system messages, check /etc/syslog.conf to see how different classes of messages are handled.

Before submitting a bug report, please check to make sure that you are using an up-to-date copy of the driver package. While it is somewhat gratifying to read bug reports for things I've already fixed, it isn't a particularly constructive use of my time.

If you do not have web access, bug reports can be sent to me at dahinds@users.sourceforge.net. However, I prefer that bug reports be posted to the pcmcia-cs SourceForge site, so that they can be seen by others.

7.2 Interpreting kernel trap reports

If your problem involves a kernel fault, the register dump from the fault is only useful if you can translate the fault address, EIP, to something meaningful. Recent versions of klogd attempt to translate fault addresses based on the current kernel symbol map, but this may not work if the fault is in a module, or if the problem is severe enough that klogd cannot finish writing the fault information to the system log.

If a fault is in the main kernel, the fault address can be looked up in the System.map file. This may be installed in /System.map or /boot/System.map. If a fault is in a module, the nm command gives the same information, however, the fault address has to be adjusted based on the module's load address. Let's say that you have the following kernel fault: 

Unable to handle kernel NULL pointer dereference
current->tss.cr3 = 014c9000, %cr3 = 014c9000
*pde = 00000000
Oops: 0002
CPU:    0
EIP:    0010:[<c2026081>]
EFLAGS: 00010282

The fault address is 0xc2026081. Looking at System.map, we see that this is past the end of the kernel, i.e., is in a kernel module. To determine which module, check the output of ``ksyms -m | sort'': 

Address   Symbol                            Defined by
c200d000  (35k)                             [pcmcia_core]
c200d10c  register_ss_entry                 [pcmcia_core]
c200d230  unregister_ss_entry               [pcmcia_core]
          ...
c2026000  (9k)                              [3c574_cs]
c202a000  (4k)                              [serial_cs]

So, 0xc2026081 is in the 3c574_cs module, and is at an offset of 0x0081 from the start of the module. We cannot look up this offset in 3c574_cs.o yet: when the kernel loads a module, it inserts a header at the module load address, so the real start of the module is offset from the address shown in ksyms. The size of the header varies with kernel version: to find out the size for your kernel, check a module that exports symbols (like pcmcia_core above), and compare a symbol address with nm output for that symbol. In this example, register_ss_entry is loaded at an offset of 0xc200d10c - 0xc200d000 = 0x010c, while ``nm pcmcia_core.o'' shows the offset as 0x00c0, so the header size is 0x010c - 0x00c0 = 0x004c bytes.

Back to 3c574_cs, our fault offset is 0x0081, and subtracting the 0x004c header, the real module offset is 0x0035. Now looking at ``nm 3c574_cs.o | sort'', we see: 

0000002c d if_names
0000002c t tc574_attach
00000040 d mii_preamble_required
00000041 d dev_info

So, the fault is located in tc574_attach().

In this example, the fault did not cause a total system lockup, so ksyms could be executed after the fault happened. In other cases, you may have to infer the module load addresses indirectly. The same sequence of events will normally load modules in the same order and at the same addresses. If a fault happens when a certain card is inserted, get the ksyms output before inserting the card, or with a different card inserted. You can also manually load the card's driver modules with insmod and run ksyms before inserting the card.

For more background, see ``man insmod'', ``man ksyms'', and ``man klogd''. In the kernel source tree, Documentation/oops-tracing.txt is also relevant. Here are a few more kernel debugging hints:

7.3 Low level PCMCIA debugging aids

The PCMCIA modules contain a lot of conditionally-compiled debugging code. Most of this code is under control of the PCMCIA_DEBUG preprocessor define. If this is undefined, debugging code will not be compiled. If set to 0, the code is compiled but inactive. Larger numbers specify increasing levels of verbosity. Each module built with PCMCIA_DEBUG defined will have an integer parameter, pc_debug, that controls the verbosity of its output. This can be adjusted when the module is loaded, so output can be controlled on a per-module basis without recompiling.

Your default configuration for syslogd may discard kernel debugging messages. To ensure that they are recorded, edit /etc/syslog.conf to ensure that ``kern.debug'' messages are recorded somewhere. See ``man syslog.conf'' for details.

There are a few register-level debugging tools in the debug_tools/ subdirectory of the PCMCIA distribution. The dump_tcic and dump_i365 utilities generate register dumps for ISA-to-PCMCIA controllers. In 3.1.15 and later releases, dump_i365 is replaced by dump_exca, which is similar but also works for PCI-to-CardBus bridges. Also new in 3.1.15 for CardBus bridges is the dump_cardbus tool, which interprets the CardBus-specific registers. These are all most useful if you have access to a datasheet for the corresponding controller chip. The dump_cis utility (dump_tuples in pre-3.0.2 distributions) lists the contents of a card's CIS (Card Information Structure), and decodes most of the important bits. And the dump_cisreg utility displays a card's local configuration registers.

The memory_cs memory card driver is also sometimes useful for debugging problems with 16-bit PC Cards. It can be bound to any card, and does not interfere with other drivers. It can be used to directly access any card's attribute memory or common memory. Similarly for CardBus cards, the memory_cb driver can be bound to any 32-bit card, to give direct access to that card's address spaces. See the man pages for more information.

7.4 /proc/bus/pccard

On 2.2 and later kernels, the PCMCIA package will create a tree of status information under /proc/bus/pccard. Much of the information can only be interpreted using the data sheets for the PCMCIA host controller. Its contents may depend on how the drivers were configured, but may include all or some of the following:

/proc/bus/pccard/{irq,ioport,memory}

If present, these files contain resource allocation information to supplement the normal kernel resource tables. Recent versions of the PCMCIA system may obtain additional resource information from the Plug and Play BIOS if configured to do so.

/proc/bus/pccard/drivers

In recent releases, this lists all currently loaded PCMCIA client drivers. Unlike /proc/modules, it also lists drivers that may be statically linked into the kernel.

/proc/bus/pccard/*/info

For each socket, describes that socket's host controller and its capabilities.

/proc/bus/pccard/*/exca

This contains a dump of a controller's ``ExCA'' Intel i82365sl-compatible register set.

/proc/bus/pccard/*/{pci,cardbus}

For CardBus bridges, a dump of the bridge's PCI configuration space, and a dump of the bridge's CardBus configuration registers.

7.5 Writing Card Services drivers for new cards

The Linux PCMCIA Programmer's Guide is the best documentation for the client driver interface. The latest version is always available from projects.sourceforge.net in /pub/pcmcia-cs/doc, or on the web at http://pcmcia-cs.sourceforge.net.

For devices that are close relatives of normal ISA devices, you will probably be able to use parts of existing Linux drivers. In some cases, the biggest stumbling block will be modifying an existing driver so that it can handle adding and removing devices after boot time. Of the current drivers, the memory card driver is the only ``self-contained'' driver that does not depend on other parts of the Linux kernel to do most of the dirty work.

In many cases, the largest barrier to supporting a new card type is obtaining technical information from the manufacturer. It may be difficult to figure out who to ask, or to explain exactly what information is needed. However, with a few exceptions, it is very difficult if not impossible to implement a driver for a card without technical information from the manufacturer.

I have written a dummy driver with lots of comments that explains a lot of how a driver communicates with Card Services; you will find this in the PCMCIA source distribution in clients/dummy_cs.c.

7.6 Guidelines for PCMCIA client driver authors

I have decided that it is not really feasible for me to distribute all PCMCIA client drivers as part of the PCMCIA package. Each new driver makes the main package incrementally harder to maintain, and including a driver inevitably transfers some of the maintenance work from the driver author to me. Instead, I will decide on a case by case basis whether or not to include contributed drivers, based on user demand as well as maintainability. For drivers not included in the core package, I suggest that driver authors adopt the following scheme for packaging their drivers for distribution.

Driver files should be arranged in the same directory scheme used in the PCMCIA source distribution, so that the driver can be unpacked on top of a complete PCMCIA source tree. A driver should include source files (in ./modules/), a man page (in ./man/), and configuration files (in ./etc/). The top level directory should also include a README file.

The top-level directory should include a makefile, set up so that ``make -f ... all'' and ``make -f ... install'' compile the driver and install all appropriate files. If this makefile is given an extension of .mk, then it will automatically be invoked by the top-level Makefile for the all and install targets. Here is an example of how such a makefile could be constructed: 

# Sample Makefile for contributed client driver
FILES = sample_cs.mk README.sample_cs \
        modules/sample_cs.c modules/sample_cs.h \
        etc/sample.conf etc/sample etc/sample.opts \
        man/sample_cs.4
all:
        $(MAKE) -C modules MODULES=sample_cs.o
install:
        $(MAKE) -C modules install-modules MODULES=sample_cs.o
        $(MAKE) -C etc install-clients CLIENTS=sample
        $(MAKE) -C man install-man4 MAN4=sample_cs.4
dist:
        tar czvf sample_cs.tar.gz $(FILES)

This makefile uses install targets defined in 2.9.10 and later versions of the PCMCIA package. This makefile also includes a ``dist'' target for the convenience of the driver author. You would probably want to add a version number to the final package filename (for example, sample_cs-1.5.tar.gz). A complete distribution could look like: 

sample_cs.mk
README.sample_cs
modules/sample_cs.c
modules/sample_cs.h
etc/sample.conf
etc/sample
etc/sample.opts
man/sample_cs.4

With this arrangement, when the contributed driver is unpacked, it becomes essentially part of the PCMCIA source tree. It can make use of the PCMCIA header files, as well as the machinery for checking the user's system configuration, and automatic dependency checking, just like a ``normal'' client driver.

In this example, etc/sample and etc/sample.opts would be the new driver's configuration scripts (if needed), and etc/sample.conf would contain any additions to the PCMCIA card configuration file. Starting with the 3.1.6 release, cardmgr will automatically process any *.conf files installed in /etc/pcmcia, so installation of contributed drivers should no longer require hand editing configuration files.

I will accept client drivers prepared according to this specification and place them in the /pub/pcmcia-cs/contrib directory on projects.sourceforge.net. The README in this directory will describe how to unpack a contributed driver.

The client driver interface has not changed much over time, and has almost always preserved backwards compatibility. A client driver will not normally need to be updated for minor revisions in the main package. I will try to notify authors of contributed drivers of changes that require updates to their drivers.

7.7 Guidelines for Linux distribution maintainers

If your distribution has system configuration tools that you would like to be PCMCIA-aware, please use the *.opts files in /etc/pcmcia for your ``hooks.'' These files will not be modified if a user compiles and installs a new release of the PCMCIA package. If you modify the main configuration scripts, then a fresh install will silently overwrite your custom scripts and break the connection with your configuration tools. Contact me if you are not sure how to write an appropriate option script, or if you need additional capabilities.

It is helpful for users (and for me) if you can document how your distribution deviates from the PCMCIA package as described in this document. In particular, please document changes to the startup script and configuration scripts. If you send me the appropriate information, I will include it in the Notes about specific Linux distributions.

When building PCMCIA for distribution, consider including contributed drivers that are not part of the main PCMCIA package. For reasons of maintainability, I am trying to limit the core package size, by only adding new drivers if I think they are of particularly broad interest. Other drivers will be distributed separately, as described in the previous section. The split between integral and separate drivers is somewhat arbitrary and partly historical, and should not imply a difference in quality.

Next Previous Contents

Hosting by: Hurra Communications Ltd.
Generated: 2007-01-26 17:58:33