I've written about AUXV previously, focusing on one of its most interesting applications -- its role in helping find linux-gate.so.1.

If you're starting your program, you can get the dynamic loader to echo out the AUXV fields with the environment variable LD_SHOW_AUXV, but if your process has started you'll need to pull the values out of /proc/pid/auxv directly.

This is pretty internal stuff for the dynamic loader and is probably only useful if you're writing a debugger or doing some other low-level tricks (such as debugging!). However, should you need to, here is some sample code which does just that. Hopefully it will save someone else some time!

posted at: Fri, 03 Oct 2008 16:08 | in /linux | permalink | add comment (0 others)

Here are some slides and examples I used for a kernel course I developed (some time ago now).

The course was aimed at C developers who wanted an introduction to both general UNIX-style user-space and Linux kernel development with a focus on embedded systems issues. The course is aimed at two 8-hour days, and is pretty packed in even then.

The first day is user-space development and kernel building, focusing on things like make, autotools, advanced gcc, getting cross-compilers working, configuring the kernel and building. The second day we get into kernel internals; building up a kernel module to produce some simple proc nodes, take data, crash and debug, etc, look at internals like concurrency and the driver model, and focus on USB quite a bit.

Here is a tarball of the entire thing, including the examples.

Hopefully these can help out anyone tackling the design of such a course.

• Day 1
  • Linux philosophy, people and processes
  • UNIX build environment
    • Makefiles, Autotools
  • Executable and Linker Format
  • Advanced GCC
  • Linker, Dynamic Linker, Shared Libraries
  • Cross compilation toolchain
    • Building toolchains on Debian
  • Kernel development process
  • Source control, patches, GIT, Quilt
  • Configuring and building a kernel
  • Cross compiling kernel
  • initrd, initramfs, init tools

• Day 2
  • Kernel source layout
  • Kbuild, Kconfig
  • Syscalls (x86-32)
  • Kernel view of userspace
  • Virtual Memory
  • Getting data in/out kernel
  • Crashing, Oops, backtraces
  • Various tools
    • systemtap, kprobes
    • perfmon, oprofile
    • valgrind
  • Basic constructs
    • Concurrency, locks
    • Interrupts, sleeping
    • Atomic access
    • Memory allocation
  • VFS layer
  • /proc
  • Char and block devices
  • Driver model
    • sysfs, kobjects
    • buses, devices
  • USB Overview
    • Protocol overview
    • Registering drivers
    • uevents
  • Useful development paradigms
  • Unit testing

posted at: Mon, 29 Sep 2008 21:27 | in /linux | permalink | add comment (1 others)

A recent post reminded me of a problem I once had; determine the glibc version and its support for various things.

There's actually a little known but useful confstr call defined for just this sort of thing. Here's a minimal example:

#include <stdio.h>
#include <unistd.h>
#include <alloca.h>
#include <string.h>

int main (void)
{
        size_t n = confstr (_CS_GNU_LIBC_VERSION, NULL, 0);
        if (n > 0)
        {
                char *buf = alloca (n);
                confstr (_CS_GNU_LIBC_VERSION, buf, n);
                printf("%s\n", buf);
        }
        return 0;
}

man confstr has all the details. If you don't need it in your program, you can also just run /lib/libc/so.6 and it will print out it's version info, e.g.

$ /lib/libc.so.6
GNU C Library stable release version 2.7, by Roland McGrath et al.
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.3.1 20080523 (prerelease).
Compiled on a Linux >>2.6.24.4<< system on 2008-06-02.
Available extensions:
	crypt add-on version 2.1 by Michael Glad and others
	GNU Libidn by Simon Josefsson
	Native POSIX Threads Library by Ulrich Drepper et al
	BIND-8.2.3-T5B
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>.

There's also another glibc trick that often comes in useful; the LD_DEBUG environment variable. Start with help and you can get more details from there.

$ LD_DEBUG=help ls
Valid options for the LD_DEBUG environment variable are:

  libs        display library search paths
  reloc       display relocation processing
  files       display progress for input file
  symbols     display symbol table processing
  bindings    display information about symbol binding
  versions    display version dependencies
  all         all previous options combined
  statistics  display relocation statistics
  unused      determined unused DSOs
  help        display this help message and exit

To direct the debugging output into a file instead of standard output
a filename can be specified using the LD_DEBUG_OUTPUT environment variable.

posted at: Fri, 08 Aug 2008 13:51 | in /linux/tips | permalink | add comment (0 others)

I recently picked up the Netgear WG311 V3 very cheap from Office Depot.

The card seems to work fine with ndiswrapper; there are other guides on getting it working.

The first problem was the reception was, in a word, rubbish. After putting the box back in it's usual home the house wireless was lucky to get a 7/100 signal rating. I found a work-around while the cheap external antenna I ordered is arriving; shield the antenna with foil. This increased signal to between 30-40/100, a considerable boost making it actually useful.

The second problem was coming to terms with wpa_supplicant, of which the details often become very confusing very quickly. Here's the Debian 2-second guide for a simple, standard WPA network I was looking for:

1. Add to /etc/network/interfaces

   iface wlan0 inet dhcp
   # Useful with ifup -v 
   #    wpa-debug-level 3
        wpa-conf /etc/wpa_supplicant.conf
   

2. /etc/wpa_supplicant.conf should look like:

   network={
           ssid="your_ssid"
           psk="your_password"
           key_mgmt=WPA-PSK
           proto=WPA
   }
   

posted at: Sun, 06 Jul 2008 20:48 | in /linux/tips | permalink | add comment (0 others)

Although it's not particularly clear, a neat feature of xtightvnc is the ability to modify the pop-up menu that appears when you press F8 to send arbitrary keystrokes.

The trick is overriding some of the X11 resources of the file. If you have a look in /etc/X11/app-defaults/Vncviewer (make sure you have a recent package) you can see the default keybindings for the popup menu, which are a good template. Assuming you want to keep them all, you can add your own buttons by starting at button 9 and re-defining the total button count in your ~/.Xresources:

xtightvncviewer*popupButtonCount: 11

xtightvncviewer*popup*button9.label: Alt-F1
xtightvncviewer*popup*button9.translations: #override\n\
                                            <Btn1Down>,<Btn1Up>: \
                                            SendRFBEvent(keydown,Alt_L) \
                                            SendRFBEvent(keydown,F1) \
                                            SendRFBEvent(keyup, F1) \
                                            SendRFBEvent(keyup, Alt_L)

xtightvncviewer*popup*button10.label: Alt-F2
xtightvncviewer*popup*button10.translations: #override\n\
                                            <Btn1Down>,<Btn1Up>: \
                                            SendRFBEvent(keydown,Alt_L) \
                                            SendRFBEvent(keydown,F2) \
                                            SendRFBEvent(keyup, F2) \
                                            SendRFBEvent(keyup, Alt_L)

xtightvncviewer*popup*button11.label: Alt-F12
xtightvncviewer*popup*button11.translations: #override\n\
                                            <Btn1Down>,<Btn1Up>: \
                                            SendRFBEvent(keydown,Alt_L) \
                                            SendRFBEvent(keydown,F12) \

Make sure you merge this with xrdb -merge ~/.Xresources if you don't want to bother logging out and in. This way I can easily send Alt-F1, etc. through to the other side; useful for things like switching virtual terminals in VMware workstation running remotely. I imagine if you were insane you could do all sorts of other tricks too!

posted at: Thu, 22 May 2008 16:29 | in /linux/tips | permalink | add comment (0 others)

I found a few people asking how to automate scripts using ssh with the -f option but not many solutions. Th -f option makes the ssh process fork into the background once started, but since it doesn't give you it's PID it becomes hard to kill it once the script ends.

The solution is to setup a control channel with -M. This allows you to communicate to the ssh process and close it down when your process finishes. For example, here is how I open a VNC connection to my desktop at work, which involves going through a corporate ssh firewall box.

ssh -M -S /tmp/vncssh-firewall.ctl -L 2022:desktop.internal.company.com:22 -f -N username@firewall.company.com
ssh -M -S /tmp/vncssh-localhost.ctl -N -p 2022 -L 5901:localhost:5901 -f -N username@localhost
xtightvncviewer -encodings tight localhost:1
ssh -S /tmp/vncssh-firewall.ctl -O exit localhost
ssh -S /tmp/vncssh-localhost.ctl -O exit firewall.company.com

posted at: Tue, 04 Mar 2008 21:51 | in /linux/tips | permalink | add comment (0 others)

Last night, after dropping a package from a control file, I was wondering why my package.install file for the remaining package seemed to be ignored (upstream installs a bunch of stuff that I don't want installed in the Debian package; the symptom was all that junk was making its way into the package). Turns out it comes down to the following logic in CDBS:

ifeq ($(words $(DEB_ALL_PACKAGES)),1)
        DEB_DESTDIR = $(CURDIR)/debian/$(strip $(DEB_ALL_PACKAGES))/
else
        DEB_DESTDIR = $(CURDIR)/debian/tmp/
endif

i.e. if there is only one package, then by default install in debian/package. Therefore whatever make install does is what you end up with in your package. Although I can see the reasoning behind this, it wasn't what I wanted since I need the package installed into a temporary location (i.e. debian/tmp) which then uses a .install file to pull out only those files I want. The solution is simple, override DEB_DESTDIR to $(CURDIR)/debian/tmp/ in rules.

I hope this saves someone the half hour or so I spent investigating why my install file was "corrupt"!

posted at: Tue, 05 Feb 2008 15:30 | in /linux/debian | permalink | add comment (0 others)

Or, a modern guide to CPU frequency scaling with Linux. Having setup my laptop long ago, I had a strange hybrid of daemons running all trying to scale the frequency of my laptop periodically based on a myriad of different situations. Having decided to fix this, it appears the simplest approach is, as usual, the best.

The way of the future appears to be to let the kernel do all the work with the ondemand govener. The cpufrequtils Debian package will arrange this for you on boot and by default should just work, although you can of course tweak parameters like minimum speed to decrease to and so forth. You can use the cpufreq-set utility to fiddle with settings, or just go into /sys/devices/system/cpu/cpuN/cpufreq/ to tweak them by hand. The many available daemons (cpufreqd, powernowd, etc.) appear to be largely redundant and are probably best removed.

This dropped the temperature of my laptop about 10 degrees C and, apart from a much cooler lap, is so far imperceptible.

posted at: Tue, 29 Jan 2008 16:30 | in /linux/tips | permalink | add comment (0 others)

I was recently on a wireless network I trust even less than I usually trust wireless networks, so was looking for a way to ensure a little more security. I've previously setup a PPTP tunnel, but that server is on a boat heading to San Francisco so that was not an option. I have a linode with a generous bandwidth limit, so my first thought was to set that up and route through it with OpenVPN. It started getting a bit complicated and didn't work out-of-the-box with NetworkManager so I gave up.

Then I found that ssh has a really nifty -D option to implement a SOCKS5 proxy. Therefore all I needed to do was ssh -D localhost:8080 remote.box and then setup Firefox to use localhost:8080 as a SOCKS proxy server. But it gets better; I did wonder if my DNS requests were leaking onto the local network, which a quick packet sniff confirmed. It turns out with SOCKS5 all you need to is go to the Firefox about:config page and turn on network.proxy.socks_remote_dns and DNS is tunnelled too.

Since my mail already comes and goes via encrypted channels, this zero maintenance approach pretty much wraps up everything I need from a VPN solution!

posted at: Mon, 21 Jan 2008 11:04 | in /linux | permalink | add comment (0 others)

You may have noticed a range of things mounted on a tmpfs file system.

$ mount | grep tmpfs
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)

tmpfs is actually more pervasive than it might appear. It was created to build a filesystem interface ontop of shared pages of RAM. This is useful because there are a number of ways to access shared RAM; notably SysV IPC Shared Memory (shmget, shmat, etc) and mmap of anonymous memory (MAP_ANONYMOUS).

In order to have a consistent infrastructure implementing all of this the kernel abstracts RAM as a filesystem and your mapping as a file. To this end the tmpfs filesystem was created to present this abstraction. It deals with issues such as caching, security and the ability to swap pages in a consistent manner (i.e. using the same infrastructure every other file system does).

Overall, tmpfs is implemented in mm/shmem.c. In the initialisation function the file system is mounted internally by the kernel with vfs_kern_mount so it is always around. It gets fairly complicated because it has to keep track of swapped out pages and the general complexity that come with implementing a full filesystem.

As described above, there are several distinct ways to get access to this file system.

• The SysV IPC shared memory layer (ipc/shm.c) is a wrapper layer which manages IPC keys and internally plugs itself into the tmpfs code. Essentially creating a key with shmget() ends up calling ipc/shmem.c:newseg() which calls mm/shmem.c:shmem_file_setup() which then creates a struct file for the shared memory. This new file reuses the file operations from the tmpfs filesystem, but is not actually put into the file system (i.e. linked to the superblock). When shat() is called to attach to this memory the SysV IPC layer will internally translate this into a mmap of this file. This is what is used by the Shared Global Area of Oracle and some Java implementations, amongst many other things (see /proc/sysvipc/shm for a comprehensive list).
• The second common thing to do is an anonymous mmap. By default MAP_ANONYMOUS ignores the file-descriptor, but an analagous methodology is to explicitly map /dev/zero. If you look at the implementation of /dev/zero in drivers/char/mem.c you can see code such as

  static int mmap_zero(struct file * file, struct vm_area_struct * vma)
  {
          int err;
  
          if (vma->vm_flags & VM_SHARED)
                  return shmem_zero_setup(vma);
  

  which ends up backing shared areas with tmpfs.
• The third way is actually mounting tmpfs and using it from userspace as a normal file system. glibc implements the POSIX shared memory call shm_open by creating and mmaping files on a tmpsfs mount (by default looking in /dev/shm). Other utilities also use the tmpfs semantics and mount it at other points; the obvious one of course being mounting /tmp on it!

tmpfs: more than meets the eye!

Update: hopefully a little clearer, thanks to Helge Bahmann for comments.

posted at: Thu, 08 Nov 2007 17:11 | in /linux | permalink | add comment (1 others)

Sometimes you need direct make to progress through various phases which may not have a particular file target associated with them. For example the first step might be to extract a compressed tarball of data and run a script over it in preparation for compiling that data into something else. Equally you might need to patch some files or maybe download some data. The general idiom to accomplish this sort of thing is the stamp file.

The general principle is to make a phony target that depends on a file called target-stamp. The target-stamp target can then do what it needs to do and simply create a dummy file via touch. This file indicates to make that the phase is complete, and targets that depend on this phase being complete can now depend on the stamp file. A small example follows

all: final

.PHONY: prepare clean

final: prepare
        # run build process

prepare: prepare-phase1-stamp prepare-phase2-stamp

prepare-phase1-stamp:
        # untar data or something
        touch $@

prepare-phase2-stamp: prepare-phase1-stamp
        # run a script or do something else
        touch $@

clean:
        rm -f *-stamp

One place you may notice extensive use of this is the -stamp files in the /debian directory of packages built with CDBS.

posted at: Thu, 27 Sep 2007 11:46 | in /linux/tips | permalink | add comment (2 others)

On many architectures, calling a function does not just mean jumping straight to an address. For example, on Itanium calling a function involves setting a global pointer register before jumping to the function code. The global pointer is used as a known fixed point which can be offset against very quickly. Thus a function is described by two things; the code address and a global pointer value. Unsurprisingly, this information is kept in a function descriptor.

Now, the general process when calling a function from a dynamically linked library is as follows:

1. Your code calls a stub entry in the Procedure Lookup Table (PLT).
2. The PLT stub reads the function descriptor (address and GP) to call from the Global Offset Table (GOT).
3. At first, this will refer to a function in the dynamic linker which will find the library and function you actually want to call, and then patch the GOT function descriptor entry so next time the PLT stub looks at it, you go there directly.

This all works fine. The problem occurs when you try to take the address of a function. On x86 you can just read the function address from the GOT and return it. However, with function descriptors you can not just return the code address -- it is invalid to call the function without the GP set correctly. You can't return the address of your copy of function descriptor, because then what should be the same function doesn't have the same address, which is obviously incorrect.

The Itanium solution is an "official" function descriptor. This is kept by the dynamic linker in private memory, and whenever a function address is requested it will be returned. Consider pretty much the smallest possible example:

$ cat libtest.c
void* function(void)
{
	return function;
}

$ cat test.c
extern void* function(void);

int main(void)
{
        void*(*ptr)(void) = function;
}

$ gcc -shared -o libtest.so ./libtest.c
$ gcc -o test test.c -Wl,-L. -ltest

If we have a look at the relocations in the library we see the following

$ readelf --relocs ./libtest.so

Relocation section '.rela.dyn' at offset 0x440 contains 10 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
[...]
000000010cc0  000b00000047 R_IA64_FPTR64LSB  00000000000007c0 function + 0

hen the dynamic linker is going through the relocations, it responds to any R_IA64_FPTR64LSB relocations by creating a new official function descriptor.

Now we can have a look at the binary

$ readelf --relocs ./test

Relocation section '.rela.dyn' at offset 0x458 contains 3 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
[...]
6000000000000e80  000400000047 R_IA64_FPTR64LSB  0000000000000000 function + 0

This is a bit different, and requires some more investigation. Firstly lets look at the disassembly:

40000000000007c0 <main>:
40000000000007c0:       02 10 00 18 00 21       [MII]       mov r2=r12
40000000000007c6:       e0 40 05 00 48 00                   addl r14=40,r1;;
40000000000007cc:       00 00 04 00                         nop.i 0x0
40000000000007d0:       0a 70 00 1c 18 10       [MMI]       ld8 r14=[r14];;
40000000000007d6:       00 70 08 30 23 80                   st8 [r2]=r14
40000000000007dc:       01 10 00 84                         mov r12=r2
40000000000007e0:       11 00 00 00 01 00       [MIB]       nop.m 0x0
40000000000007e6:       00 00 00 02 00 80                   nop.i 0x0
40000000000007ec:       08 00 84 00                         br.ret.sptk.many b0;;

We know that r1 points to the GOT (in fact, that is the GP value). We see that r14 is being loaded with a value from the GOT which is then de-referenced. Notice the offset of the address, correlating with the section output

There are 41 section headers, starting at offset 0x1690:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
[...]
  [25] .got              PROGBITS         6000000000000e58  00000e58
       0000000000000050  0000000000000000 WAp       0     0     8

Now, 0x6000000000000e80 - 0x6000000000000e58 = 0x28 = 40 which is the offset we are loading from the GOT! So we can deduce that a R_IA64_FPTR64LSB relocation with a symbol value of 0 gets the dynamic linker to setup a pointer to the official function descriptor in the GOT. This is the value returned when you take the address of a function, so now function comparisions work!

Power and PA-RISC are two other examples of architectures that use function descriptors. I think they are generally a feature of architectures who are not register-starved, because you generally pass along a pointer value to global data, which makes offsetting into that global data much faster than looking it up from memory.

posted at: Wed, 15 Aug 2007 12:53 | in /linux | permalink | add comment (0 others)

Update: You need to be careful if you are updating to the latest Debian Exim (~4.67), since the Debian config file format has changed slightly. I'm pretty sure this whole thing could be easier, so I have filed #430057. Instructions below updated slightly.

Sending secure mail seems to have two possible implementations; firstly you can connect over an insecure channel and issue a command (STARTTLS) which tells the SMTP server to start a secure channel. The other option is where you use a secure channel to start with. Usually this happens with an SSL (TLS) connection on port 465 which you then probably have to authenticate over.

Exim doesn't support this second model, seemingly by design. Which is a little annoying if that's all your ISP offers! You may like this on your laptop, since by authenticating you should be able to send email from anywhere through the ISP mail server.

What you need is a wrapper that provides the SSL connection between your computer and the ISP. Then you have to fool exim into using it, and direct it to send passwords unencrypted (though the underlying channel is safely encrypted).

Firstly, install stunnel; I found stunnel4 didn't work that well. Then create a script to start it and make a tunnel to your ISP. Put the following a file /etc/init.d/ssmtp-tunnel (change to your ISP's secure email server) and then run update-rc.d ssmtp-tunnel defaults (and start it with /etc/init.d/ssmtp-tunnel start).

#! /bin/sh -e

case "$1" in
  start)
    echo -n "Starting ssmtp tunnel "
    start-stop-daemon --start --quiet --exec /usr/sbin/stunnel -- -c -d ssmtp -r securemail.internode.on.net:ssmtp
    echo "stunnel."
    ;;
  stop)
    echo -n "Stopping ssmtp tunnel "
    start-stop-daemon --stop --quiet --oknodo --retry 2 --exec /usr/sbin/stunnel
    echo "stunnel."
    ;;
  restart)
    $0 stop
    sleep 1
    $0 start
    ;;
  *)
    echo "Usage: /etc/init.d/ssmtp-tunnel {start|stop|restart|reload|force-reload}"
    exit 1
esac

exit 0

If you telnet localhost 465 and see a normal SMTP connection, which is running over SSL, you have things working correctly.

Now you need to configure exim to use this to firstly authenticate, then send the email onto the smarthost.

Make sure you're using the big config file option with dpkg-reconfigure exim4-config. When it asks you what the smarthost should be, tell it localhost.

• Firstly create the file /etc/exim4/exim4.conf.localmacros (if it doesn't already exist) and add a line AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS = true. This forces using authentication even though it looks like an unencrypted channel.
• Then in /etc/exim4/exim4.conf.template, under the smarthost router (i.e. the line that starts smarthost:) add self = send. This allows what exim thinks is a router to an external MTA to actually go back to the localhost.
• In the same file change the remote_smtp_smarthost (i.e the line that starts remote_smtp_smarthost:) transport to have:
  • hosts_avoid_tls = localhost
  • hosts_require_auth = localhost
  • port = 465
  (all on separate lines).
• Add a line to /etc/exim4/passwd.client for localhost with your ISP username/password (or just use * if this is the only entry).

Finally, update the config file with update-exim4.conf and restart exim /etc/init.d/exim4 restart. All going well, Exim will now get the mail out wherever you are!

posted at: Fri, 22 Jun 2007 12:50 | in /linux/tips | permalink | add comment (10 others)

If you use Iceweasel and a website is complaining you don't have Firefox® (such as the Google toolbar installation page), download the user-agent switcher plugin and use it to import this specification. You can then "fake" being a plain-old Firefox® running on on Linux; the Iceweasel site provides one that fakes Firefox® on Windows XP should you need that.

posted at: Thu, 17 May 2007 11:56 | in /linux/debian | permalink | add comment (0 others)

Today I wanted a way to run a program to completion but stop it just before it is destroyed, therefore allowing me to inspect some /proc variables I was interested in.

As it happens the ptrace option PTRACE_O_TRACEEXIT is designed for just that. To use it, you first either attach to the process, or if forking a new process call PTRACE_TRACEME and exec as usual.

Then you use the ptrace call PTRACE_SETOPTIONS with the PTRACE_O_TRACEEXIT option in the data pointer, and wait for the traced process as usual (unfortunately due to a bug you will have to clag the #defines from the kernel header for now). There are now two conditions to handle:

1. The child received a SIGTRAP with the PTRACE_EVENT_EXIT status set, signalling to us that it has exited. This is a terminal state; you can inspect the registers, etc, but the process can not come back to life now. This status is "overloaded" in the status returned by wait, e.g.

   /* wait for signal from child */
   waitpid(pid, &status, 0);
   
   /* any signal stops the child, so check that the incoming signal is a
   SIGTRAP and the event_exit flag is set */
   
   if ((WSTOPSIG(status) == SIGTRAP) &&
       (status & (PTRACE_EVENT_EXIT << 8)))
        ...
   

2. Any other signal will have also caused the tracing parent to be woken, so we need to pass that signal through to the child process with a PTRACE_CONT call specifing the signal in the data pointer; e.g.

   /* if not, pass the original signal onto the child */
   if (ptrace(PTRACE_CONT, pid, 0, WSTOPSIG(status))) {
   	   perror("stopper: ptrace(PTRACE_CONT, ...)");
              exit(1);
   }
   

A straight forward example is stopper.c. The program allows you to run a program (or attach to a running program) and effecitvley pause it just before it exits, allowing you to inspect it and then manually dispose of it once you have the data you need.

posted at: Wed, 16 May 2007 13:32 | in /linux | permalink | add comment (1 others)