technovelty

weblog of Ian Wienand

RSS  |  technovelty home  |  page of ian  |  ian@wienand.org

Converting DICOM images, part 2

In a previous post I discussed converting medical images. I tried again today and hit another issue which could do with some google help.

If you see the following:

$ dcm2pnm --all-frames input.dcm
E: can't change to unencapsulated representation for pixel data
E: can't determine 'PhotometricInterpretation' of decompressed image
E: can't decompress frame 0: Illegal call, perhaps wrong parameters
E: can't convert input pixel data, probably unsupported compression
F: Invalid DICOM image

and dcmdump reports somewhere:

# Dicom-Data-Set
# Used TransferSyntax: JPEG Baseline

Try the dcmj2pnm program, rather than dcm2pnm. The man page does mention this, but that assumes that you know you have a JPEG encoded image :) It works the same, can extract multiple frames and can be converted to a movie as discussed in the previous post. Easy when you know how!

posted at: Mon, 14 Mar 2011 23:02 | in /linux/tips | permalink | add comment (0 others)

vi backup files considered harmful

Mark this one down as another in the long list of "duh" — once you realise what is going on!

Bug report comes in about a long running daemon that has stopped logging. lsof reports the log file is now named logfile~ and further more is deleted! This happens after a system upgrade scenario, so of course I go off digging through a multitude of scripts and what-not to find the culprit...

Have you got it yet?

Try this...

# lsof | grep syslogd | grep messages
syslogd    1376        root   15w      REG        3,1    99851    4605625 /var/log/messages
# cd /var/log/
# vi messages (and save the file)
root@jj:/var/log# lsof | grep syslogd | grep messages
syslogd    1376        root   15w      REG        3,1    99851    4605625 /var/log/messages~ (deleted)

vi is very careful and renames your existing file, so that if anything goes wrong when writing the new version you can get something back. It's a shame the daemon doesn't know about this! The kernel is happy to deal with the rename, but when the backup file is unlinked you're out of luck. Confusingly to a casual inspection your log file looks like it's there ... just that nothing is going into it. (oh, and if you tried that, you might like to restart syslogd now :)

Moral of the story -- overcome that finger-memory and never use vi on a live file; you're asking for trouble!

posted at: Fri, 08 Jan 2010 16:15 | in /linux/tips | permalink | add comment (6 others)

SIGTTOU and switching to canonical mode

Here's an interesting behaviour that, as far as I can tell, is completley undocumented, sightly consfusing but fairly logical. Your program should receive a SIGTTOU when it is running in the background and attempts to output to the terminal -- the idea being that you shouldn't scramble the output by mixing it in while the shell is trying to operate. Here's what the bash manual has to say

Background processes are those whose process group ID differs from the
terminal's; such processes are immune to key- board-generated signals.
Only foreground processes are allowed to read from or write to the
terminal.  Background processes which attempt to read from (write to)
the terminal are sent a SIGTTIN (SIGTTOU) signal by the terminal
driver, which, unless caught, suspends the process.

So, consider the following short program, which writes some output and catches any SIGTTOU's, with an optional flag to switch between canonical and non-canonical mode.

#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <termios.h>
#include <unistd.h>

static void sig_ttou(int signo) {
   printf("caught SIGTTOU\n");
   signal(SIGTTOU, SIG_DFL);
   kill(getpid(), SIGTTOU);
}

int main(int argc, char *argv[]) {

   signal(SIGTTOU, sig_ttou);

   if (argc != 1) {
      struct termios tty;

      printf("setting non-canoncial mode\n");
      tcgetattr(fileno(stdout), &tty);
      tty.c_lflag &= ~(ICANON);
      tcsetattr(fileno(stdout), TCSANOW, &tty);
   }

   int i = 0;
   while (1) {
      printf("  *** %d ***\n", i++);
      sleep(1);
   }
}

This program ends up operating in an interesting manner.

  1. Run in the background, canonical mode : no SIGTTOU and output gets multiplexed with shell.
    $ ./sigttou &
      *** 0 ***
    [1] 26171
    $   *** 1 ***
      *** 2 ***
      *** 3 ***
    
  2. Run in the background, non-canonical mode : SIGTTOU delivered
    $ ./sigttou 1 &
    [1] 26494
    ianw@jj:/tmp$ setting non-canoncial mode
    caught SIGTTOU
    
    
    [1]+  Stopped                 ./sigttou 1
    
  3. Run in the background, canonical mode, tostop set via stty : SIGTTOU delivered, seemingly after a write proceeds
    $ stty tostop
    $ ./sigttou &
    [2] 26531
    ianw@jj:/tmp$   *** 0 ***
    caught SIGTTOU
    
    
    [2]+  Stopped                 ./sigttou
    

You can see a practical example of this by comparing the difference between cat file & and more file &. The semantics make some sense -- anything switching off canonical mode is like to be going to really scramble your terminal, so it's good to stop it and let it's terminal handling functions run. I'm not sure why canoncial background is considered useful mixed in with your prompt, but someone, somewhere must have decided it was so.

Update: upon further investigation, it is the switching of terminal modes that invokes the SIGTTOU. To follow the logic through more, see the various users of tty_check_change in the tty driver.

posted at: Fri, 21 Aug 2009 11:02 | in /linux/tips | permalink | add comment (0 others)

Converting DICOM images

If you go for an ultrasound or some other imaging procedure, they may give you a CD with the images that requires some overly complicated and under-featured Windows viewer. Chances are these images are in DICOM format, which is like the AVI of the medical world.

Your first clue will be that file might report the file as an unoptimised QuickTime movie, e.g.

$ file ./QMAG0001
./QMAG0001: Apple QuickTime movie (unoptimized)

After figuring out the file type wasn't actually anything to do with QuickTime, I tried some of the many different tools and methods to convert this to something viewable. Unfortunatley, the DICOM viewer in GIMP and ImageMagick (probably the same thing?) didn't like the files at all, and neither did a range of other tools. I finally managed to do it with the dcm2pnm tool from the Debian dcmtk package -- just point it at the file and it spits out a PNM which is easily converted by all graphics tools.

You can also encapsulate a series of images in a DICOM file, like a little movie. dcm2pnm extracts these easily, but requires the --all-frames options. An ffmpeg recipe to turn these extracted files into a more easily viewable movie is:

$ ffmpeg -qscale 5 -r 20 -b 9600 -i foo.%d.ppm movie.mp4

I certainly can't guarantee this will actually work for you, as DICOM appears to be an extremely complicated format with many possible vendor extensions. But hopefully it's a starting point!

posted at: Sun, 08 Feb 2009 14:11 | in /linux/tips | permalink | add comment (2 others)

NoMachine NX - the missing non-manual

I've been meaning to try NoMachine NX for a while. Its promise of fast remote X11 sessions sounded exactly like what I wanted to log into my work desktop remotely (I really like having a remote desktop with saved state you can just pick up from when using remote access). That was pretty much all I knew about the software, so I was a completely blank slate.

The getting started guide is the perfect example of how not to write a getting started guide.

Firstly, Section 1 - "Getting started" - gives me a full history of the product, goes into significant depth about the challenges of forwarding X11 requests, talks about the caching and compression implementation, round-trip latency measurement, the details of two-way proxying system and discusses every other feature of the software.

My eyes glazed over after about the first paragraph. That's all great -- I just want to know what to do!

At this point, I assume that I'm required to run some sort of daemon at the remote end. I download and install the server package (it is explained that the server package requires the client and agent packages as well, fine).

I'm paging down, looking for something to get me started. I'm happy to see Section 7 - "Set up your NX Server environment" (remember, at this point I though I needed some daemon running in the background constantly). It even has some commands commands to type, so I tap away, running nxserver --useradd nxtest --system. My server binary doesn't even seem to recognise these options. I give up, assuming that the server isn't running and nothing will work. The getting started guide has abruptly ended and I have no idea what to do.

As it turns out, it's all completely trivial. Here's the missing "getting started guide".

  • Download and install the client, agent and server packages on the remote end. You need to have ssh access to this box.
  • Install the client on your end.
  • Run /usr/NX/bin/nxclient. It will start a wizard where you input the remote host name.
  • The client will, under the hood, ssh to the remote end, open the tunnel it needs, start the server and do all the magic required to make things "just work". A remote desktop will appear.
  • That's it!

Additional tips:

  • It's easy to tunnel this connection (for example, if you have to bounce through a ssh gateway to your internal network). Do something like /usr/NX/bin/nxssh -o 'Compression=no' -L 2022:remote.host:22 -f -N user@sshgateway.company.com and then connect the client to localhost:2022. You don't want to compress this link, as NX is already doing it.
  • The only way I can find to make a new session is to start nxclient with the --wizard command.
  • Don't click "Disable encryption of all traffic" if you're tunneling. AFAICT this tries to redirect the client to a non-encrypted port, which obviously won't get through.

Other than the documentation, it really works as promised, making remote X11 usable. One really nice feature is that it is smart about the resolution of the remote desktop, filling up your local screen. Add to that you don't need anything setup but your normal ssh connection, and it's a great remote desktop solution.

posted at: Wed, 04 Feb 2009 15:10 | in /linux/tips | permalink | add comment (11 others)

How to find the current glibc version

A recent post reminded me of a problem I once had; determine the glibc version and its support for various things.

There's actually a little known but useful confstr call defined for just this sort of thing. Here's a minimal example:

#include <stdio.h>
#include <unistd.h>
#include <alloca.h>
#include <string.h>

int main (void)
{
        size_t n = confstr (_CS_GNU_LIBC_VERSION, NULL, 0);
        if (n > 0)
        {
                char *buf = alloca (n);
                confstr (_CS_GNU_LIBC_VERSION, buf, n);
                printf("%s\n", buf);
        }
        return 0;
}

man confstr has all the details. If you don't need it in your program, you can also just run /lib/libc/so.6 and it will print out it's version info, e.g.

$ /lib/libc.so.6
GNU C Library stable release version 2.7, by Roland McGrath et al.
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.3.1 20080523 (prerelease).
Compiled on a Linux >>2.6.24.4<< system on 2008-06-02.
Available extensions:
	crypt add-on version 2.1 by Michael Glad and others
	GNU Libidn by Simon Josefsson
	Native POSIX Threads Library by Ulrich Drepper et al
	BIND-8.2.3-T5B
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>.

There's also another glibc trick that often comes in useful; the LD_DEBUG environment variable. Start with help and you can get more details from there.

$ LD_DEBUG=help ls
Valid options for the LD_DEBUG environment variable are:

  libs        display library search paths
  reloc       display relocation processing
  files       display progress for input file
  symbols     display symbol table processing
  bindings    display information about symbol binding
  versions    display version dependencies
  all         all previous options combined
  statistics  display relocation statistics
  unused      determined unused DSOs
  help        display this help message and exit

To direct the debugging output into a file instead of standard output
a filename can be specified using the LD_DEBUG_OUTPUT environment variable.

posted at: Fri, 08 Aug 2008 13:51 | in /linux/tips | permalink | add comment (0 others)

Netgear WG311 shielding

I recently picked up the Netgear WG311 V3 very cheap from Office Depot.

The card seems to work fine with ndiswrapper; there are other guides on getting it working.

The first problem was the reception was, in a word, rubbish. After putting the box back in it's usual home the house wireless was lucky to get a 7/100 signal rating. I found a work-around while the cheap external antenna I ordered is arriving; shield the antenna with foil. This increased signal to between 30-40/100, a considerable boost making it actually useful.

Cheap RF Shielding

The second problem was coming to terms with wpa_supplicant, of which the details often become very confusing very quickly. Here's the Debian 2-second guide for a simple, standard WPA network I was looking for:

  1. Add to /etc/network/interfaces
    iface wlan0 inet dhcp
    # Useful with ifup -v 
    #    wpa-debug-level 3
         wpa-conf /etc/wpa_supplicant.conf
    
  2. /etc/wpa_supplicant.conf should look like:
    network={
            ssid="your_ssid"
            psk="your_password"
            key_mgmt=WPA-PSK
            proto=WPA
    }
    

posted at: Sun, 06 Jul 2008 20:48 | in /linux/tips | permalink | add comment (0 others)

Sending arbitrary keys through xtightvnc

Although it's not particularly clear, a neat feature of xtightvnc is the ability to modify the pop-up menu that appears when you press F8 to send arbitrary keystrokes.

The trick is overriding some of the X11 resources of the file. If you have a look in /etc/X11/app-defaults/Vncviewer (make sure you have a recent package) you can see the default keybindings for the popup menu, which are a good template. Assuming you want to keep them all, you can add your own buttons by starting at button 9 and re-defining the total button count in your ~/.Xresources:

xtightvncviewer*popupButtonCount: 11

xtightvncviewer*popup*button9.label: Alt-F1
xtightvncviewer*popup*button9.translations: #override\n\
                                            <Btn1Down>,<Btn1Up>: \
                                            SendRFBEvent(keydown,Alt_L) \
                                            SendRFBEvent(keydown,F1) \
                                            SendRFBEvent(keyup, F1) \
                                            SendRFBEvent(keyup, Alt_L)

xtightvncviewer*popup*button10.label: Alt-F2
xtightvncviewer*popup*button10.translations: #override\n\
                                            <Btn1Down>,<Btn1Up>: \
                                            SendRFBEvent(keydown,Alt_L) \
                                            SendRFBEvent(keydown,F2) \
                                            SendRFBEvent(keyup, F2) \
                                            SendRFBEvent(keyup, Alt_L)

xtightvncviewer*popup*button11.label: Alt-F12
xtightvncviewer*popup*button11.translations: #override\n\
                                            <Btn1Down>,<Btn1Up>: \
                                            SendRFBEvent(keydown,Alt_L) \
                                            SendRFBEvent(keydown,F12) \

Make sure you merge this with xrdb -merge ~/.Xresources if you don't want to bother logging out and in. This way I can easily send Alt-F1, etc. through to the other side; useful for things like switching virtual terminals in VMware workstation running remotely. I imagine if you were insane you could do all sorts of other tricks too!

posted at: Thu, 22 May 2008 16:29 | in /linux/tips | permalink | add comment (0 others)

Forwarding VNC via ssh, or finding the PID of a running ssh

I found a few people asking how to automate scripts using ssh with the -f option but not many solutions. Th -f option makes the ssh process fork into the background once started, but since it doesn't give you it's PID it becomes hard to kill it once the script ends.

The solution is to setup a control channel with -M. This allows you to communicate to the ssh process and close it down when your process finishes. For example, here is how I open a VNC connection to my desktop at work, which involves going through a corporate ssh firewall box.

ssh -M -S /tmp/vncssh-firewall.ctl -L 2022:desktop.internal.company.com:22 -f -N username@firewall.company.com
ssh -M -S /tmp/vncssh-localhost.ctl -N -p 2022 -L 5901:localhost:5901 -f -N username@localhost
xtightvncviewer -encodings tight localhost:1
ssh -S /tmp/vncssh-firewall.ctl -O exit localhost
ssh -S /tmp/vncssh-localhost.ctl -O exit firewall.company.com

posted at: Tue, 04 Mar 2008 21:51 | in /linux/tips | permalink | add comment (0 others)

A cool laptop

Or, a modern guide to CPU frequency scaling with Linux. Having setup my laptop long ago, I had a strange hybrid of daemons running all trying to scale the frequency of my laptop periodically based on a myriad of different situations. Having decided to fix this, it appears the simplest approach is, as usual, the best.

The way of the future appears to be to let the kernel do all the work with the ondemand govener. The cpufrequtils Debian package will arrange this for you on boot and by default should just work, although you can of course tweak parameters like minimum speed to decrease to and so forth. You can use the cpufreq-set utility to fiddle with settings, or just go into /sys/devices/system/cpu/cpuN/cpufreq/ to tweak them by hand. The many available daemons (cpufreqd, powernowd, etc.) appear to be largely redundant and are probably best removed.

This dropped the temperature of my laptop about 10 degrees C and, apart from a much cooler lap, is so far imperceptible.

posted at: Tue, 29 Jan 2008 16:30 | in /linux/tips | permalink | add comment (0 others)

The stamp idiom with make

Sometimes you need direct make to progress through various phases which may not have a particular file target associated with them. For example the first step might be to extract a compressed tarball of data and run a script over it in preparation for compiling that data into something else. Equally you might need to patch some files or maybe download some data. The general idiom to accomplish this sort of thing is the stamp file.

The general principle is to make a phony target that depends on a file called target-stamp. The target-stamp target can then do what it needs to do and simply create a dummy file via touch. This file indicates to make that the phase is complete, and targets that depend on this phase being complete can now depend on the stamp file. A small example follows

all: final

.PHONY: prepare clean

final: prepare
        # run build process

prepare: prepare-phase1-stamp prepare-phase2-stamp

prepare-phase1-stamp:
        # untar data or something
        touch $@

prepare-phase2-stamp: prepare-phase1-stamp
        # run a script or do something else
        touch $@

clean:
        rm -f *-stamp

One place you may notice extensive use of this is the -stamp files in the /debian directory of packages built with CDBS.

posted at: Thu, 27 Sep 2007 11:46 | in /linux/tips | permalink | add comment (2 others)

exim4 with ssmtp on Debian

Update: You need to be careful if you are updating to the latest Debian Exim (~4.67), since the Debian config file format has changed slightly. I'm pretty sure this whole thing could be easier, so I have filed #430057. Instructions below updated slightly.

Sending secure mail seems to have two possible implementations; firstly you can connect over an insecure channel and issue a command (STARTTLS) which tells the SMTP server to start a secure channel. The other option is where you use a secure channel to start with. Usually this happens with an SSL (TLS) connection on port 465 which you then probably have to authenticate over.

Exim doesn't support this second model, seemingly by design. Which is a little annoying if that's all your ISP offers! You may like this on your laptop, since by authenticating you should be able to send email from anywhere through the ISP mail server.

What you need is a wrapper that provides the SSL connection between your computer and the ISP. Then you have to fool exim into using it, and direct it to send passwords unencrypted (though the underlying channel is safely encrypted).

Firstly, install stunnel; I found stunnel4 didn't work that well. Then create a script to start it and make a tunnel to your ISP. Put the following a file /etc/init.d/ssmtp-tunnel (change to your ISP's secure email server) and then run update-rc.d ssmtp-tunnel defaults (and start it with /etc/init.d/ssmtp-tunnel start).

#! /bin/sh -e

case "$1" in
  start)
    echo -n "Starting ssmtp tunnel "
    start-stop-daemon --start --quiet --exec /usr/sbin/stunnel -- -c -d ssmtp -r securemail.internode.on.net:ssmtp
    echo "stunnel."
    ;;
  stop)
    echo -n "Stopping ssmtp tunnel "
    start-stop-daemon --stop --quiet --oknodo --retry 2 --exec /usr/sbin/stunnel
    echo "stunnel."
    ;;
  restart)
    $0 stop
    sleep 1
    $0 start
    ;;
  *)
    echo "Usage: /etc/init.d/ssmtp-tunnel {start|stop|restart|reload|force-reload}"
    exit 1
esac

exit 0

If you telnet localhost 465 and see a normal SMTP connection, which is running over SSL, you have things working correctly.

Now you need to configure exim to use this to firstly authenticate, then send the email onto the smarthost.

Make sure you're using the big config file option with dpkg-reconfigure exim4-config. When it asks you what the smarthost should be, tell it localhost.

  • Firstly create the file /etc/exim4/exim4.conf.localmacros (if it doesn't already exist) and add a line AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS = true. This forces using authentication even though it looks like an unencrypted channel.
  • Then in /etc/exim4/exim4.conf.template, under the smarthost router (i.e. the line that starts smarthost:) add self = send. This allows what exim thinks is a router to an external MTA to actually go back to the localhost.
  • In the same file change the remote_smtp_smarthost (i.e the line that starts remote_smtp_smarthost:) transport to have:
    • hosts_avoid_tls = localhost
    • hosts_require_auth = localhost
    • port = 465
    (all on separate lines).
  • Add a line to /etc/exim4/passwd.client for localhost with your ISP username/password (or just use * if this is the only entry).

Finally, update the config file with update-exim4.conf and restart exim /etc/init.d/exim4 restart. All going well, Exim will now get the mail out wherever you are!

posted at: Fri, 22 Jun 2007 12:50 | in /linux/tips | permalink | add comment (15 others)

Getting a tick in LaTeX

So you need to show something is good in your LaTeX document, and want a tick (or a cross)? The pifont package gives you access to ZapfDingbats, which has lots of neat characters as shown in the documentation (along with dinglists and other nifty features).

So, after \usepackage{pifont} a quick \newcommand{\tick}{\ding{52}} gives you \tick for all your ticking needs!

The MarvoSym package also has some Winding type things, like laundry instructions (?).

file this under the "easy when you know how but otherwise takes you at least half an hour" category...

posted at: Wed, 19 Jul 2006 14:16 | in /linux/tips | permalink | add comment (4 others)

xfig and colours

User defined colours in xfig are per file. To get a decent selection of colours, a good way is to have the xfig-libs package installed and import one of the complex example pictures from the library. This way you get all its colours. But be careful; these colours can make it into exported postscript and cause your interpreter to really crawl.

This tip courtesy of Charles Gray, who is currently in the process of obtaining a PhD in xfigology.

posted at: Thu, 30 Mar 2006 23:16 | in /linux/tips | permalink | add comment (1 others)

How to cancel a pending CVS remove

If you accidently locally do cvs rm on a file and want it back (and you haven't committed, of course), just cvs add it and it will restore the latest version.

The trick is to not copy back the missing file and try to add that, otherwise you'll get an error about how the file should be removed and is still there (or is back again). If you have a later version of file you are trying to resurrect move it back after you do the (re)add.

posted at: Tue, 17 Jan 2006 17:01 | in /linux/tips | permalink | add comment (0 others)

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.