rstdiary

I find it very useful to spend 5 minutes a day to keep a small log of what was worked on, major bugs or reviews and a general small status report. It makes rolling up into a bigger status report easier when required, or handy as a reference before you go into meetings etc.

I was happily using an etherpad page until I couldn't save any more revisions and the page got too long and started giving javascript timeouts. For a replacement I wanted a single file as input with no boilerplate to aid in back-referencing and adding entries quickly. It should be formatted to be future-proof, as well as being emacs, makefile and git friendly. Output should be web-based so I can refer to it easily and point people at it when required, but it just has to be rsynced to public_html with zero setup.

rstdiary will take a flat RST based input file and chunk it into some reasonable looking static-HTML that looks something like this. It's split by month with some minimal navigation. Copy the output directory somewhere and it is done.

It might also serve as a small example of parsing and converting RST nodes where it does the chunking; unfortunately the official documentation on that is "to be completed" and I couldn't find anything like a canonical example, so I gathered what I could from looking at the source of the transformation stuff. As the license says, the software is provided "as is" without warranty!

So if you've been thinking "I should keep a short daily journal in a flat-file and publish it to a web-server but I can't find any software to do just that" you now have one less excuse.

Bash arithmetic evaluation and errexit trap

In the "traps for new players" category:

count=0
things="0 1 0 0 1"

for i in $things;
do
   if [ $i == "1" ]; then
       (( count++ ))
   fi
done

echo "Count is ${count}"

Looks fine? I've probably written this many times. There's a small gotcha:

((expression))
The expression is evaluated according to the rules described below under ARITHMETIC EVALUATION. If the value of the expression is non-zero, the return status is 0; otherwise the return status is 1. This is exactly equivalent to let "expression".

When you run this script with -e or enable errexit -- probably because the script has become too big to be reliable without it -- count++ is going to return 0 (post-increment) and per above stop the script. A definite trap to watch out for!

Finding out if you're a Rackspace instance

Different hosting providers do things slightly differently, so it's sometimes handy to be able to figure out where you are. Rackspace is based on Xen and their provided images should include the xenstore-ls command available. xenstore-ls vm-data will give you a handy provider and even region fields to let you know where you are.

function is_rackspace {
  if [ ! -f /usr/bin/xenstore-ls ]; then
      return 1
  fi

  /usr/bin/xenstore-ls vm-data | grep -q "Rackspace"
}

if is_rackspace; then
  echo "I am on Rackspace"
fi

Other reading about how this works:

ip link up versus status UP

ip link show has an up filter which the man page describes as display running interfaces. The output also shows the state, e.g.

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
  link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
  link/ether e8:03:9a:b6:46:b3 brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT group default qlen 1000
  link/ether c4:85:08:2a:6e:3a brd ff:ff:ff:ff:ff:ff

It is not described what the difference is between these two.

The state output is derived from IFLA_OPERSTATE as per this little bit in ip/ipaddress.c in iproute2

if (tb[IFLA_OPERSTATE])
  print_operstate(fp, rta_getattr_u8(tb[IFLA_OPERSTATE]));

as per operstates.txt a status of UP means that the interface is up and can be used to send packets.

In contrast, ip link show up runs a different filter

if (filter.up && !(ifi->ifi_flags&IFF_UP))
  return 0;

Thus only interfaces that are in state IFF_UP are shown. This means it is administratively up, but this does not mean it is available and ready to send packets.

This may make a difference if you're trying to use the output of ip link in any sort of automation where you have to probe the available network-interfaces and know where packets are going to get out or not.

Non-stick tortilla technique

I have never had much success with tortilla making, generally finding the fragile masa would stick to everything and tear easily. However I was listening to the Cook's Illustrated podcast where they mentioned that cooking a pizza on a stone with silicone-paper makes no difference over doing it on the stone directly. For some reason I had just never thought of cooking it with the silicone-paper on; I'd always frustrated myself trying to separate the pressed tortilla in various ways.

So here's my technique:

  1. Mix the masa as on the packet; 2 cups to 2-half cups water
  2. Leave for 20 minutes or so
  3. Use plastic-wrap on the top of the tortilla press
  4. Cut a silicone-paper square for the bottom
  5. Ball the masa and press
  6. Now the plastic-wrap will peel easily off the top of the uncooked tortilla
  7. Take the silcone-paper and put the tortilla down on the pan/hotplate
  8. Give a slight press with the spatula around the edges
  9. After about 20 seconds, the silicone-paper should peel off the tortilla fairly easily.
  10. repeat!

I've tried mixing in suet, oil and copha and none of them are worth the extra calories over just plain masa in my opinion and don't help with sticking anyway. Enjoy!

Gyro Wheel Battery Pack

Several years ago I bought a Gyro Wheel for my daughter, who probably couldn't even walk at the time. The idea seemed really good; a heavy cast-iron wheel is spun up and the angular momentum keeps the new-rider upright while they learn to ride.

I don't suppose it revolutionised bike-riding, as it seems that the company has disappeared. Now my daughter can reach the pedals, when I finally dug it out the other day it was sad to find it wouldn't charge. Googling seems to show that it's a fairly common problem.

After pulling it apart, I found that it was fairly straight-forward to replace the battery-pack. Getting in requires a torx screwdriver but is otherwise straight-forward.

The battery looks like a bit of a custom job, with two-halves of a 9.6V battery-pack split into two. You won't be able to buy anything that directly replaces this, but you should be able to find individual AA-sized NiMH rechargable batteries with welded-tabs that you can use to create a new battery-pack (don't buy batteries without tabs and try to solder directly onto the battery terminals, it won't work). Unwrap the plastic from the existing packs and just follow the polarity; if you carefully de-solder the existing wires you can save yourself a lot of work making new ones. Wrap it up securely and put everything back together.

Below are some photos just to give you an idea of what to expect.

Tenvis IP391W meta-page

Recently I purchased a Tenvis IP391W-HD camera.

I would be unlikely to recommend it. The price is certainly right and the picture quality is quite good. The Android and iPhone apps do work to watch the stream live.

However, the interface is terrible and almost useless without Internet Explorer. There is a RTSP stream (rtsp://admin:password@ip) which VLC can seem to handle, but not mplayer. The recording format (.h264) is not viewable by VLC or mplayer and all I could find is a Windows .exe to convert them to an .avi.

The motion detection gets troubled by the dark. It would really only be useful for something permanently well-lit. It did send me emails via gmail.

I have got it recording to a NFS server, but I don't have a lot of confidence in the reliability of it. I think I have it configured to record in 3600-second blocks (given the interface, it's hard to tell if I've set it up to the network, or to the internal flash, etc), but it seems to intersperse 60 minute recordings with random small recordings. Given the whole idea of a security camera is to record the unexpected, you want a lot of confidence you're actually recording, which you don't get with this. You can see below it recorded 3 hour blocks, then started going a little crazy...

-rw-r--r-- 1 nobody nogroup  69M Mar 11 01:25 0-003035.v264
-rw-r--r-- 1 nobody nogroup  69M Mar 11 02:25 0-013049.v264
-rw-r--r-- 1 nobody nogroup  69M Mar 11 03:26 0-023103.v264
-rw-r--r-- 1 nobody nogroup 5.9M Mar 11 03:31 0-033117.v264
-rw-r--r-- 1 nobody nogroup 1.5M Mar 11 03:40 0-034350.v264
-rw-r--r-- 1 nobody nogroup  17M Mar 11 04:02 0-035259.v264
-rw-r--r-- 1 nobody nogroup 306K Mar 11 04:10 0-041548.v264
-rw-r--r-- 1 nobody nogroup 4.9M Mar 11 04:23 0-042457.v264

There is a support forum, where I found the following files scattered in various posts. From what I can tell, they are the latest as of this writing. I can confirm they work with my IP391W-HD, which the system tells me is GM8126 hardware and came with firmware 1.2.8.3.

  • 1.3.3.3.pk2 - firmware (b56f211a569fb03a37d13b706c660dcb)
  • web.pk2 - a UI update that includes dropbox support. This is really for the model that has pan and tilt, so those buttons don't work. (0e42e42bd6f8034e87dcd443dcc3594d)
  • V264ToAVIen.exe - converts the output to an AVI file that mplayer will play (with some complaints) (9c5a858aa454fed4a0186cf244c0d234)

www.modern.ie offers free limited-time Windows VM's which will work to upload this firmware. Just make sure you use a bridged network in the VM; I'm guessing the firmware ActiveX control tells the camera to TFTP the data from it, which doesn't work via NAT.

Somewhat worryingly, you can telnet to it and get a login prompt (TASTECH login). So it has a built-in backdoor you can't disable.

There have been some efforts to hack the device. leecher@dose.0wnz.at did an excellent job reverse engineering the .pk2 format and writing tenvis_pack.c (no license, I'm generously assuming public domain). I used this to recreate the firmware above with a telnet daemon listening with a shell on port 2525 (no password, just telnet to it)

It's interesting to poke around, but it seems like the whole thing is really driven by a binary called ipc8126

/ # ipc8126 --help
*** TAS-Tech IPCAM/DVS
*** Version: 1.3.3.3
*** Release date: 2013-08-05 15:48:32

In general, I'd say hackability is quite low.

Warning : any of the above might turn your camera into a paperweight. It worked for me, but that's all I can say...

Skipping pages with django.core.paginator

Here's a little snippet for compressing the length of long Django pagination by just showing context around the currently selected page. What I wanted was the first few and last few pages always selectable with some context pages around the currently selected page; e.g.

Example of skip markers in paginator

If that's what you're looking for, some variation on below may be of use. In this approach, you build up a list of pages similar to the paginator object page_range but with only the relevant pages and the skip-markers identified.

from django.core.paginator import Paginator
import unittest

class Pages:

    def __init__(self, objects, count):
        self.pages = Paginator(objects, count)

    def pages_to_show(self, page):
        # pages_wanted stores the pages we want to see, e.g.
        #  - first and second page always
        #  - two pages before selected page
        #  - the selected page
        #  - two pages after selected page
        #  - last two pages always
        #
        # Turning the pages into a set removes duplicates for edge
        # cases where the "context pages" (before and after the
        # selected) overlap with the "always show" pages.
        pages_wanted = set([1,2,
                            page-2, page-1,
                            page,
                            page+1, page+2,
                            self.pages.num_pages-1, self.pages.num_pages])

        # The intersection with the page_range trims off the invalid
        # pages outside the total number of pages we actually have.
        # Note that includes invalid negative and >page_range "context
        # pages" which we added above.
        pages_to_show = set(self.pages.page_range).intersection(pages_wanted)
        pages_to_show = sorted(pages_to_show)

        # skip_pages will keep a list of page numbers from
        # pages_to_show that should have a skip-marker inserted
        # after them.  For flexibility this is done by looking for
        # anywhere in the list that doesn't increment by 1 over the
        # last entry.
        skip_pages = [ x[1] for x in zip(pages_to_show[:-1],
                                         pages_to_show[1:])
                       if (x[1] - x[0] != 1) ]

        # Each page in skip_pages should be follwed by a skip-marker
        # sentinel (e.g. -1).
        for i in skip_pages:
            pages_to_show.insert(pages_to_show.index(i), -1)

        return pages_to_show

class TestPages(unittest.TestCase):

    def runTest(self):

        objects = [x for x in range(0,1000)]
        p = Pages(objects, 10)

        self.assertEqual(p.pages_to_show(0),
                         [1, 2, -1, 99, 100])
        self.assertEqual(p.pages_to_show(1),
                         [1,2,3,-1,99,100])
        self.assertEqual(p.pages_to_show(2),
                         [1,2,3,4,-1,99,100])
        self.assertEqual(p.pages_to_show(3),
                         [1,2,3,4,5,-1,99,100])
        self.assertEqual(p.pages_to_show(4),
                         [1,2,3,4,5,6,-1,99,100])
        self.assertEqual(p.pages_to_show(5),
                         [1,2,3,4,5,6,7,-1,99,100])
        self.assertEqual(p.pages_to_show(6),
                         [1,2,-1,4,5,6,7,8,-1,99,100])
        self.assertEqual(p.pages_to_show(7),
                         [1,2,-1,5,6,7,8,9,-1,99,100])

        self.assertEqual(p.pages_to_show(50),
                         [1,2,-1,48,49,50,51,52,-1,99,100])

        self.assertEqual(p.pages_to_show(93),
                         [1,2,-1,91,92,93,94,95,-1,99,100])
        self.assertEqual(p.pages_to_show(94),
                         [1,2,-1,92,93,94,95,96,-1,99,100])
        self.assertEqual(p.pages_to_show(95),
                         [1,2,-1,93,94,95,96,97,-1,99,100])
        self.assertEqual(p.pages_to_show(96),
                         [1,2,-1,94,95,96,97,98,99,100])
        self.assertEqual(p.pages_to_show(97),
                         [1,2,-1,95,96,97,98,99,100])
        self.assertEqual(p.pages_to_show(98),
                         [1,2,-1,96,97,98,99,100])
        self.assertEqual(p.pages_to_show(99),
                         [1,2,-1,97,98,99,100])
        self.assertEqual(p.pages_to_show(100),
                         [1,2,-1,98,99,100])


if __name__ == '__main__':
    unittest.main()

Then somehow pass through the pages_to_show to your view (below I added it to the paginator object passed) and use a template along the lines of

<ul class="pagination">

{% if pages.has_previous %}
  <li><a href="foo.html?page={{ pages.previous_page_number }}">&laquo;</a></li>
{% else %}
  <li class="disabled"><a href="#">&laquo;</a></li>
{% endif %}

{% for page in pages.pages_to_show %}
  {% if page == -1 %}
  <li class="disabled"><a href="#">&hellip;</a></li>
  {% elif page == pages.number %}
  <li class="active"><a href="#">{{ page_num }}</a></li>
  {% else %}
  <li><a href="foo.html?page={{ page_num }}">{{page_num}}</a>
  {% endif %}
{% endfor %}

{% if pages.has_next %}
  <li><a href="foo.html?page={{ pages.next_page_number }}">&raquo;</a></li>
{% else %}
  <li class="disabled"><a href="#">&raquo;</a></li>
{% endif %}

</ul>