It failed to build. Given my previous lack of success with VMware server, I wasn’t too surprised.  What was surprising was it wasn’t too hard to fix it.  The problem was that the vmnet module would not compile and that was due to three things:

• net device ops no longer has set_multicast_list (in netif.c)
• the linux module header needs to be included to define THIS_MODULE
• skb_frag_t has been redefined and needs an adjustment

vmnet.patch

To use it, you will need to find where the modules are built, for me it is /usr/lib/vmware/modules/source

1. mv vmnet.tar vmnet2.tar
2. tar xf vmnet2.tar
3. patch -p1 < vmnet.patch
4. tar cf vmnet.tar vmnet-only

Related articles

• imabonehead: Cross-compiling VMWare View for ARM Linux (in Debian/Ubuntu) | CNXSoft – Embedded Software Development (cnx-software.com)
• Virtually Run Another OS in your System (tech2copycat.wordpress.com)
• Upgrade to Oneiric breaks VMWare Player (aslakjohansen.wordpress.com)

fglrx is (apparently) fixed

So I kicked off the very very slow process of reformatting the crypto drive (it has taken 5 hours and still going) let it gurgle on its merry way and started to read my email.  One of the  emails was that my bug about fglrx not working is closed, apparently it is fixed.  If I had read that 10 minutes earlier, a simple ‘apt-get install fglrx-driver‘ would of perhaps fixed it; oh well.

My problem is now is do I move to the latest driver and hope their fix is my fix or leave it with some ancient version?  My preference is the former; I only hope it works!

psmisc 22.15 and buffer overflows

I was a little surprised that I received an important bug.  The report was saying I had a buffer overflow introduced in 22.15-1, but no relevant code had changed.  The compiler options had done their job and stopped a buffer being overflowed.

But where exactly was the overflow?  Running gdb on pstree quickly showed that it was line 267 of pstree.c which uses strcpy().  That function set off warning bells. The relevant code is:

    PROC *new;
 
    if (!(new = malloc(sizeof(PROC)))) {
        perror("malloc");
        exit(1);
    }
    strcpy(new->comm, comm);

Now comm is the short command name you find in /proc//stat.  It is fixed in the kernel at 16 characters.  The PROC structure has this field as 17 characters long, one extra for the NUL.  I went and checked the Linux source and yes, it is still 16 characters long.  The clue was in the name of the program that it died on.

#6  new_proc (comm=0x6111b0 "{console-kit-dae}", pid=1571, uid=0)
    at pstree.c:267

That string is 17 characters long. The problem is that 16 characters is for the name only. If the name is in brackets or braces, then that 16 character limit doesn’t apply.  The buffer overflow bug has been there for a long time, but only with the compiler flags did it become visible.

Given you need to read names out of the /proc filesystem and if someone can fiddle with that you have bigger problems it doesn’t seem to be too much of an issue.  It should be (and is in Debian 22.15-2) fixed but is a nice example of the compiler catching bad things.

Related articles

• Detecting security problems – using static analysis to catch them early and less expensively (blogs.windriver.com)
• A Textbook Buffer Overflow: A Look at the FreeBSD telnetd Code (thexploit.com)
• Exploit the buffer – Buffer Overflow Attack (alitarhini.wordpress.com)
• Aurelien Jarno: Performances of open-source Radeon driver (aurel32.net)

Image by Cayusa via Flickr

Like a lot of  people, I have one of those set-top TV boxes that can record TV shows at set times.  I made sure that I could get at the files (using a FTP server in this case) and that the files were some sort of common standard (MPEG 4 TS).  I also have a bunch of mp3 music files.

That’s fine when I’m on the desktop because the files are local.  I wanted to make these available to anyone in the household. DLNA seemed to be a reasonably OK way of doing this, the problem was, how to get it working in Linux?

A lot of the problem is that it is hard to find a DLNA only server.  Sure MythTV could do it, but it needs a tv tuner or a lot of fiddling around.  XMBC can also do it, but it needs to be running a GUI.  I even tried mediatomb but could not get the thing to compile as the library calls to mozjs were all using deprecated functions. I just wanted a daemon that served stuff, nothing more; no fancy ui, no need for X just file serving goodness.

Rygel is almost that.  You could say it is a user server much like a torrent client/server.  The nice thing is you can fiddle around with rygel so it becomes close to a real server.  This is how I did it.

First, I made a rygel user with a home directory, but disabled login. I don’t like programs running root if they don’t need it and rygel doesn’t need it.  The home directory needs to be writeable to the rygel user too otherwise the program doesn’t work too well. I use /var/local/rygel as its home.

For the configuration, copy /etc/rygel.conf to ~rygel/.config/rygel.conf  This is the configuration file for rygel. I disabled all of the modules except MediaExport. Make sure you disable Tracker otherwise MediaExport will not work. Tracker is only useful for real users who are logged in and have dbus etc going which this user is certainly not.

I made a simple rygeld file in /usr/local/sbin which basically starts the program, forks and grabs the PID to write to a pidfile. This mean it was easier to track the program in the init scripts.

#!/bin/sh
#
# Rygel daemon handling
RYGEL='/usr/bin/rygel'
RYGEL_ARGS=''
su -s /bin/sh -c "nohup $RYGEL $RYGEL_ARGS > /var/local/rygel/rygel.log 2>&1 &"
rygel
EXIT_CODE=$?
if [ $EXIT_CODE != 0 ] ; then
        return $EXIT_CODE
fi
PGROUP=`ps --no-headers -o pgrp $$`
PID=`pgrep -g $PGROUP -f $RYGEL`
echo $PID > /var/run/rygel.pid
exit 0

In case you were wondering, the pgrp finds the program group so the pgrep finds the right rygel process that has the same program group as the starting shell.

The init script is a standard init script except the –exec flag checks for /usr/bin/rygel but the start line starts /usr/local/sbin/rygeld  This is because we want to kill the real rygel process but start it with the script.

This setup works rather well. You do get some messages in the logfile about dbus not working but it is harmless. I tried disabling the mpris and external plugins but no matter what flag or configuration file option I tried, they would always try to start and fail with no dbus.

Rygel is a a reasonably light-weight way of serving media to your home network. It idles 200 MB virtual with 16MB resident and when idle uses almost no CPU.

Related articles

• Verizon planning Media Server for FiOS TV (techradar.com)
• Infographic: The Changing TV Landscape (readwriteweb.com)
• Evan Shapiro: TV: The Great Uniter? (huffingtonpost.com)
• Ubuntu Expands To TV, A Video Shows A Gorgeous Ubuntu TV Prototype In Action (essayboard.com)

As suggested in the multi arch implementation wiki page, I tested the libprocps0-dev package by compiling something against it, in this case another Debian package xmem. Doing this was very useful for teasing out some bugs on the dev package itself that did not appear while linking the library to the procps binaries.

In short, the new procps has a lot fewer patches than the old ones and the next version will have less as I have already included the current changes into the upstream git repository. The main differences are now

• freebsd linux version is read from a file not from uts
• includes use __restrict not the auto make defined restrict which may not be present in third party packages
• libnucrsesw conditionally linked with watch for 8bit watch

Related articles

• procps-ng version 3.3.2 released (enc.com.au)

Image via Wikipedia

It’s funny, in  bad way, when certain sorts of bugs come back to bite you.  People may remember the fun debian had with rpath issues around in the late 90s where some binaries wanted to set the path of where their libraries were located.  After much contraversy the consensus was at least in Debian (with some specific exceptions) RPATH is bad and don’t use it.  I lived through that “fun” and remember hacking libtool or autoconf or some such file.

The horror of those times came back to haunt me today while working on procps 3.3.2 packages.  lintian was suddenly complaining that binary-or-shlib-defines-rpath for all my binaries.  I’ve not seen rpath set for many years and as I’m also part of the upstream and never saw it set there it took me a while to work out what was going on.

Certainly, my binaries were setting the RPATH, it wasn’t lintian getting the wrong idea.  You can see this with objdump myfile | grep RPATH. If you get a hit, you got RPATH set, an empty result means no RPATH. libtool was definitely setting the rpath, with

cc -Iproc -g -O2 -o \$progdir/\$file pmap.o  ./proc/.libs/libprocps.so \
 -Wl,-rpath -Wl,/home/csmall/debian/procps/procps/proc/.libs \
 -Wl,-rpath -Wl,//usr/lib/x86_64-linux-gnu

That’s not one rpath setting but two, for double insult. Configure flags such as –disable-rpath wouldn’t work. I flicked back to a pure upstream archive and there was no rpath there in either the compile flags or objdump. There was obviously some sort of side-effect going on.

The problem, after much testing, was in the debian/rules file:

        ./configure \
          --build=$(DEB_BUILD_GNU_TYPE) \
          --enable-watch8bit --enable-w-from \
          --prefix=/ \
          --mandir=\$${prefix}/usr/share/man  \
          --libdir=\$${prefix}/usr/lib/$(DEB_HOST_MULTIARCH)

The problem is above, and all it was was a “/”. libtool does some checking to see if the library directory exists and if it doesn’t silently turns on rpath. It’s not smart enough to know that //usr/lib/x86_64-linux-gnu is the same as /usr/lib/x86_64-linux-gnu and so it enables rpath. Removing the slash before the usr fixed the problem.

I’d say this is bad behavour on libtools part. It should know that / is // or all the other alternative ways of specifying paths.  Hopefully this post will stop someone else wasting their time like I did.

NLS

Library Changes

Patch Backporting

debian Packages

Now, there seems only two approaches to this setup.  The first is to have the function stay in the library and for the program to call this function, just like all the other library functions.  But this means the library has exposed for all other programs that link to it and that doesn’t really make a lot of sense.

The second method is to have the functions defined in both places, perhaps in a file that is linked with the program and the library.  That seems to be duplicating the code in two different places.  Namespace collision could be a problem but that easily fixed with using some unqiue prefix.  I really don’t think having procfsmalloc, procfscalloc and procfs_strdup are that useful.

I’m sure this sort of problem has been solved elsewhere.  Has anyone else come across this? The functions are simple utility type functions such as malloc a block of memory, if there is a problem print to stderr.

pgrep crashes, pgrep -u not finding processes and a problem with top forest view have all been fixed.

An important addition to this release is that test scripts have now been added using the Dejagnu framework.  This framework tests most of the programs that ship with procps-ng.  There is only 100 tests so far and they are only tested for Linux architecture, but we expect to increase this test count and sophistication with each release.

The test scripts even found long-present but unknown bugs in the package. For example, running ps with output flags around the signal masks work as expected in Linux, but fail on kFreeBSD.  This is due to the unusual output of the relevant lines on that architecture. This is not a new bug, bug has been around for quite some time; noone ever noticed (or reported) it.

The Debian package has already been built and uploaded to the master FTP server.

Oh, and it is the first package with my new GPG key!

One regression that is on purpose, watch is currently not 8-bit clean again until I can sort out how to get the linking right with the new build process.

Testing Systems

The problem with packages like procps (psmisc too) is that you really need to test the entire program, not just a stub, and that the program needs access to a know level of /proc.  The only thing I have seen that is even remotely what is needed is the bunch of scripts that coreutils uses which creates dummy files and directories to operate the commands on.  I expect we could do something similar with some scripts that create a known process but if anyone has a better idea about how to test a command line program let me know.

While the upstream version is now released, the debian package is not quite ready to go as it involves removing a lot of patches and adjusting some others.  The build system is also now a reasonably standard (but different to previous) setup that means the packing scripts need adjusting.

The best place to find out what is different is the procps NEWS file. The source code is available from Gitorious  at https://gitorious.org/procps

Related articles

• Useful but Unknown Unix Tools: watch (noone.org)