March 4, 2012
No Comments
JFFNMS version 0.9.2 was released today both as an upstream tar.gz file and a new debian package. This version fixes some bugs including making sure it works with PHP5.4.
The biggest change in PHP 5.4 is that you can no longer call by reference. Previously you could call a function like myfunc(&blah); which would send a pointer to blah and not the item itself. Now the function definition needs to define what it wants rather than change it each time.
January 27, 2012
2 Comments
Sometimes life throws little curves at you to see if you are still awake, today has been one of those days.
fglrx is (apparently) fixed
I’ve had a long-running problem with fglrx on my laptop. The problem stems from
ATI closed-source drivers with one of those laptops that has an ATI and Intel driver. It means I am basically using the slow Intel chip only. This morning I had enough and backed up my home and started to rebuild the laptop with
debian 6.0.3.
So I kicked off the very very slow process of reformatting the crypto drive (it has taken 5 hours and still going) let it gurgle on its merry way and started to read my email. One of the emails was that my bug about fglrx not working is closed, apparently it is fixed. If I had read that 10 minutes earlier, a simple ‘apt-get install fglrx-driver‘ would of perhaps fixed it; oh well.
My problem is now is do I move to the latest driver and hope their fix is my fix or leave it with some ancient version? My preference is the former; I only hope it works!
psmisc has a program called pstree which prints the set of processes in a tree fashion. It hasn’t changed much for quite a while. I released version 22.15 and the Debian package 22.15-1. 22.15-1 I also adopted the
harden CFLAGS as suggested for procps.
I was a little surprised that I received an important bug. The report was saying I had a buffer overflow introduced in 22.15-1, but no relevant code had changed. The compiler options had done their job and stopped a buffer being overflowed.
But where exactly was the overflow? Running gdb on pstree quickly showed that it was line 267 of pstree.c which uses strcpy(). That function set off warning bells. The relevant code is:
PROC *new;
if (!(new = malloc(sizeof(PROC)))) {
perror("malloc");
exit(1);
}
strcpy(new->comm, comm);
Now comm is the short command name you find in /proc//stat. It is fixed in the kernel at 16 characters. The PROC structure has this field as 17 characters long, one extra for the NUL. I went and checked the Linux source and yes, it is still 16 characters long. The clue was in the name of the program that it died on.
#6 new_proc (comm=0x6111b0 "{console-kit-dae}", pid=1571, uid=0)
at pstree.c:267
That string is 17 characters long. The problem is that 16 characters is for the name only. If the name is in brackets or braces, then that 16 character limit doesn’t apply. The buffer overflow bug has been there for a long time, but only with the compiler flags did it become visible.
Given you need to read names out of the /proc filesystem and if someone can fiddle with that you have bigger problems it doesn’t seem to be too much of an issue. It should be (and is in Debian 22.15-2) fixed but is a nice example of the compiler catching bad things.
January 10, 2012
No Comments
Following up from the upstream release of a new procps, the debian packages have also been updated. This upload has a significant change in that, I hope, procps is now multi-arch compliant. To make this happen, the libprocps library is now in it’s own package, separate from the binaries. It also means that if you have programs not from procps that link to this library they are now broken. I put in a Breaks: line for the three I know about (xmem, guymager and open-vm-tools) which will need a recompile with a small tweak in the control file and linked statements.
As suggested in the multi arch implementation wiki page, I tested the libprocps0-dev package by compiling something against it, in this case another Debian package xmem. Doing this was very useful for teasing out some bugs on the dev package itself that did not appear while linking the library to the procps binaries.
In short, the new procps has a lot fewer patches than the old ones and the next version will have less as I have already included the current changes into the upstream git repository. The main differences are now
- freebsd linux version is read from a file not from uts
- includes use __restrict not the auto make defined restrict which may not be present in third party packages
- libnucrsesw conditionally linked with watch for 8bit watch
The three are really bugs, especially the last, which is why the patches will disappear next release.
January 7, 2012
4 Comments
Image via Wikipedia
It’s funny, in bad way, when certain sorts of bugs come back to bite you. People may remember the fun debian had with rpath issues around in the late 90s where some binaries wanted to set the path of where their libraries were located. After much contraversy the consensus was at least in Debian (with some specific exceptions) RPATH is bad and don’t use it. I lived through that “fun” and remember hacking libtool or autoconf or some such file.
The horror of those times came back to haunt me today while working on procps 3.3.2 packages. lintian was suddenly complaining that binary-or-shlib-defines-rpath for all my binaries. I’ve not seen rpath set for many years and as I’m also part of the upstream and never saw it set there it took me a while to work out what was going on.
Certainly, my binaries were setting the RPATH, it wasn’t lintian getting the wrong idea. You can see this with objdump myfile | grep RPATH. If you get a hit, you got RPATH set, an empty result means no RPATH. libtool was definitely setting the rpath, with
cc -Iproc -g -O2 -o \$progdir/\$file pmap.o ./proc/.libs/libprocps.so \
-Wl,-rpath -Wl,/home/csmall/debian/procps/procps/proc/.libs \
-Wl,-rpath -Wl,//usr/lib/x86_64-linux-gnu
That’s not one rpath setting but two, for double insult. Configure flags such as –disable-rpath wouldn’t work. I flicked back to a pure upstream archive and there was no rpath there in either the compile flags or objdump. There was obviously some sort of side-effect going on.
The problem, after much testing, was in the debian/rules file:
./configure \
--build=$(DEB_BUILD_GNU_TYPE) \
--enable-watch8bit --enable-w-from \
--prefix=/ \
--mandir=\$${prefix}/usr/share/man \
--libdir=\$${prefix}/usr/lib/$(DEB_HOST_MULTIARCH)
The problem is above, and all it was was a “/”. libtool does some checking to see if the library directory exists and if it doesn’t silently turns on rpath. It’s not smart enough to know that //usr/lib/x86_64-linux-gnu is the same as /usr/lib/x86_64-linux-gnu and so it enables rpath. Removing the slash before the usr fixed the problem.
I’d say this is bad behavour on libtools part. It should know that / is // or all the other alternative ways of specifying paths. Hopefully this post will stop someone else wasting their time like I did.
January 6, 2012
1 Comment
procps-ng version 3.3.2 was released today. This version fixes some bugs introduced in version 3.3.1 as well as a number of enhancements. Below is the most significant set of changes that 3.3.2 brings.
NLS
The most visible change is that procps-ng is now international. The NLS changes took a long windy path but we got there in the end. This means all procps-ng programs can now use standard gettext PO files to output in any supported language. While the programs have been enabled for translations, there are no po files as yet but we expect them to follow soon.
Library Changes
procps always had a “closed” library, meaning that it wasn’t supposed to be used for other non-procps programs. This meant the library was always called libprocps-(version) rather than using a SONAME. Procps 3.3.2 now uses a SONAME of version 0.0.0 The API hasn’t changed but it will be in subsequent versions.
Patch Backporting
Due to the stagnant natute of procps development in the past, there have been a large number of patches each distribution carries for procps. A significant number of patches have been incorportated into procps-ng, giving a more consistent look across the distributions and meaning any subsquent fixes or enhancements are done in one place. A major goal of procps-ng was to reduce the number of distribution specific patches which this change has helped greatly.
The Debian packages will be worked on soon and will clear up some confusion about the procps binaries and library packages as this will now be split into two. It also means that some programs that depend on libprocps will break, but going on from this point will be able to use the normal shlibs process to manage that, rather than the ugly crunch we will have now.
December 6, 2011
No Comments
Procps-ng, the debian, Fedora and OpenSuSE fork of procps had another release today. This is a bugfix release that fixes some important bugs that have cropped up in 3.3.0
pgrep crashes, pgrep -u not finding processes and a problem with top forest view have all been fixed.
An important addition to this release is that test scripts have now been added using the Dejagnu framework. This framework tests most of the programs that ship with procps-ng. There is only 100 tests so far and they are only tested for Linux architecture, but we expect to increase this test count and sophistication with each release.
The test scripts even found long-present but unknown bugs in the package. For example, running ps with output flags around the signal masks work as expected in Linux, but fail on kFreeBSD. This is due to the unusual output of the relevant lines on that architecture. This is not a new bug, bug has been around for quite some time; noone ever noticed (or reported) it.
The Debian package has already been built and uploaded to the master FTP server.
Oh, and it is the first package with my new GPG key!
November 21, 2011
5 Comments
Coming soon after the upstream procps 3.3.0 being released, the debian packages have been uploaded tonight. The Debian packages have had the added benefit of being a day late by having a tiny 2 line patch to stop pgrep from crashing.
One regression that is on purpose, watch is currently not 8-bit clean again until I can sort out how to get the linking right with the new build process.
Testing Systems
The embarrassing problem with pgrep did start a discussion about testing, especially
regression testing. I’ve recently started using
unit testing in my python programs and love the level of assurance those tests give me that I haven’t broken anything (to a degree anyhow). I’d really like to be able to type “make test” and have each of the programs run through a series of tests.
The problem with packages like procps (psmisc too) is that you really need to test the entire program, not just a stub, and that the program needs access to a know level of /proc. The only thing I have seen that is even remotely what is needed is the bunch of scripts that coreutils uses which creates dummy files and directories to operate the commands on. I expect we could do something similar with some scripts that create a known process but if anyone has a better idea about how to test a command line program let me know.
September 30, 2011
3 Comments
The new ncurses library that is currently sitting in the NEW queue will have a significant change that, if we’ve done it right, should be an invisible change. If we’ve got it wrong, then the BTS (and us) may be a little busy.
What has happened is that the low-level terminfo library has been split out from the general ncurses library making a new package called libtinfo5. This also means that there only needs to be one libtinfo5 package because at this low level it doesn’t care about wide characters, so there is no libtinfo5w (and that gets important later). libncurses is linked to libtinfo so from the point of view of using libncurses there should be no major change in how things are done.
So, why bother?
The way ncurses was built before causes a nice long “conga line of trouble”. There really are (was) two main ncurses library packages, libncurses5 and libncursesw5. The w meant it was for wide characters. So anything that uses ncurses needs to decide which one to link to or perhaps both. A problem came up with the readline library that needs to link to ncurses because programs that link to IT (such as python) want a wide library but other programs do not. One solution is to have two readline libraries, but then any other library would need to be compiled and built twice as well and so on and so on.
readline didn’t really want the full ncurses library, it only wants the terminfo part but there was (until now) no way of picking up that bit only. Once this new version of ncurses gets released then readline can link to libterminfo which is agnostic to wide and non-wide characters which means it is as well which them means everything else should, in theory work. If you are using ncurses for some low-level terminfo work only, you may want to have a look to see if you can link to libtinfo5 only which will mean depending on libtinfo5 and not libncurses5 or libncursesw5.
I’m just writing this up mainly and uploading the packages, the bulk of the work was done by Sven and Matthias as well as some help from our ever-helpful upstream Thomas.
