The C language still surprises me

What does this code print?

1
2
3
4
5
6
7
8
9
10
11
12
#include <stdio.h>
 
int main()
{
    signed char x = -128;
 
    if (x < 0) {
        x = -x;
    }
    printf("x = %d\n", (int) x);
    return 0;
}

I’ve been reading Matters Computational, an excellent (free) book by Jörg Arndt about programming and algorithms. It surprised me in the very first pages with this pitfall in two’s complement – there is always a number that is equal to its own negative, besides zero. The code above prints −128!

In hindsight, it’s pretty obvious. A signed char can hold values from −128 to 127 — that is, there are 127 positive numbers and 128 negative numbers! Therefore, it’s impossible to the unary negative operator to be one-to-one. The smallest negative number will always be mapped to itself. Of course, this also applies to int, etc.

The main implication of this fact is that, after the innocent-looking code

if (x < 0) x = -x;

x is not guaranteed to be positive!

The black magic of GDI+

One of the things I am most ashamed of on Quivi is its speed when opening large images (which are not uncommon nowadays, specially with digital photos). It’s embarrassing that the lame Windows Picture and Fax Viewer is lightning fast when opening those images!

I’ve always wondered how the Viewer did that. I’ve searched about it in the past but could never find anything about it. Then one of these days I was browsing the Wikipedia page about the Viewer and there I learned that it uses GDI+.

GDI+ is a C++ library, but it is built upon a flat C API which I could easily use in Python via ctypes. Long story short, I was able to modify Quivi to add support for viewing images with GDI+. And the result was amazing!

What about Linux? Well, something “equivalent” to GDI+ would be Cairo, so I did some tests with it too (luckily, support for it was included in wxPython recently).

Here are the results for the time to load a huge 5704px x 5659px PNG image and rescale it to 1/20 of its size:

Time to load and scale a PNG image
Library Time (s)
FreeImage 10.38
PIL 9.90
GDI+ 2.22
Cairo (from FreeImage) 3.60
Cairo (direct) 3.28

The reason for the two Cairo timings is that it supports reading directly from a PNG file but for e.g. JPG files you need to read the image with another library and to the format Cairo uses. I’ve used FreeImage to load the image and converted it to a Cairo surface.

Here are the results for the time to load and scaling the same image, but as a JPG:

Time to load and scale a JPG image
Library Time (s)
FreeImage 6.91
PIL 8.38
GDI+ 0.19
Cairo (from FreeImage) 1.43

Scary! How does GDI+ manages to do that? According to Wikipedia it uses hardware acceleration… Cairo doesn’t lag begind considering that the scaling only takes 0.015 (!) but I did notice that even its best quality scaling isn’t so good in comparison to the others, which is kinda odd.

Anyway, I’ll try to release a new version of Quivi with GDI+ and Cairo support soon. Stay tuned.

Access violation errors with callbacks in ctypes

I’ve just spent a few hours trying to solve this bug, so I’m publishing this so maybe it will help someone with this issue…

Assume that you’re working with a DLL/.so library through ctypes in Python, and this library allows you to set a callback for some other function. In my case, I was working with unrar.dll. The code was something among these lines:

UNRARCALLBACK = ctypes.WINFUNCTYPE(ctypes.c_int, ctypes.c_uint, ctypes.c_long, ctypes.c_long, ctypes.c_long)
 
#in a class...
RARSetCallback(self.handle, UNRARCALLBACK(self.callback_fn), 0)
RARProcessFile(self.handle, RAR_TEST, None, None)

The first lines constructs the function prototype, the second sets the callback in a function of the DLL file, and the third calls a function in the DLL which will call the callback.

Can you spot the error?

The code worked fine in Python 2.5, but then I changed to 2.6 and it stopped working. I got a “WindowsError: exception: access violation reading…” (or writing) exception in the third call.

The reason, which is obvious in hindsight, is cleared explained in the docs:

Make sure you keep references to CFUNCTYPE objects as long as they are used from C code. ctypes doesn’t, and if you don’t, they may be garbage collected, crashing your program when a callback is made.

(Though it’s not explicit, it applies to WINFUNCTYPE objects too)

The WINFUNCTYPE object created in the second line no longer exists in the third line, so when the callback was called, it no longer pointed to a valid address. The solution is simple — just keep a reference to the object:

UNRARCALLBACK = ctypes.WINFUNCTYPE(ctypes.c_int, ctypes.c_uint, ctypes.c_long, ctypes.c_long, ctypes.c_long)
 
#inside a class...
self.callback_ref = UNRARCALLBACK(self.callback_fn)
RARSetCallback(self.rarFile.RAR._handle, self.callback_ref, 0)
RARProcessFile(self.rarFile.RAR._handle, RAR_TEST, None, None)

The only mystery left is why the old code worked on 2.5!

Why I rewrote Quivi from scratch

Joel Spolsky, popular software engineering, said in this blog:

(…) They [Netscape] did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

I agree, mostly. Many projects market that they’ve been “rewritten from scratch” as if it was something marvelous, and most of time, it’s just a sign that the new version probably has more bugs than the previous. But, wait, I just rewrote Quivi (a image viewer and comic / manga reader) from scratch! Well, why?

Well, I thought I really had to justify this. So here are the reasons:

  • It is a small project. Rewriting it certainly isn’t as hard as rewriting a browser!
  • I couldn’t stand C++ anymore (the programming language I used before). Of course, this isn’t C++ fault per se. It has its uses — the right tool for the right job, and so on. But to write an desktop application? It’s overkill. If you manage to pull it of, hey, kudos to you. But I had no motivation to work on it anymore.
  • I love Python (the programming language I use now), and of course, I’m not the only one. Programming is fun in Python, so much that had the motivation to rewrite Quivi from scratch in the first place! And of course, it will be much more easier for me to keep working on Quivi.
  • I could change GUI libraries. SmartWin is a nice library, and uses templates in very interesting ways; when programming Quivi I ended up involved with its development too. But it has its bugs, and because it uses templates extensively, it’s awful slow to compile an application that uses it. I’ve changed to wxPython, which is a very mature GUI library – and cross platform to boot.

Of course, there are some downsides with the change. The whole software package is much bigger due to the dependencies (the installer jumped from 900K to 5MB!). And the program is a little bit slower, mainly when starting up, and uses more memory (9MB to 30MB with no images loaded). But I think it was a good enough trade-off.

Rewriting from scratch must be considered carefully, and in this case, I think it was a good idea. Quivi is a hobby project, and I guess the main point of it is to have fun writing it, and to make users happy. I hope I can do both with this new version (which will be released soon).