I’ve just spent a few hours trying to solve this bug, so I’m publishing this so maybe it will help someone with this issue…
Assume that you’re working with a DLL/.so library through ctypes in Python, and this library allows you to set a callback for some other function. In my case, I was working with unrar.dll. The code was something among these lines:
UNRARCALLBACK = ctypes.WINFUNCTYPE(ctypes.c_int, ctypes.c_uint, ctypes.c_long, ctypes.c_long, ctypes.c_long) #in a class... RARSetCallback(self.handle, UNRARCALLBACK(self.callback_fn), 0) RARProcessFile(self.handle, RAR_TEST, None, None)
The first lines constructs the function prototype, the second sets the callback in a function of the DLL file, and the third calls a function in the DLL which will call the callback.
Can you spot the error?
The code worked fine in Python 2.5, but then I changed to 2.6 and it stopped working. I got a “WindowsError: exception: access violation reading…” (or writing) exception in the third call.
The reason, which is obvious in hindsight, is cleared explained in the docs:
Make sure you keep references to CFUNCTYPE objects as long as they are used from C code. ctypes doesn’t, and if you don’t, they may be garbage collected, crashing your program when a callback is made.
(Though it’s not explicit, it applies to WINFUNCTYPE objects too)
The WINFUNCTYPE object created in the second line no longer exists in the third line, so when the callback was called, it no longer pointed to a valid address. The solution is simple — just keep a reference to the object:
UNRARCALLBACK = ctypes.WINFUNCTYPE(ctypes.c_int, ctypes.c_uint, ctypes.c_long, ctypes.c_long, ctypes.c_long) #inside a class... self.callback_ref = UNRARCALLBACK(self.callback_fn) RARSetCallback(self.rarFile.RAR._handle, self.callback_ref, 0) RARProcessFile(self.rarFile.RAR._handle, RAR_TEST, None, None)
The only mystery left is why the old code worked on 2.5!
Hah, I just got bit by this too! Thanks for pointing out the obvious.
Thank you for this post ! I also solved my bug ! It was also working on 2.5 but stopped working on 2.6. I think it is stricter in garbage collection. Keeping a global or instance reference to the callback solves the problem.
Thanks for keeping me from going crazy — very useful! Makes me remember how frustrating programming before google used to be 🙂
Thank you for this post 🙂 I got bit by this bug earlier and nearly pulled my hair out.
Thank you very much, this thing has drove me crazy the last 3 hours…
An old post, but very usefull.
Ah, so good to read this after a couple hours of frustration.