Latest Blogs

Making ctypes Structures beautiful

Quick link to code: https://github.com/theY4Kman/pysmx/blob/master/smx/newstruct.py

Whenever I need to read in structured binary data in Python, I'm always frustrated at the disconnect between ctypes and struct. On the one hand, ctypes Structures provide a mechanism for naming bunches of binary data, but on the other hand, it doesn't provide an easy way to read in that data. The recommended method of reading in structured binary data is with struct.unpack, but unpack doesn't know or care about names, leaving extra work for us simple-minded programmers.

As a flagrant Django enthusiast, I want something like its Models:

class MyModel(models.Model):
published = models.BooleanField()
value = models.IntegerField()

After all, if data can be modeled like that in C, well, goddammit, it should be that easy in Python. With that in mind, I made it possible. I call it newstruct, and while the source may be a bit ugly now, using it is painless:

from ctypes import sizeof
from newstruct import *

class DataHeader(Struct):
address = cf_long()
typ = cf_byte()
tag = cf_short()

with open('mydata', 'rb') as fp:
header = DataHeader(fp.read(sizeof(DataHeader)))
print header.address, header.typ, header.tag

It's important to note that the attributes on DataHeader begin with "cf" (for "ctype field"). So if you want to use a ctypes c_long as a field in your Struct, you'd assign it cf_long(). The reason for the difference is Struct needs a custom field type to retain the ordering of fields.

I took my cues from Django's Field class, which increments a creation_counter variable on the Field class every time a new Field is instantiated. It then saves the current value of creation_counter to the new Field instance. Grabbing an ordered list of fields is a cinch sorting by creation_counter:

ordered_fields = sorted(fields, key=lambda field: field.creation_counter)

That's my module. Now, how can I make it better?

No Comments