Abstract root class for field containers.
API reference: Container
Packets can be seen as field containers. That is, a packet is formed by a sequence of fields. The Container class provides this vision. A Container is also a Field itself. Therefore, a Container might also accomodate other Containers.
Consider the first three bytes of the IP header:
| version | hlen | tos | length |
|---|---|---|---|
| 4 bits | 4 bits | 1 byte | 2 bytes |
We can see the IP header as a Container with a sub-Container holding two bit fields (version and hlen) and two additional fields (tos and length).
The Container class is just an abstract class that allows adding fields. That is, it provides the base methods to build Containers.
A container implementation for bit fields.
API reference: BitStructure
The BitStructure class must be used, in conjunction with BitField, to create byte-aligned fields formed, internally, by bit fields.
It is really important to understand that BitPacket is byte oriented, therefore, a BitStructure must be byte-aligned.
Consider the first byte of an IP header packet:
| version | hlen |
|---|---|
| 4 bits | 4 bits |
This packet can be constructed as:
>>> ip = BitStructure("IP")
The line above creates an empty structure named IP. Now, we need to add fields to it. As BitStructure is a Container subclass the Container.append() function can be used:
>>> ip.append(BitField("version", 4, 0x0E))
>>> ip.append(BitField("hlen", 4, 0x0C))
>>> print ip
(IP =
(version = 0x0E)
(hlen = 0x0C))
Note that the size of a BitStructure is returned in bytes. Remember that the purpose of a BitStructure is to create a byte-aligned value that is built internally with bits:
>>> ip.size()
1
BitStructure fields can be obtained as in a dictionary, and as in any Container subclass. Following the last example:
>>> ip["version"]
14
>>> ip["hlen"]
12
As with any BitPacket field, packing a BitStructure is really simple. Considering the IP header exampe above we can easily create an array of bytes with the contents of the structure:
>>> ip_data = array.array("B")
>>> ip.array(ip_data)
>>> print ip_data
array('B', [236])
Or also create a string of bytes from it:
>>> ip.bytes()
'\xec'
To be able to unpack an integer value or a string of bytes into a BitStructure, we only need to create the desired structure and assign data to it.
>>> bs = BitStructure("mypacket")
>>> bs.append(BitField("id", 8))
>>> bs.append(BitField("address", 32))
>>> print bs
(mypacket =
(id = 0x00)
(address = 0x00000000))
So, now we can unpack the following array of bytes:
>>> data = array.array("B", [0x38, 0x87, 0x34, 0x21, 0x40])
into our previously defined structure:
>>> bs.set_array(data)
>>> print bs
(mypacket =
(id = 0x38)
(address = 0x87342140))
Also, new data can also be unpacked (old data will be lost):
>>> data = array.array("B", [0x45, 0x67, 0x24, 0x98, 0xFB])
>>> bs.set_array(data)
>>> print bs
(mypacket =
(id = 0x45)
(address = 0x672498FB))
A container implementation for fields of different types.
API reference: Structure
The Structure class provides a byte-aligned Container implementation. This means that all the fields added to a Structure should be byte-aligned. This does not mean that a BitField can not be added, but if added, it should be added within a BitStructure. This is because bit and byte processing is done differently, and that’s why BitStructure was created.
Consider the first three bytes of the IP header:
| version | hlen | tos | length |
|---|---|---|---|
| 4 bits | 4 bits | 1 byte | 2 bytes |
For simplicity, we can create only a Structure with the last two fields, tos and length.
>>> ip = Structure("IP")
The line above creates an empty packet named ‘IP’. Now, we can add the two fields to it with an initial value:
>>> ip.append(UInt8("tos", 3))
>>> ip.append(UInt16("length", 146))
>>> print ip
(IP =
(tos = 3)
(length = 146))
Structure fields, as in any other Container, can be obtained like in a dictionary, that is, by its name. Following the last example:
>>> ip["tos"]
3
>>> ip["length"]
146
As with any other BitPacket field, packing a Structure is really simple. Considering the IP header exampe above, we can easily create an array of bytes with the contents of the structure:
>>> ip_data = array.array("B")
>>> ip.array(ip_data)
>>> print ip_data
array('B', [3, 0, 146])
Or also create a string of bytes from it:
>>> ip.bytes()
'\x03\x00\x92'
To be able to unpack an integer value or a string of bytes into a Structure, we only need to create the desired packet and assign data to it.
>>> bs = Structure("mypacket")
>>> bs.append(UInt8("id"))
>>> bs.append(UInt32("address"))
>>> print bs
(mypacket =
(id = 0)
(address = 0))
So, now we can unpack the following array of bytes:
>>> data = array.array("B", [0x38, 0x87, 0x34, 0x21, 0x40])
into our previously defined structure:
>>> bs.set_bytes(data.tostring())
>>> print bs
(mypacket =
(id = 56)
(address = 2268340544))
An interesting use of structures is to subclass them to create our own reusable ones. As an example, we could create the structure defined in the previous section as a new class:
>>> class MyStructure(Structure):
... def __init__(self, id = 0, address = 0):
... Structure.__init__(self, "mystructure")
... self.append(UInt8("id", id))
... self.append(UInt32("address", address))
...
... def id(self):
... return self["id"]
...
... def address(self):
... return self["address"]
...
>>> ms = MyStructure(0x33, 0x50607080)
>>> print ms
(mystructure =
(id = 51)
(address = 1348497536))
We can now use the accessors of our class to print its content:
>>> print "0x%X" % ms.id()
0x33
>>> print "0x%X" % ms.address()
0x50607080
An structure for fields of the same type.
API reference: Array
Sometimes we need to create packets that have a number of repeated fields in it. Normally, these kind of packets have a counter field indicating the number of repeated fields after it.
An Array is a subclass of Structure. Initially, it contains a length field, which is the one that will indicate how many fields the array holds. The type of the length field is specified in the Array constructor.
| count | id | address |
|---|---|---|
| 1 byte | 1 byte | 4 bytes |
| count times | ||
In order to create an Array for the depicted packet above, we can define the base type of the fields (all of the same type) that this array will contain. We will create a MyStructure class that contains two fields, id and address.
>>> class MyStructure(Structure):
... def __init__(self, name = "mystructure", id = 0, address = 0):
... Structure.__init__(self, name)
... self.append(UInt8("id", id))
... self.append(UInt32("address", address))
Now, we can define an Array that contains the default counter field of our desired name and size and a single argument function that tells how to create the array fields.
>>> packet = Array("mypacket", UInt8("counter"),
... lambda root: MyStructure())
As a second argument to the Array constructor, we specify the field that specifies how many elements the array contains. As the third argument, we have provided how to create the array elements. The anonymous function takes an argument which is the top-level root Container that the Array belongs to.
So, let’s try to unpack some data and see what happens:
>>> data = array.array("B", [0x01, 0x54, 0x10, 0x20, 0x30, 0x40])
>>> packet.set_array(data)
>>> print packet
(mypacket =
(counter = 1)
(0 =
(id = 84)
(address = 270544960)))
At this point the array contains one field of type MyStructure as it has unpacked the given array, seen that the counter field had value 1 and therefore read one MyStructure field. It is worth noting that the fields added to an Array are automatically named in a zero-based scheme. That is, to access the first address field value we could do:
>>> packet["0.address"]
270544960
We can also easily add some data to the array. Consider again our packet:
>>> packet = Array("mypacket", UInt8("counter"),
... lambda root: MyStructure())
Adding a MyStructure field is as easy as adding a field to any other Structure:
>>> packet.append(MyStructure("foo", 54, 98812383))
>>> print packet
(mypacket =
(counter = 1)
(0 =
(id = 54)
(address = 98812383)))
It is interesting to see that if we add something else that is not a MyStructure, a TypeError exception will be raised notifying about the problem:
>>> try:
... packet.append(UInt8("wrong", 12))
... except TypeError as err:
... print "Error: %s" % err
Error: Invalid field type for array 'mypacket' (expected <class 'MyStructure'>, got <class 'BitPacket.Integer.UInt8BE'>)
Array fields, as in any other Container can be obtained like in a dictionary, that is, by its name. Following the last example:
>>> packet["0.id"]
54
Note that the id field could be another array instead of a numeric field, thus we could access further by using the dot field separator (.).
We can also build more complex packets, such as the one below, where we have one Array inside another.
| count1 | id | count2 | address |
|---|---|---|---|
| 1 byte | 1 byte | 1 byte | 4 bytes |
| count2 times | |||
| count1 times | |||
We will first create a structure for the list of addresses. It will contain the count2 counter and an Array whose number of elements is provided by count2 and that will be filled with 32-bit unsigned integers.
>>> class AddressList(Structure):
... def __init__(self):
... Structure.__init__(self, "addresslist")
... self.append(UInt8("id"))
... self.append(Array("address",
... UInt8("count2"),
... lambda root: UInt32("value")))
Now, we can build our packet as an structure with the count1 counter and an Array whose number of elements is provided by count1 and that will be filled by address lists (that, remember, already has another Array).
>>> s = Array("mypacket",
... UInt8("count1"),
... lambda root: AddressList())
So, let’s try to set some data to this packet. As we have seen before with the simplest case, data should be propagated and Array meta properties will be used to build the desired fields.
>>> s.set_array(array.array("B", [0x02, # count1
... 0x01, # id (1)
... 0x01, # count2 (1)
... 0x01, 0x02, 0x03, 0x04,
... 0x02, # id (2)
... 0x02, # count2 (2)
... 0x05, 0x06, 0x07, 0x08,
... 0x09, 0x0A, 0x0B, 0x0C]))
>>> print s
(mypacket =
(count1 = 2)
(0 =
(id = 1)
(address =
(count2 = 1)
(0 = 16909060)))
(1 =
(id = 2)
(address =
(count2 = 2)
(0 = 84281096)
(1 = 151653132))))
It works! As we see, our packet consists of a mystructure that contains two AddressList fields. The first one with a single address and the second with two.
An structure that holds a String and its length.
API reference: Data
A Data field lets you store a string of characters (divided by words) and keeps its length in another field. By default, the size of a word is 1 byte. Basically, Data is a Structure with two fields in this order: length and data (internally named Data). The length is a numeric field and specifies how many words the Data field contains.
In the next example we create a Data field with six characters and a length field of 1 byte (thus, a maximum of 255 characters can be hold):
>>> data = Data("data", UInt8("Length"));
>>> data.set_value("abcdef")
>>> print data
(data =
(Length = 6)
(Data = 0x616263646566))
We can easily get back the six characters by creating the string again:
>>> "".join(data["Data"])
'abcdef'
Note that, above, “print data” returns a human-readable string with hexadecimal values and “data.value()” returns the actual string.
The length field tells us how many words the Data field contains. Above, we just saw an example with the default word size of 1. But a 12 character string and a word size of 4, gives us 3 words.
>>> data = Data("data", UInt8("Length"), 4);
>>> data.set_value("abcdefghigkl")
>>> print data
(data =
(Length = 3)
(Data = 0x616263646566676869676B6C))
Note that data length needs to be a multiple of the word size.
It is also possible to obtain the word size from the value of another field.
| WSize | Length | Data |
|---|---|---|
| 1 byte | 1 byte | Length * WSize |
Thus, instead of passing a number to the word size parameter, we pass it a single-argument function. The single-argument, as in all other BitPacket fields, is the top-level root Container field where the Data field belongs to.
>>> packet = Structure("packet")
>>> packet.append(UInt8("WSize"))
>>> data = Data("data", UInt8("Length"), lambda root: root["WSize"]);
>>> packet.append(data)
>>> buffer = array.array("B", [2, 3, 40, 55, 22, 45, 34, 89])
>>> packet.set_array(buffer)
>>> print packet
(packet =
(WSize = 2)
(data =
(Length = 3)
(Data = 0x2837162D2259)))