Codecs

Most of the codec related code is taken and adapted from aiomas: https://gitlab.com/sscherfke/aiomas/

Codecs enable the container to encode and decode known data types to send them as messages. Mango already contains two codecs: A json serializer that can (recursively) handle any json serializable object and a protobuf codec that will wrap an object into a generic protobuf message. Other codecs can be implemented by inheriting from the Codec base class and implementing its encode and decode methods. Codecs will only handle types explicitely known to them. New known types can be added to a codec with the add_serializer method. This method expects a type together with a serialization method and a deserialization method that translate the object into a format the codec can handle (for example a json-serializable string for the json codec).

Warning

When using the json codec certain types can not be exactly serialized and deserialized between containers. One example are tuple and classes derived from it like namedtuple. The core of the json codec uses pythons json encoder [1] for any type that this encoder can handle by itself. Tuples are translated to json arrays without any further information by this encoder. Consequently, a receiving container will only see a json array and deserialize it to a python list.

[1]: https://docs.python.org/3/library/json.html#json.JSONEncoder

Quickstart

general use

Consider a simple example class we wish to encode as json:

class MyClass:
    def __init__(self, x, y):
        self.x = x
        self._y = y

    @property
    def y(self):
        return self._y

    def __asdict__(self):
        return {"x": self.x, "y": self.y}

    @classmethod
    def __fromdict__(cls, attrs):
        return cls(**attrs)

    @classmethod
    def __serializer__(cls):
        return (cls, cls.__asdict__, cls.__fromdict__)

If we try to encode an object of MyClass without adding a serializer we get an SerializationError:

from mango import JSON, SerializationError

codec = JSON()

my_object = MyClass("abc", 123)
try:
    encoded = codec.encode(my_object)
except SerializationError as e:
    print(e)

No serializer found for type "<class 'MyClass'>"

We have to make the type known to the codec to use it:

codec = JSON()
codec.add_serializer(*MyClass.__serializer__())

my_object = MyClass("abc", 123)
encoded = codec.encode(my_object)
decoded = codec.decode(encoded)

print(my_object.x, my_object.y)
print(decoded.x, decoded.y)

abc 123
abc 123

The codec distinguishes different types for decoding by assigning a type id (32 bit integer) to the type. The type id can either be automatically generated (like above) or explicitely set when adding the serializer. Note that if you set a type_id yourself you need to ensure that the decoding container associated the same id with the desired type.

codec = JSON()
codec.add_serializer(*MyClass.__serializer__(), type_id=4711)

my_object = MyClass("abc", 123)
encoded = codec.encode(my_object)
decoded = codec.decode(encoded)

print(my_object.x, my_object.y)
print(decoded.x, decoded.y)

abc 123
abc 123

All that is left to do now is to pass our codec to the container. This is done during container creation in the create_container method.

from mango import Agent, create_tcp_container, activate
import asyncio

class SimpleReceivingAgent(Agent):
    def __init__(self):
        super().__init__()

    def handle_message(self, content, meta):
        if isinstance(content, MyClass):
            print(content.x)
            print(content.y)


async def main():
    codec = JSON()
    codec.add_serializer(*MyClass.__serializer__())

    # codecs can be passed directly to the container
    # if no codec is passed a new instance of JSON() is created
    sending_container = create_tcp_container(addr=("127.0.0.1", 5556), codec=codec)
    receiving_container = create_tcp_container(addr=("127.0.0.1", 5555), codec=codec)
    receiving_agent = receiving_container.register(SimpleReceivingAgent())

    async with activate(sending_container, receiving_container):
        # agents can now directly pass content of type MyClass to each other
        my_object = MyClass("abc", 123)
        await sending_container.send_message(
            content=my_object, receiver_addr=receiving_agent.addr
        )
        await asyncio.sleep(0.1)

asyncio.run(main())

abc
123

@json_serializable decorator

In the above example we explicitely defined methods to (de)serialize our class. For simple classes, especially data classes, we can achieve the same result (for json codecs) via the mango.json_serializable`() decorator. This creates the __asdict__, __fromdict__ and __serializer__ functions in the class:

from mango import json_serializable, JSON

@json_serializable
class DecoratorData:
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z

codec = JSON()
codec.add_serializer(*DecoratorData.__serializer__())

my_data = DecoratorData(1,2,3)
encoded = codec.encode(my_data)
decoded = codec.decode(encoded)

print(my_data.x, my_data.y, my_data.z)
print(decoded.x, decoded.y, decoded.z)

1 2 3
1 2 3

fast json

Besides the normal full features json codec, which is able to serialize and deserialize messages under preservation of the type information, mango provides the codecs.FastJson codec. This codec usese msgspec and does not provide any type safety. Therefore are also no custom serializer.

proto codec and ACLMessage

Serialization methods for the proto codec are expected to encode the object into a protobuf message object with the SerializeToString method. The codec then wraps the message into a generic message wrapper, containing the serialized protobuf message object and a type id. This is necessary because in general the original type of a protobuf message can not be infered from its serialized form.

The ACLMessage class is encouraged to be used for fipa compliant agent communication. For ease of use it gets specially handled in the protobuf codec: Its content field may contain any proto object known to the codec and gets encoded with the associated type id just like a non-ACL message would be encoded into the generic message wrapper.