Python3, Using some shared state in 2 async methods

June 28, 2018 2 By addshore

Python logoThe asyncio module was added to Python in version 3.4 to “provides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives”. If you want an introduction to asyncio take a look at this blog post.

This post will look at how to share state between multiple async methods that may be running at the same time without using anything silly like globals. The examples should be easy to follow and easy to extrapolate from. Also I’ll be embedding trinket.io  code snippets which you should be able to run in the browser.

Disclaimer: I don’t really write much python so there might be better solutions. This post is only meant to serve as a consolidation of knowledge to be passed to another.

One way to run 2 async methods at the same time in python and wait for them to complete is to use the asyncio.gather method.

For example to run methodOne() and methodTwo() at the same time and wait for them both to complete the code below could be used:

await asyncio.gather(methodOne(), methodTwo())

You might want to pass some data to each of these methods, for example a counter:

counter = 0
should_continue = true
await asyncio.gather(methodOne(should_continue,counter), methodTwo(should_continue,counter))

But in the above example both counter and should_continue will not be able to be altered from within methodOne and methodTwo as they are not passed by reference, but instead by value.

From https://stackoverflow.com/a/430958/4746236:

When a parameter is passed by reference, the caller and the callee use the same variable for the parameter. If the callee modifies the parameter variable, the effect is visible to the caller’s variable.

When a parameter is passed by value, the caller and callee have two independent variables with the same value. If the callee modifies the parameter variable, the effect is not visible to the caller.

One way of getting around this would be to use globals, but globals are evil, take a look at this stackoverflow post for more details.

A working example of a simple counter program using global to share state can be seen below:

A better way to do this would be to use some sort of object to hold the state required.

Objects are passed by reference and simple values such as ints and bools are passed by value.

You can create an empty object using the code below, and an explanation for the code can be found at https://stackoverflow.com/a/19476841/4746236

b = type('', (), {})()
b.this_works = 'cool'

This can be seen working below where an integer passed to a method and modified does not have the modification reflected in the outer scope, but the object containing an integer when modified is reflected in the outer scope.

Converting the above counter example using globals to use some sort of state object is pretty easy.

Such state objects can be as complex or as simple as needed.