Hi everyone, this is Abdul from Pythonist. I hope you are doing great. In this post let’s talk about the Reference Counting and Static String Types.
One of Python’s major features is automatic memory management. CPython implements this via straightforward reference counting, with an automatic garbage collector that runs periodically to clean up unreachable reference cycles.
Cython manages all reference counting for us, ensuring a Python object (whether statically typed or dynamic) is finalized when its reference count reaches zero.
CPython’s automatic memory management has various implications when mixing static and dynamic variables in Cython. Say, for instance, we have two Python bytes objects b1 and b2, and we want to extract the underlying char pointer after adding them together, as you can see on the screen.
b1 = b"Old is Gold."
b2 = b"New is Smart"
cdef char *buf = b1 + b2
The b1 + b2 expression is a temporary Python bytes object, and the assignment attempts to extract that temporary object’s char pointer using Cython’s automatic conversion rules. Because the result of the addition is a temporary object, this example cannot work the temporary result of the addition is deleted immediately after it is created, so the char buffer cannot refer to a valid Python object. Fortunately, Cython can catch the error and issue a compilation error.
Once understood, the right way to achieve what we want is straightforward just use a temporary Python variable, either dynamically typed like this:
tmp = b1 + b2
cdef char *buf = tmp
Or statically typed like this one:
cdef bytes tmp = b1 + b2
cdef char *buf = tmp
These cases are not common. It is an issue here only because a C-level object is referring to data that is managed by a Python object. Because the Python object holds the underlying string, the C char * buffer has no way to tell Python that it has another (non-Python) reference. We have to create a temporary bytes object so that Python does not delete the string data, and we must assure that the temporary object is maintained as long as the C char * buffer is required. The other C types I mentioned in the previous ideo while talking about the type correspondence between built-in Python types and C or C++ types are all value types, not the pointer. For those types, the Python data is copied during the assignment, enabling the C variable to evolve separately from the Python object used to initialize it.