In the previous post, we talked about Python functions inside Cython. Now, let’s try to understand How C functions work inside Cython?
The straightforward way to define a C function inside Python is by using cython’s cdef keyword. The cdef keyword creates a function with C- calling semantics. A cdef function’s arguments and return type are typically statically typed, and they can work with C pointer objects, structs, and other C types that cannot be automatically constrained to Python types. It is helpful to think of a cdef function as a C function that is defined with Cython’s Python-like syntax. If you remember the example we discussed in our post to calculate the factorial of a given number, then here's the C version of that function, named as C_factorial.
cdef long c_factorial(long num): """Computes num!""" if num <= 1: return 1 return num * c_factorial(num - 1)
Careful inspection of C_factorial in this example shows that the argument type and return type are statically declared, and no Python objects are used; hence, no conversions from Python types to C types are necessary. Calling the C_factorial function is as efficient as calling a pure-C function, so the function call overhead is minimal. Nothing prevents us from declaring and using Python objects and dynamic variables in cdef functions or accepting them as arguments. But cdef functions are typically used when we want to get as close to C as possible without writing C code directly.
There are few things you should keep in mind in this context. First, Cython allows cdef functions to be defined alongside Python def functions in the same Cython source file. The optional return type of a cdef function can be any static type we have seen, including pointers, structs, C arrays, and static Python types like list or dict. We can also have a return type of void. If the return type is omitted, then it defaults to object.
A function declared with cdef can be called by any other function—def or cdef—inside the same Cython source file. However, Cython does not allow a cdef function to be called from external Python code. Because of this restriction, cdef functions are typically used as fast supporting functions to help def functions do their job.
If we want to use the c_factorial function from Python code outside this extension module, we need a def function that calls c_factorial internally: as you can see in this example code:
def wrap_c_factorial(n): return c_factorial(n)
We get a nice speedup for our efforts: wrap_c_factorial(20) is about 10 times faster than the factorial(20) function, we discussed in the previous post, and its typed version, both of these have significant Python overhead.
But Unfortunately, this wrapper function comes with some limitations. One limitation is that wrap_c_factorial and its underlying c_factorial, both are restricted to C integral types only, and do not have the benefit of Python’s unlimited-precision integers. In practice, this means that wrap_c_factorial gives incorrect results for arguments larger than some small value, depending on how large an unsigned long is on our system. One option to partially address this limitation while maintaining Cython’s performance would be to use doubles rather than integral types.
This is a general issue when we are working with Python and C, and is not specific to Cython: Python objects and C types do not always map to each other perfectly, and we have to be aware of C’s limitations.
Great, we learned a lot and that’s about the C functions inside python using Cython. And I think that’s enough for this post, in our next post we will try to explore how can we combine the def and cdef keyword using cpdef.