In this post, let’s get an introduction to standard libraries of Python. Specifically Why it really useful and how it can be used? So let’s get started.

Libraries

There are different standard libraries of Python include distutils, IPython and pyximport for building, packaging, and distributing Python projects.

Distutils Library

Using distutils to compile Cython code gives us full control over every step of the process. The distutils package has many features of interest with ability to compile C source into an extension module which is the second stage in the pipeline. It manages all platform, architecture, and Python-version details for us, so we can use one distutils script and run it anywhere to generate our extension module. First pipeline stage is the cythonize command, which is included with Cython: it takes a Cython source file and compiles it to a C or C++ source file, and then distutils takes it from there.

By using Python’s distutils module combined with Cython’s cythonize command, we have explicit control over the compilation pipeline. This approach requires that we write a small Python script and run it explicitly. It is the most common way for Python projects to compile and distribute their Cython code to end-users.

We control the behavior of distutils through a Python script, typically named setup.py. A minimal setup.py script for compiling the test.pyx source file into an extension module is just a few lines long, two of which are imports as

The core of the script is in the setup(cythonize(...)) nested calls. The cythonize function in its simplest usage converts Cython to C source code by calling the cython compiler. We can pass it a single file, a sequence of files, or a glob pattern that will match Cython files.

Python Library

The IPython project has convenient commands that allow us to interactively use Cython from a live IPython session. These extra commands are IPython-specific commands called magic commands, and they start with either a single (%) or double (%%) percent sign. They provide functionality beyond what the plain Python interpreter supplies. IPython has several magic commands to allow dynamic compilation of Cython code, which we will cover in this video.

Before we can use these magic Cython commands, we first need to tell(We can use % sign to tell iPython load cython magic commands)

IPython to load them. We do that by using the % sign followed by magic command from the IPython interactive interpreter, or in an IPython notebook cell:

In [12]: %load_ext cythonmagic

There will be no output if %load_ext is successful, and IPython will issue an error message if it cannot find the Cython-related magics.[4]

Now we can use Cython from IPython via the %%cython magic command: You can see an example code here:

The %%cython magic command allows us to write a block of Cython code directly in the IPython interpreter. After exiting the block with two returns, IPython will take the Cython code we defined, paste it into a uniquely named Cython source file, and compile it into an extension module. If compilation is successful, IPython will import everything from that module to make the fib function available in the IPython interactive namespace. The compilation pipeline is still in effect, but it is all done for us automatically.

Now, we will discuss about pyximport library. Because Cython is Python-centric, it is natural to want to work with Cython source files as if they were regular, dynamic, importable Python modules. Enter pyximport: it retrofits the import statement to recognize .pyx extension modules, sends them through the compilation pipeline automatically, and then imports the extension module for use by Python. Let’s see an example. The pyximport module comes with Cython, and requires just two statements to get it up and running:


     #install() called before importing
     #Cython extension modules.
import pyximport pyximport.install()
  

We can use pyximport in an interactive IPython session to compile and load our familiar fib.pyx.

The pyximport package also handles more complex use cases. In this case, pyximport needs to recompile the .pyx file if any of its dependencies have been updated. To enable this functionality, we add a file with the same base name as the .pyx source file and with a .pyxdeps extension in the same directory as the Cython source file. It should contain a listing of files that the .pyx file depends on, one file per line. These files can be in other directories relative to the directory of the .pyxdeps file. The entries can also be glob patterns that match multiple files at once. If a .pyxdeps file exists, pyximport will read it at import time and compare the modification time of each listed file with the modification time of the .pyx file being imported. If any file that matches a pattern in the .pyxdeps file is newer than the .pyx file, then pyximport will recompile on import. The .pyxdeps file is nice to communicate file dependencies to pyximport.

We learned a lot, So in our upcoming videos we will also provide many examples to help you to use Cython effectively in your own projects.

Posted 
Oct. 20, 2021
 in 
Web Development
 category

More from 

Web Development

 category

View All