{"id":6760,"date":"2021-03-30T12:18:16","date_gmt":"2021-03-30T11:18:16","guid":{"rendered":"https:\/\/www.blopig.com\/blog\/?p=6760"},"modified":"2021-04-13T17:59:05","modified_gmt":"2021-04-13T16:59:05","slug":"c-python-bindings-in-5-minutes","status":"publish","type":"post","link":"https:\/\/www.blopig.com\/blog\/2021\/03\/c-python-bindings-in-5-minutes\/","title":{"rendered":"C++ python bindings in 5 minutes"},"content":{"rendered":"\n

You don’t even need to use CMake!<\/em><\/p>\n\n\n\n

Most of the time, we can use libraries like numpy (which is largely written in C) to speed up our calculations, which works when we are dealing with matrices or vectors – but sometimes loops are unavoidable. In those instances, it would be nice if we could use a compiled language such as C++ to remove the bottleneck.<\/p>\n\n\n\n

This can be achieved extremely easily using pybind11<\/a>, which enables us to export C++ functions and classes as importable python objects. We can do all of this very easily, without using CMake, using pybind11’s Pybind11Extension class, along with a modified setup.py. Pybind11 can be compiled from source or installed using:<\/p>\n\n\n\n

pip install pybind11<\/pre>\n\n\n\n\n\n\n\n
The project directory structure in this case should look like this:<\/p>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
We are going to use an extremely inefficient method for calculating the nth number in the Finbinacci sequence (we will write fibinacci.h<\/code> in a second). This is the function, written in fibinacci.cpp<\/code>:<\/p>\n\n\n\n
#include \"fibinacci.h\"\n\nunsigned int fibinacci(const unsigned int n){\n\n    if (n < 2){\n        return i;\n    }\n    return fibinacci(n - 1) + fibinacci(n - 2);\n}<\/pre>\n\n\n\n
This recursive algorithm is O(2^n<\/sup>)<\/code>, and will illustrate our speed gains nicely. Next we must write fibinacci.h<\/code>, which will also include the code which tells pybind11 how to glue everything together into a python module:<\/p>\n\n\n\n
#include <pybind11\/pybind11.h>\n\nunsigned int fibinacci(const unsigned int n);\n\nnamespace py = pybind11;\n\nPYBIND11_MODULE(pybind_11_example, mod) {\n    mod.def(\"fibinacci_cpp\", &fibinacci, \"Recursive fibinacci algorithm.\");\n}<\/pre>\n\n\n\n
In the header file, we have our usual function declaration, followed by some more interesting code. The macro PYBIND11_MODULE<\/code> defines a module that we can import using python, when everything has been compiled into a .so<\/code> library file. We use mod.def<\/code> to define a function in the library, with three argument:<\/p>\n\n\n\n
  1. The (python) name of the function (fibinacci_cpp<\/code>)<\/li>
  2. A pointer to the (cpp) function definition (&fibinacci<\/code>)<\/li>
  3. A brief description of the function<\/li><\/ol>\n\n\n\n
    We are already finished with our C++! Now onto setup.py<\/code> :<\/p>\n\n\n\n
    from pathlib import Path\n\nfrom pybind11.setup_helpers import Pybind11Extension, build_ext\nfrom setuptools import setup\n\nexample_module = Pybind11Extension(\n    'pybind_11_example',\n    [str(fname) for fname in Path('src').glob('*.cpp')],\n    include_dirs=['include'],\n    extra_compile_args=['-O3']\n)\n\nsetup(\n    name='pybind_11_example',\n    version=0.1,\n    author='Joe Bloggs',\n    author_email='joe_bloggs@email.com',\n    description='Barebones pybind11+setup.py example project',\n    ext_modules=[example_module],\n    cmdclass={\"build_ext\": build_ext},\n)<\/pre>\n\n\n\n
    The only difference from a standard setup.py<\/code> is in the last two arguments of setup<\/code>.<\/p>\n\n\n\n
    The ext_modules<\/code> argument includes modules which must be compiled separately, which is where the magic comes in – pybind11 comes with an automatic extension builder, accessed using pybind11.setup_helpers.Pybind11Extension<\/code>. Here we provide the name of the module, the source files (globbing the src directory means we’ll catch any new files), the include directory, and extra compiler flags<\/a> (-O3<\/code> tells the compiler to use lots of tricks to make our code run faster).<\/p>\n\n\n\n
    The cmdclass<\/code> argument overrides the usual build_ext<\/code> setuptools class so that it does some things automatically (like finding the highest available version g++).<\/p>\n\n\n\n
    We can now compile and install our module:<\/p>\n\n\n\n
    pip install -e . -vvv<\/pre>\n\n\n\n
    And if we write main.py<\/code>, such that we time both the C++ and Python versions of our function:<\/p>\n\n\n\n
    import time\nimport pybind_11_example as pbe\n\ndef fibinacci_py(x):\n    if x < 2:\n        return x\n    return fibinacci_py(x - 1) + fibinacci_py(x - 2)\n\nn = 40\n\nprint('Python:')\nstart_time = time.perf_counter_ns()\nprint('Answer:', fibinacci_py(n))\nprint('Time:', (time.perf_counter_ns() - start_time) \/ 1e9, 's')\nprint()\n\nprint('C++:')\nstart_time = time.perf_counter_ns()\nprint('Answer:', pbe.fibinacci_cpp(n))\nprint('Time:', (time.perf_counter_ns() - start_time) \/ 1e9, 's')<\/pre>\n\n\n\n
    Python:\nAnswer: 102334155\nTime: 21.634114871 s\n\nC++:\nAnswer: 102334155\nTime: 0.274699966 s<\/pre>\n\n\n\n
    Our C++ function is about 80 times faster. Not bad!<\/p>\n","protected":false},"excerpt":{"rendered":"
    You don’t even need to use CMake! Most of the time, we can use libraries like numpy (which is largely written in C) to speed up our calculations, which works when we are dealing with matrices or vectors – but sometimes loops are unavoidable. In those instances, it would be nice if we could use […]<\/p>\n","protected":false},"author":61,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","wikipediapreview_detectlinks":true,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"ngg_post_thumbnail":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[226,29,227],"tags":[380,223,379,383,378,382,381,152],"ppma_author":[541],"class_list":["post-6760","post","type-post","status-publish","format-standard","hentry","category-cpp","category-code","category-python-code","tag-bindings","tag-c-2","tag-cpp","tag-faster-code","tag-optimisation","tag-pybind","tag-pybind11","tag-python"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"authors":[{"term_id":541,"user_id":61,"is_guest":0,"slug":"jack","display_name":"Jack Scantlebury","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/2d30962dbe9d08db0ac110abbf9ffd5bd52f4eb7da79636d286fb584280feb2c?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts\/6760","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/users\/61"}],"replies":[{"embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/comments?post=6760"}],"version-history":[{"count":4,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts\/6760\/revisions"}],"predecessor-version":[{"id":6788,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts\/6760\/revisions\/6788"}],"wp:attachment":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/media?parent=6760"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/categories?post=6760"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/tags?post=6760"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=6760"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}