Watch videos with subtitles in your language, upload your videos, create your own subtitles! Click here to learn more on "how to Dotsub"

A Whirlwind Excursion through Writing a C Extension

0 (0 Likes / 0 Dislikes)
Hi, my name is Ned Batchelder. I am going talk to you for the next forty minutes About how to write C extensions for Python. This is a very big topic. So, some guidelines. First of all, I had trouble reading the text on some of these slides for this talk So this tinyurl up here, tinyurl/whirlext is a link to my page on my site Where these slides and some prose and the code for download is available. So, this is a big topic, we are going to go very very fast. I have real code examples. As I said you can download and compile them for yourself But they are toy examples, they are silly functions that you never really want to write in C But they are here for (?) purposes, to make a point And not get tangled up in some kind of bizarre specialized domain knowledge that you don't care about. I will give you information about how to write C stuff and you can provide interesting domain knowledge yourself and your own extensions. I am also assuming that you know Python and you know C. I am not going to say: "does anyone here not know what generator is" or something like that. I am just gonna go. And hope that you guys can keep up. The roadmap for today: we are going talk about C API. We are gonna build an extension Which of course is a hello world extension Cause what else would you start with and sometimes you need to get a little more performance out of your hello world programs. We'll go into details about what really in the C API. We'll talk about error handling, we'll talk about memory management, we'll talk about creating a type and we'll be done. First, why write in C? The main reason people go to C is for speed. Python interpreter—we don't have Unladen Swallow yet— We still working with slower interpreter So, sometimes you just need more speed than the Python interpreter can provide you. Maybe your compute (?), maybe you have very intensive algorithms That you want to manipulate memory more efficiently Than Python interpreter will let you do. Maybe you need to integrate with legacy code: you have existing C library and you need to call it. Maybe you want to integrate Python into your existing C application turning things upside down: your main program is in C but you want to use Python scripts internally as an extension language or as a way to add more features more efficiently than writing in C. And lastly for community: if you want to contribute to the Python core this is the API you are gonna be using. Other options for speed: there are a lot of options for making Python go faster. I don't know what they all do I don't know how they work I've never really used any of them There are lots of differences among them You should go and investigate them But we are gonna be talking about the old style legacy C API here. So, what is the C API? It's Python's foundation. Unlike some other languages which have an extension mechanism bolted on This is actually the API that is used inside the Python to build Python itself. There are six hundreds entry points That is a rough estimate that I did (?) So, it is a big API Everything internal to Python that builds Python itself is exposed in this API So you can there lots of things. And last it's only applies to CPython One of a big themes this week has been all these other Python implementations Which is very very cool But this is one of the areas that are very specific to C Python implementation. IronPython has a project called Ironclad which is performing impossible of letting you write C API code against IronPython and I don't believe it exists cause it's clearly not possible to do that but apparently it is out there. The good thing about using the same API that Python developers use to build Python Is that the entire codebase of Python itself is a code example library for you. If you need to do something kind of like the built-in set type go look at how it's implemented. It's built using the same tools that you're gonna be using to build your type whatever it needs to do. The downside of working in C is that it's very different world There is a reason we're here in Python Python is very nice. It does memory management for you, it has exceptions it has dictionary literals and all sorts of things that make it very efficient to write in even if it is not efficient to execute. By switching back over that (?) into the C world you are gonna be manually dealing with a lot of the things that are messy in C. And not only that but you will have to do messy things in C at the same time that you do all the work to make them nice and clean up in the Python world. So you kind of have the worst of both worlds. It can be done but you have to take a lot of care with it. So, let's build a real extension. This is a hello world extension. This is the python code we are going to write in C. Now, I am not saying that we are going to automatically convert this python into C. I am using python here as our favorite sudo language to show what our C code is going to do.

Video Details

Duration: 39 minutes and 54 seconds
Country: United States
Language: English
Producer: PyCon
Views: 111
Posted by: antonkovalyov on Apr 13, 2009

Everyone's heard of C extensions to Python, but few have ventured into those murky depths. This is a quick overview of how to write a C extension, complete with highlights of the C API, memory management, error handling, and type creation. Whether you want to speed up your Python, integrate with legacy code, contribute to Python's core, or just learn a little about how it all works under the covers, you will enjoy this trip through unfamiliar territory. I guarantee you will return safely.

Caption and Translate

    Sign In/Register for Dotsub above to caption this video.