Category Archives: python

Creating Nagios Plugins using Python

Posted on by

Like just about everyone else on the planet, we use Nagios to monitor our servers. We needed to create a few plugins for Nagios to monitor some services that Nagios didn’t have plugins for; namely Cassandra and Gearman. I wanted to be able to easily create plugins and have them installed with setuptools.

The code above is a simple class that will implement all of the option parsing and such to run a Nagios plugin from the command line. From there you need to implement a plugin. Below is the Gearman plugin that I created. It runs a simple job I created that sums numbers and returns the results. If all goes well then the job should run and the correct sum of the randomly selected numbers should be returned.

Once you’ve got all of that working it’s pretty simple to then add the following snippet of code to your setup.py file.


      entry_points = {
          'console_scripts': [
              'check_gearmand = path.to.nagios.plugins:check_gearmand',
          ]
      },

Why I switched from PHP to Python

Posted on by

When it came time to start putting code to paper at CrashCorp I was faced with the decision of choosing both a language and a platform. After 11 years of coding solely in PHP I’d grown tired of the language and, to some extent, the community (not the people, who are great, but the way the community is organized).

First the language. What makes PHP, as a language, awesome is also what makes it horrible to work with, which is that it’s not really a language, but rather a giant plugin architecture for exposing lower level libraries in a high level fashion. Most of the language that developers use are, in fact, thin wrappers around popular C functions (curl, mysql, gd, etc.). Most of the time these libraries’ functions are simply exposed as-is. Anyone who’s coded curl in C will feel right at home while using curl from PHP. The problem with this is it leads to wildly inconsistent API’s.

Another touchy problem with the language is actually a byproduct of the way PHP, the core language, is managed. It’s, essentially, designed by committee. Anyone who’s ever tried to design a large scale anything knows how problematic this can be. The second problem with this approach is that nobody from on high is setting any kind of recognizable standards. PEAR has its standards and PHP has its standards, while everyone else codes however they damn well please. This leads to SPL classes being more Java style, while PEAR classes look a lot different (e.g. ArrayObject vs. HTTP_Request2).

The ultimate problem of this committee approach is that, before a feature can be integrated, the whole committee has to be on board. This is especially true for core language functionality. For instance, PHP just recently got anonymous functions and short-hand array slicing. Don’t get me started on namespaces in PHP.

I know quite a few core PHP coders personally and, from what I understand, they have a number of problems when evolving PHP. Besides the committee issues and the fact that extensions are coded by a few thousand different people, there’s the fact that PHP is installed on just about every machine on the planet so backwards incompatible changes wreak havoc on code everywhere.

At the end of the day I was tired of PHP’s inconsistent language syntax and waiting for more modern language features. Enter Python.

Python’s approach to creating a language is about as completely opposite as you can get from PHP. First, and foremost, Python is lead by Benevolent Dictator for Life, Guido van Rossum. The result is that the language’s development takes its cues from a single person with a consistent longterm vision of how things should be. Guido and the core Python coders set standards, via PEP’s, on everything from how common interfaces (e.g. DB’s) should work to coding standards (the infamous PEP8). Furthermore, practices Guido thinks are poor coding practices are simply not supported at the language level (e.g. there is no ++ operator nor can you do assignment in comparison operators).

The byproduct of this is that it permeates throughout the Python community. Due to the fact that Python has significant whitespace, combined with PEP8, you’d be hard pressed to find Python code that looks and feels drastically different between various projects.

But, overall, the thing I like most about Python is it explicitness. When you open a file in Python you know precisely what code is affecting that file. How many times I got burned by spaghetti require/include code I can’t tell you so this is a welcome addition.

On top of all of this Python has evolved significantly with regards to systems-level features. Want a daemon? Sure, just do import daemon \ daemon.daemonize(). Want threading? Sure, it’s all there. How about CLI option parsing? Just do from optparse import OptionParser.

Another thing I love about Python, is a religious adherence to KISS. You want namespaces? Fine the name of the file is the namespace. You want modules? Fine just replace / with . along with an __init__.py file and you’re good to go. Would you like to rename that function to something else? Fine just do new_func = old_func.

Finally, a stark difference between PHP and Python is that Guido, essentially, treats the developers as adults while PHP puts significant effort into protecting developers from themselves (I’m looking at you safe_mode). My favorite quote from Guido, while commenting on why Python doesn’t enforce private/protected/public variables was, “Hey, we’re all consenting adults here.” In addition to this, as my friend and Python hacker Mike Malone puts it, is that you can mangle whatever you want in Python. For instance, at runtime you can automatically extend class Foo from class Bar by doing Foo.__bases__ += Bar (Tip: This is especially handy for extending Django’s base User functionality). Much like UNIX, Python gives you more than enough rope to hang yourself, but at least hanging yourself is an option.

Overall, I’m really enjoying my decision to switch over and recommend you check out Python for your next project.

A discussion on languages and frameworks

Posted on by
Reply

I start all of my talks at conferences on architecture and scaling with describing the distinct differences between scaling and performance. I define scaling in one word: specialization. I, somewhat jokingly, retort to the question of what performance is with “Who cares?”.

The reason for not caring (much) about actual performance is that whether or not you use single quotes, double quotes, objects, functions, Python, Ruby, PHP, foreach, etc. has nothing to do with whether or not your application and site will scale.

Scaling is entirely about IO. Fundamentally, it’s about whether your data is being stored in a manner that makes retrieving it at the rate of today’s high traffic websites possible. In other words, Ruby isn’t the reason you can’t store 250,000,000 records in MySQL and do range scans. It’s because MySQL (and most RDBMS’s) suck.

I’ve been playing with Django a lot lately for a side project. Despite being a pragmatic coder to a fault, I’ve decided to truly learn another language and I felt Django would ease the shock a bit. So far I love it. Django’s patterns make a lot of sense and I’m loving the goodies a true OO language like Python gives me. Since starting down this path I’ve been getting two questions over and over:

  1. Does this mean Digg is going to be using Django?
  2. Why didn’t you choose Ruby on Rails?

Digg will not be using Django or any other framework anytime soon. We deal with traffic that most Django developers will probably never see. Our stack receives billions of requests a month. That kind of traffic, as I stated earlier, requires specialization. Django is the exact opposite of this. It’s a generic web framework made to answer the majority of web programmer’s basic needs. The majority of web programmers don’t deal with the problems we deal with. I’m sure if you ripped out a lot of what makes Django so great (e.g. the models, admin, etc.) Django would be fine (e.g. If we used it only for mapping requests to views and templates), but then it wouldn’t really be Django would it?

I chose Python and Django over Ruby and Ruby on Rails for a number of reasons. First and foremost is that we use Python here at Digg. Learning Python will only enhance my ability to perform my duties at Digg. I, personally, dislike the Perlisms in Ruby. Additionally, Ruby seems to skew towards implicitness in the language, while Python skews toward explicitness. I like explicitness.

To sum things up, I have nothing technically against Ruby as a language, I love frameworks for regular development work and Perl’s syntax kills small children.

Choose what you love and be happy coders.