Creating Nagios Plugins using Python

Posted on January 6, 2010 by joestump

Like just about everyone else on the planet, we use Nagios to monitor our servers. We needed to create a few plugins for Nagios to monitor some services that Nagios didn’t have plugins for; namely Cassandra and Gearman. I wanted to be able to easily create plugins and have them installed with setuptools.

The code above is a simple class that will implement all of the option parsing and such to run a Nagios plugin from the command line. From there you need to implement a plugin. Below is the Gearman plugin that I created. It runs a simple job I created that sums numbers and returns the results. If all goes well then the job should run and the correct sum of the randomly selected numbers should be returned.

Once you’ve got all of that working it’s pretty simple to then add the following snippet of code to your setup.py file.


      entry_points = {
          'console_scripts': [
              'check_gearmand = path.to.nagios.plugins:check_gearmand',
          ]
      },

Why I switched from PHP to Python

Posted on September 4, 2009 by joestump

When it came time to start putting code to paper at CrashCorp I was faced with the decision of choosing both a language and a platform. After 11 years of coding solely in PHP I’d grown tired of the language and, to some extent, the community (not the people, who are great, but the way the community is organized).

First the language. What makes PHP, as a language, awesome is also what makes it horrible to work with, which is that it’s not really a language, but rather a giant plugin architecture for exposing lower level libraries in a high level fashion. Most of the language that developers use are, in fact, thin wrappers around popular C functions (curl, mysql, gd, etc.). Most of the time these libraries’ functions are simply exposed as-is. Anyone who’s coded curl in C will feel right at home while using curl from PHP. The problem with this is it leads to wildly inconsistent API’s.

Another touchy problem with the language is actually a byproduct of the way PHP, the core language, is managed. It’s, essentially, designed by committee. Anyone who’s ever tried to design a large scale anything knows how problematic this can be. The second problem with this approach is that nobody from on high is setting any kind of recognizable standards. PEAR has its standards and PHP has its standards, while everyone else codes however they damn well please. This leads to SPL classes being more Java style, while PEAR classes look a lot different (e.g. ArrayObject vs. HTTP_Request2).

The ultimate problem of this committee approach is that, before a feature can be integrated, the whole committee has to be on board. This is especially true for core language functionality. For instance, PHP just recently got anonymous functions and short-hand array slicing. Don’t get me started on namespaces in PHP.

I know quite a few core PHP coders personally and, from what I understand, they have a number of problems when evolving PHP. Besides the committee issues and the fact that extensions are coded by a few thousand different people, there’s the fact that PHP is installed on just about every machine on the planet so backwards incompatible changes wreak havoc on code everywhere.

At the end of the day I was tired of PHP’s inconsistent language syntax and waiting for more modern language features. Enter Python.

Python’s approach to creating a language is about as completely opposite as you can get from PHP. First, and foremost, Python is lead by Benevolent Dictator for Life, Guido van Rossum. The result is that the language’s development takes its cues from a single person with a consistent longterm vision of how things should be. Guido and the core Python coders set standards, via PEP’s, on everything from how common interfaces (e.g. DB’s) should work to coding standards (the infamous PEP8). Furthermore, practices Guido thinks are poor coding practices are simply not supported at the language level (e.g. there is no ++ operator nor can you do assignment in comparison operators).

The byproduct of this is that it permeates throughout the Python community. Due to the fact that Python has significant whitespace, combined with PEP8, you’d be hard pressed to find Python code that looks and feels drastically different between various projects.

But, overall, the thing I like most about Python is it explicitness. When you open a file in Python you know precisely what code is affecting that file. How many times I got burned by spaghetti require/include code I can’t tell you so this is a welcome addition.

On top of all of this Python has evolved significantly with regards to systems-level features. Want a daemon? Sure, just do import daemon \ daemon.daemonize(). Want threading? Sure, it’s all there. How about CLI option parsing? Just do from optparse import OptionParser.

Another thing I love about Python, is a religious adherence to KISS. You want namespaces? Fine the name of the file is the namespace. You want modules? Fine just replace / with . along with an __init__.py file and you’re good to go. Would you like to rename that function to something else? Fine just do new_func = old_func.

Finally, a stark difference between PHP and Python is that Guido, essentially, treats the developers as adults while PHP puts significant effort into protecting developers from themselves (I’m looking at you safe_mode). My favorite quote from Guido, while commenting on why Python doesn’t enforce private/protected/public variables was, “Hey, we’re all consenting adults here.” In addition to this, as my friend and Python hacker Mike Malone puts it, is that you can mangle whatever you want in Python. For instance, at runtime you can automatically extend class Foo from class Bar by doing Foo.__bases__ += Bar (Tip: This is especially handy for extending Django’s base User functionality). Much like UNIX, Python gives you more than enough rope to hang yourself, but at least hanging yourself is an option.

Overall, I’m really enjoying my decision to switch over and recommend you check out Python for your next project.

It's not the language stupid

Posted on April 15, 2008 by joestump

I’ve said it once, I’ve said it twice, I’ve screamed it from the top of mountains and yet nobody listens. I’m sitting in a session at the MySQL Conference and the person presenting just said, “You have to have well written code to avoid bottlenecks.” This is, put bluntly, stupid and patently false. Let me explain.

Your true bottlenecks when scaling are very rarely, if ever, because of your language. Sure Ruby is slower than PHP or Perl or Python, but only incrementally so and it’s only going to get faster. Even if your language is your problem it’s the easiest part of your architecture to scale; add more hardware.
Just because your code is well written doesn’t mean it will perform well and, conversely, just because you write shitty code doesn’t mean your code will not perform well. I’ve seen some seriously shitty PHP code that’s blazing fast because it’s so simple.
Depending on your application, as you grow you’ll find that your scaling issues come down to one fundamental problem: I/O. DB I/O, file system / disk I/O, network traffic, etc, etc. Ask anyone who’s written a large scale application where their growing pains were and I’ll bet my last dollar it wasn’t “PHP/Python/Ruby/Perl/Java/COBOL is slow”. I’m betting they’ll say something along the lines of “MySQL took a crap on us after we hit 200,000,000 records and had to do date range scans.” Or they’ll say, “I was storing user generated content and NFS couldn’t scale to the amount of requests for that content.”

I’m sick and tired of the language zealots who say PHP is slower than Perl or Ruby is slower than PHP or Java sucks because which language you’re using has zero to do with that missing index on your table or the fact that you can’t store all of that user generated content.

It comes down to your architecture and, despite what the zealots would have you believe, the language you choose is only one component of your overall architecture. Choose what you know and run with it.

Digg is hiring LAMP programmers

Posted on January 11, 2008 by joestump

It’s 6PM on a Thursday night and I’m about ready to head over to the Open Web Awards presented by Mashables.com to celebrate Digg’s wins with a few of my fellow Diggers. The only downer is that we don’t have more Diggers to share the fruits of our labor with. It reminded me that I should probably tell all 10 people who read this site that we’re looking for talented people to work with us in our San Francisco office (in Potrero Hill). Below is a little insight into what you’d be doing if you worked at Digg.

Program for the 36^th largest site on the intertubes according to Compete.com. Digg.com does 20,000,000+ unique visits a month. That’s a lot of zeros!
We use the LAMP stack (Debian GNU/Linux, Apache, MySQL and PHP with some Python thrown around from time to time) and expect you to be proficient with that.
Learn from some of the brightest minds in the PHP, design and operations communities.
Play with Memcache, Gearman, Mogile, PEAR, etc. in a high volume environment.
Create and contribute to open source projects while you work on high traffic and scalability problems at Digg.
20 days of PTO a year and access to all sorts of great benefits (medical, dental, vision, etc.).

I love working at Digg. It’s fun, fast pace and I work with some of the brightest minds in the industry. If you have any questions or interest please email jobs@digg.com.

Framework 0.4.0

Posted on December 29, 2007 by joestump

Thanks to Southwest delaying my 1.5 hour flight to Portland by 2 hours I’ve finally gotten time to package up the latest release of Framework. I jumped ahead to 0.4.0 from the 0.2.x series due to the fact that 0.4.0 isn’t really 100% backwards compatible and because I felt it was a major enough release to skip ahead a number. Lots of bug fixes, features, etc.

The biggest enhancements are that I switched to a pure exception model for error handling. I even went so far to switch the default PEAR_Error handler to throw Framework_Exception’s. Also, as asked by a ton of people I’ve created the ability to change your DB abstraction layers or not use one at all.

Writing phpt files for PEAR and profit

Posted on April 11, 2007 by joestump

I’ve been programming PHP now for, oh, almost a decade and I’ve been a contributor to PEAR for about four years now. It took me this long to finally get to the point that I think writing unit tests, in some form, is a good idea. I jumped on the OOP bandwagon about 6 years ago and drank the MVC koolaid about 4 years ago. As of a few weeks ago, I’ve jumped on the phpt bandwagon. I’m not going to cover the basics, because they’re well covered in a few different places around the web, but I am going to cover some of the nuances I’ve learned in the last few weeks.First off, you can run phpt files using the PEAR command line utility. I’ve been creating a tests directory and putting all of my phpt files in there. I normally name them 000-class-method.phpt and then increment the sequence number at the front for each test by 5 so I can do a decent job of predicting how tests run. Once you have a few tests you can run your tests in two different manners from the command line:

pear run-tests
pear run-tests 000-class-method.phpt

When you run your tests you’ll get results of which tests passed, which failed, if any of them were skipped and the reason why they were skipped, and files for debugging. If 000-class-method.phpt fails, there are a few files that will be of interest to you:

000-class-method.php is the actual PHP file ran, which is found in the --FILE-- portion of your phpt file.
000-class-method.outis what was actually output by the PHP file ran from the --FILE-- portion of your phpt file.
000-class-method.exp is what the test expected the output of 000-class-method.php to be, which is found in the --EXPECT-- portion of your phpt file.

You pretty much have everything you need to debug why a test failed. I normally diff the exp and out files and then debug the test (or my code, which is usually the case) using the php file.

Another great thing about phpt files is that you can use a cgi PHP binary to debug actual GET requests (phpt supports spoofing $_GET, $_POST and $_COOKIE using the cgi binary). This lets you create tests like the following:
--TEST-- Test a GET request --GET-- foo=bar&baz=foo --FILE-- <?phpecho $_GET['bar'] . "\n"; echo $_GET['baz'] . "\n";
?> --EXPECT-- bar foo

This test should pass without incident. The only catch is that you need to specify your cgi binary path using the --cgi argument when you run the tests (e.g. pear run-tests --cgi=/path/to/php-cgi).

Overall, I’m pretty happy with my newfound testing experiences. My only complaint is that phpt files don’t support spoofing of $_SESSION natively. A small complaint to be sure.

My first Digg code goes live

Posted on March 29, 2007 by joestump

So if you’re interested in seeing what I’ve been doing for Digg lately, you can now go to the site, log in and send invites to your friends. You’ve actually always been able to do this, but now you’re able to keep track of how many friends you’ve invited, those friend invites will be announced to the world on the homepage and you’ll automagically be added to the invitee’s friends list.

Once you’ve invited a friend, the friend needs to register and digg at least three stories before they count as a referral. At that point a little note is posted to the frontpage announcing your invite and your friend referral count in your user profile goes up. I, of course, invite all of you to join.

I’ve also been working on other magical doings, but I can’t talk about those quite yet. I hope to talk more about that work after it goes live as well.

stu.mp

Just another WordPress weblog

Category Archives: code