Moving on to the next chapter

This week I gave notice to Digg that I’m going to be leaving the company to pursue other opportunities. When I first joined Digg I had a list of goals in my head that I wanted to accomplish and, I feel, they’ve either been accomplished or are well on their way to being fulfilled by other shepherds at Digg. Luckily, we’ve managed to recruit some of the industry’s leading brains (all of whom are far smarter and more capable than I) to continue building great products for Digg and I’m excited to see where they take things.

So what’s next? Well, I’m spending July couch surfing through Europe, working on a few side projects, putting together a small conference on scaling and generally taking things easy. I’ll also be continuing to annoy the coders at Digg as an advisor moving forward.

It’s hard to put into words what my time at Digg means to me. I’ve had the distinct pleasure of working with many of Silicon Valley’s brightest minds and they’ve been gracious enough to not only give me a chance to prove myself, but also impart some of their knowledge to me. For that I’ll be forever grateful and indebted.

Thankfully, Kevin accepts beer as payment.

Year in Review

  1. The end of 2008 marks the end of my first year as Digg’s Lead Architect. In that time we’ve rewritten the majority of the site using frameworks that I built. We’re currently rewriting the underlying data access layer to be horizontally partitioned, elastic, services oriented and multi-homed.
  2. In early January, Digg Images launched and, with it, the result of months of work resulting in a completely rewritten submission framework for Digg. This project resulted in me writing and releasing Net_Gearman. I consider this project to be some of my best work at Digg.
  3. In early January I snuck off to Vail for one last snowboarding trip before back surgery. It was on this trip that I finally became comfortable with Western black diamonds, including an awkward drop off of an 8+ foot precipice into 3+ feet of fluffy powder.
  4. On January 23rd, 2008 I went in for back surgery. Two hours after surgery I was up and walking around without a hint of sciatica or back pain. I can’t thank Dr. Fred Naraghi enough for what I view as a second chance at life.
  5. 2008 will be known as the Year of the Conferences for me. I spoke at Future of Web Apps in Miamion a panel at SXSW on scaling websitesMySQL Conference on Services Oriented Architecture, Web 2.0 Expo in New York City, Future of Web Apps in London with Blaine Cook, Future of Web Design on the friction between developers and designers, and Q-Con in San Francisco on Digg’s architecture.
  6. The summer brought another bout of triathlon training. Along with my friend Mark Lewandowski, I trained for my first Olympic distance triathlon, which I ended up finishing in 2 hours, 50 minutes and change. As part of our training Mark and I also did a 72 mile bike race around Lake Tahoe. The race included 3,900 feet of vertical gain over 72 miles and is, without a doubt, the most challenging endurance race of my life. I finished the race in 4 hours, 15 minutes and change.
  7. In June I was elected to the PEAR Group, which is the governing board of my favorite PHP project.
  8. In early September I launched PleaseDressMe with my friends AJ and Gary Vaynerchuk. The site continues to gain traction in the tshirt arena and is, to date, my most successful side project.
  9. In October Aubrey, Kevin and I went on a whirlwind tour of Europe that included Oktoberfest in Munich, London, and Amsterdam.
  10. November brought big news at Digg with the hire of my friend and release manager for PHP6, Andrei Zmievski, as Digg’s first Open Source Fellow.
  11. November also brought about me finally diving into Python and Django for a side project. I’ve built an API for iPhone games that my friend Garren and I plan on releasing soon. More on this to come.
  12. December brought another trip to Thailand with my good friend Chris Lea. We’d originally planned to do Thailand, Cambodia and either Laos or Vietnam, however the islands of Koh Phangan and Koh Samui had other ideas. I type this sitting on Haad Lamai on Koh Samui. So far it’s been an epic trip with highlights including New Year’s Eve on Haad Rin Nok and a trip back to Haad Rin Nok tomorrow for another Full Moon Party.

This year I’m going to follow the year in cities theme that so many other blogs follow because I feel I really have done a ridiculous amount of travel this year.

  • San Francisco, CA
  • Miami, FL
  • Austin, TX
  • San Diego, CA
  • Seattle, WA
  • Vail, CO
  • East Jordan, MI
  • New York, NY
  • Munich, Germany
  • London, United Kingdom
  • Amsterdam, Netherlands
  • Los Angeles, CA
  • Bangkok, Thailand
  • Haad Leela, Koh Phangan, Thailand
  • Haad Lamai, Koh Samui, Thailand

I’m going to start a new theme here today. Below is my year in open source software. This is a list of projects I’ve released publicly and/or have contributed to. I’m not sure how many lines of code this is, but this is, by far, my most prolific year in FOSS contributions.

Giving back to the community

At Digg we use a lot of open source software. A short list includes PHP, Memcached, MogileFS, Gearman, Debian GNU/Linux, Python, Perl, MySQL, Apache, APC and PEAR. Something that may not be quite as well known is that Digg developers have been busy giving back to the community as well.

The best part, in my opinion, about all of this is that we release our code under the most liberal license possible given the circumstances – the New BSD License (We use New BSD to protect Digg’s trademarks).

Of course there are other companies that contribute significantly to FOSS. Flickr, Facebook, Yahoo!, IBM and Google are just a few and I’m more than happy to say that Digg is giving back as well.

Parsing PUT requests in PHP

I’m working on the next generation data access layer for Digg right now, which is basically a REST layer built on top of a partitioned and multihomed database setup. The general idea is that we’ll send GET, POST, PUT and DELETE requests to URI’s on our services layer to access and manipulate data. PHP makes accessing GET and POST easy via $_GET and $_POST. DELETE isn’t an issue since what we’re deleting is just the entity defined by the URI (e.g. Sending DELETE to /2.0/User/1234.xml will delete User 1234).

After a few days work I can create, fetch and delete entities from this setup. Today I started working on implementing the PUT method. I always knew PHP wasn’t exactly top notch when it came to PUT support, but I had no idea how annoying it would be to find a simple solution for parsing PUT information. After some digging around this is what I’ve figured out.

$put = array();
parse_str(file_get_contents('php://input'), $put);

That should parse everything into a native PHP array, including arguments like foo[bar]=1&foo[baz]=2. If anyone knows of a more native way of doing this please let me know.

Digg is hiring LAMP programmers

It’s 6PM on a Thursday night and I’m about ready to head over to the Open Web Awards presented by Mashables.com to celebrate Digg’s wins with a few of my fellow Diggers. The only downer is that we don’t have more Diggers to share the fruits of our labor with. It reminded me that I should probably tell all 10 people who read this site that we’re looking for talented people to work with us in our San Francisco office (in Potrero Hill). Below is a little insight into what you’d be doing if you worked at Digg.

  • Program for the 36th largest site on the intertubes according to Compete.com. Digg.com does 20,000,000+ unique visits a month. That’s a lot of zeros!
  • We use the LAMP stack (Debian GNU/Linux, Apache, MySQL and PHP with some Python thrown around from time to time) and expect you to be proficient with that.
  • Learn from some of the brightest minds in the PHP, design and operations communities.
  • Play with Memcache, Gearman, Mogile, PEAR, etc. in a high volume environment.
  • Create and contribute to open source projects while you work on high traffic and scalability problems at Digg.
  • 20 days of PTO a year and access to all sorts of great benefits (medical, dental, vision, etc.).

I love working at Digg. It’s fun, fast pace and I work with some of the brightest minds in the industry. If you have any questions or interest please email jobs@digg.com.

New iPhone version of Digg

Like many other interesting projects the Digg iPhone project started with a conversation over a few beers and a challenge: code it in 48 hours and Kevin would give me a free iPhone. Being the unabashed Apple fanboy I am and, also, being a self-respecting coder I set out to create Digg for the iPhone.  After spending a short time white boarding the application, Daniel mocked up the design and I set off to code it.

Technically speaking, it’s no revolutionary application and I didn’t spend the entire 48 hours working on the application. I did, of course, find time to go see Transformers (awesome) over the weekend. The JavaScript was borrowed from Joe Hewitt and adapted a bit using jQuery. The application iteself is based on our API using the Services_Digg PEAR package I maintain. I’ve been talking with the jQuery team about some limitation in the animations and plan on packaging up a more robust iPhone JavaScript library based on jQuery sometime in the near future (hopefully).

Yesterday Kevin announced the iPhone application and Daniel Burka, our designer at Digg, has covered the details about designing for the iPhone. So far the response to the application has been positive. It was fun and I’m glad everyone is enjoying it. I know I did while riding the bus to work today.

And to answer everyone’s questions. I got an 8GB version last night, the keyboard is interesting, it’s breathtakingly gorgeous and I’ll write YAiPR (Yet Another iPhone Review) soon.

Technical Background of Digg's new Comment System

UPDATE: As you might have noticed, we’re having some technical issues rolling out the new comments. Please bare with us as we work out the kinks.

UPDATE: We reworked a few things and the comments are now live again.

Today Digg launched it’s redesign of the comment system, which was programmed by yours truly. Daniel has written up a detailed overview of the design decisions so if you’re interested in the design aspects you’ll definitely want to check that out.

There were a few fairly complex technical changes to the comment systems, which I’ll outline and then go into a little detail about.

  • We’ve been talking about moving towards a services oriented architecture on and off since I started in February. Steve had coded the API and the decision was made that the new comments system would use Digg’s public API.
  • Comment threads would be loaded dynamically using AJAX and JSON.
  • All commenting, editing, etc. would happen via AJAX.

The API ended up making the PHP code behind the scenes relatively painless. Each page loads using two small calls; one to figure out how many comments are on a story and another to fetch the first 50 or so comments (we use a fuzzy limit so it’ll just load all of them if there’s, say, 55 comments on a story). Both our local proxy and the comments code use the Services_Digg package we released via PEAR when the API launched. In fact, the entire permalink page is built using the API now, which is pretty neat.

By far the most complex portion of the comments system was how dynamic it was going to be. Threads would be zipping in and out, we’d be creating 90% of the HTML dynamically in the DOM from JSON, posting and editing over AJAX, etc. It was during design that Micah and I also plotted to remove script.aculo.us and replace it with the smaller jQuery library. The entire comment system is, in fact, a series of jQuery plugins.

Probably the coolest, technically speaking, portion of the new comments is the manner in which most of the page is created. No longer do we create static HTML in PHP and send you a huge HTML page. Instead we give you the basics and, via AJAX/JSON, we make requests to the API and dynamically create the DOM using the FlyDOM jQuery plugin. The FlyDOM JSON templates are a stroke of genius if you’re looking at loading JSON dynamically into the DOM. The advantage of this is that initial page loads are much snappier and you can load the threads you wish to read on demand.

I really picked up the whole dynamically created DOM ball and ran with if. If you notice, on the initial page load there aren’t any forms anywhere in the DOM. Those, also, are created dynamically on request. An interesting side effect to this is that there’s about 4x as much JavaScript code on the new comments than there is PHP.

The major technical and design changes of comments should lead to faster load times, less bandwidth being eaten up and, hopefully, a better user experience. I hope you enjoy them and, as always, welcome comments and input.

As Kevin would say, “Digg on!”

My first Digg code goes live

So if you’re interested in seeing what I’ve been doing for Digg lately, you can now go to the site, log in and send invites to your friends. You’ve actually always been able to do this, but now you’re able to keep track of how many friends you’ve invited, those friend invites will be announced to the world on the homepage and you’ll automagically be added to the invitee’s friends list.

Once you’ve invited a friend, the friend needs to register and digg at least three stories before they count as a referral. At that point a little note is posted to the frontpage announcing your invite and your friend referral count in your user profile goes up. I, of course, invite all of you to join.

I’ve also been working on other magical doings, but I can’t talk about those quite yet. I hope to talk more about that work after it goes live as well.

My First Week at Digg

The new sign at the Digg office

So I made it down to San Francisco in one piece. After spending a week in a hotel I’m now up in Concord, CA staying with Jeremy.

Tomorrow I start my second week at Digg. I can’t really talk about what’s going on or what I’m doing, but I’m surrounded by lots of geeky people and back in the thick of Silicon Valley, which is always interesting. As expected, things are exciting and there’s ton of potential everywhere.

One thing that will take some getting used to at Digg is that there are people who handle servers and such, which is new to me. At all my other jobs I was the PHP Developer/MySQL Administrator/Systems Administrator. At Digg I just say “Hey, Time, I need a DB.” or “Hey, Ron, can you upgrade GD?”

I don’t think I’ll be blogging about work very much as we have people who do that as well, but suffice it to say I’m excited about the job, city and possibilities.

I'm employee #20 at Digg

It’s official. As of today I’m employee #20 at Digg. I sent in my resume in September fully expecting not to hear back, but over the course of the last few months it became apparent I was being considered for a position. I didn’t think much of it after not hearing from them for almost a month, but Brian Link sent me an email around the first of the year which turned out to be an official offer.

For those of you that don’t know, Digg is the 19th most visited site on the internet in the United States. It serves up more traffic than Wal-Mart, the New York Times or Best Buy. So, as you can imagine, I’m pretty excited about this opportunity. I’ll be arriving in San Francisco somewhere towards the beginning of February. If you’re in the area and would like to get pints please let me know.

Taking the job means that I won’t be doing any contract work and my open source projects will most likely sit idle for the time being. If you’re interested in maintaining Framework or any of my PEAR packages please let me know.