Technical Background of Digg's new Comment System

UPDATE: As you might have noticed, we’re having some technical issues rolling out the new comments. Please bare with us as we work out the kinks.

UPDATE: We reworked a few things and the comments are now live again.

Today Digg launched it’s redesign of the comment system, which was programmed by yours truly. Daniel has written up a detailed overview of the design decisions so if you’re interested in the design aspects you’ll definitely want to check that out.

There were a few fairly complex technical changes to the comment systems, which I’ll outline and then go into a little detail about.

  • We’ve been talking about moving towards a services oriented architecture on and off since I started in February. Steve had coded the API and the decision was made that the new comments system would use Digg’s public API.
  • Comment threads would be loaded dynamically using AJAX and JSON.
  • All commenting, editing, etc. would happen via AJAX.

The API ended up making the PHP code behind the scenes relatively painless. Each page loads using two small calls; one to figure out how many comments are on a story and another to fetch the first 50 or so comments (we use a fuzzy limit so it’ll just load all of them if there’s, say, 55 comments on a story). Both our local proxy and the comments code use the Services_Digg package we released via PEAR when the API launched. In fact, the entire permalink page is built using the API now, which is pretty neat.

By far the most complex portion of the comments system was how dynamic it was going to be. Threads would be zipping in and out, we’d be creating 90% of the HTML dynamically in the DOM from JSON, posting and editing over AJAX, etc. It was during design that Micah and I also plotted to remove script.aculo.us and replace it with the smaller jQuery library. The entire comment system is, in fact, a series of jQuery plugins.

Probably the coolest, technically speaking, portion of the new comments is the manner in which most of the page is created. No longer do we create static HTML in PHP and send you a huge HTML page. Instead we give you the basics and, via AJAX/JSON, we make requests to the API and dynamically create the DOM using the FlyDOM jQuery plugin. The FlyDOM JSON templates are a stroke of genius if you’re looking at loading JSON dynamically into the DOM. The advantage of this is that initial page loads are much snappier and you can load the threads you wish to read on demand.

I really picked up the whole dynamically created DOM ball and ran with if. If you notice, on the initial page load there aren’t any forms anywhere in the DOM. Those, also, are created dynamically on request. An interesting side effect to this is that there’s about 4x as much JavaScript code on the new comments than there is PHP.

The major technical and design changes of comments should lead to faster load times, less bandwidth being eaten up and, hopefully, a better user experience. I hope you enjoy them and, as always, welcome comments and input.

As Kevin would say, “Digg on!”

13 thoughts on “Technical Background of Digg's new Comment System

  1. Pingback: iFrankie » Digg’s New Comment System?

  2. Pingback: Digg the Blog » Blog Archive » New Comments Live

  3. For a long time, we were building all our elements using DOM from JSON results – we ended up abandoning it and constructing the HTML on the server end in order to use smarty. We had been using the actual DOM calls, not FlyDOM. I’m looking forward to a followup post a few weeks / months from now with your impressions.

  4. Pingback: jQuery: Digg’s new comment system powered by jQuery « Rip’s Domain

  5. I must say, to be quite honest, that you probably didn’t really think this through and test it. The speed of the new system is horrendous. And why? The DOM is slow. Nobody’s going to care that the story page loads well when their browser is slow in loading the comments which they wish to read.

    http://www.quirksmode.org/dom/innerhtml.html

  6. This is very interesting — thanks for the post.

    (1) I think it’s interesting that you’ve chosen not to provide an alternative for users with Javascript turned off — hiding information from users who choose to do so is considered a “no no” by the web standards and accessbility police. I think the biggest problem is probably for users who use screen readers. When/how was the choice made to *not* degrade the site gracefully?

    (2) Kudus for “eating your own dog food” and using your public APIs and library. I’m working on a project at my own gig where we’ve taken this approach. We’re actually co-developing a public API and an app that implements much of it, and it is helping us make good choices about how to design the API.

    (3) How big a benefit do you think you get by generating the DOM on the fly vs. using an approach like jhaddad suggests and generating and returning the rendered HTML instead of the JSON? It seems to me the former approach is a little easier maintain and I’d only take the latter approach if the UX benefit (or bandwidth costs) associated where significant…

  7. Joe, the jQuery project team is ecstatic that you’ve chosen to use the jQuery javascript library for the comment system. If there’s anything we can do to help or if you encounter any issues, please do not hesitate to contact us. You can reach me at reybango (at) gmail {dot} com.

  8. Pingback: Javascript, oh the choices! : [ mkhairul.com ]

  9. Pingback: Another win for jQuery « These are the days

  10. Pingback: Ryan » Blog Archive » Digg.com Adds Ajax Icing and JQuery

  11. Joe, if it turns out that the DOM generation is the bottleneck, but you still want to download JSON instead of HTML, consider generating HTML code in your JavaScript and using .innerHTML (or .html() in jQuery).

    I wrote the first DOM creation plugin for jQuery, but I no longer use it much, nor any of the other DOM creation plugins that have come along since then. innerHTML is so much faster that there’s no comparison. I use [].join(”) to generate the HTML code – it is very fast and the code is readable too, e.g.

    var url = ‘http://example.com/’;
    var html = [
    ”,
    ‘,
    ‘My Link’,
    ‘,

    ].join(”);

    Let’s hope that the comment system keeps the formatting in that code (fat chance?). 🙂

  12. Well, I’ve been waiting a bit, but there are a few stylistic issues I’m having problems with.

    First, I can’t post a reply to anyone if I have the page sorted by most diggs (which I usually do). However, I could understand that comments could be taken out of context in this sense.

    I’m also no big fan of clipping the comments at the end of the page, there should really be an option for this.

    Readability of the comments page would improve if responses were indented one level instead of completely boxed.

    |Person 1 says
    | yadayadayadayada
    | __________________
    | |1 reply to Person 1
    | | ________________
    | | |Person 2 says
    | | | yabbadabbadoo
    | | |________________
    | |Reply to person 1
    | |__________________
    |____________________

    vs.

    |Person 1 says |
    | yadayadyadayada |
    | reply |
    . |Person 2 says |
    . | yabbadabbadoo |
    . | reply |

    Which one looks easier to read?

    These two styles look similar to these pages of the GNOME HIG v2:
    http://developer.gnome.org/projects/gup/hig/2.0/controls-frames.html
    http://developer.gnome.org/projects/gup/hig/2.0/design-window.html

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.