• Latest
• All Sites
• Why?
• Suggest

EveryBlock

A news feed for your block — Saturday, 3rd January, 2009

Adrian Holovaty is no stranger to Django, and his name is known throughout the community as one of the brains behind Django's birth and subsequent open-source release. His latest project EveryBlock is the evolution of an earlier mashup, chicagocrime.org, which won Adrian a number of awards. So what are the driving forces behind EveryBlock? I recently spoke with Adrian to find out.

Vital Stats

• 11 major US cities covered down to the street level
• Custom mapping using mapnik
• Strong use of Django's caching and nginx reverse-proxying mechanisms to manage loads
• Development done by a team of 3 developers, plus a designer

The Down-Low

As one of Django's founders and core developers, Adrian Holovaty is known in Django circles as the friendly BDFL who plays the guitar as well as he codes. One of his earliest Django projects outside the Lawrence Journal-World (where he began the Django project with Simon Willison was chicagocrime.org, a mash-up that showed crime details from the City of Chicago Police Department overlaid on Google Maps. It was a unique idea that spawned a number of Google Maps-based mash-ups.

However he had bigger targets in mind. In January of 2008, EveryBlock was opened to the world.

Behind it was not only Adrian's experience with chicagocrime.org and Django, but developer Paul Smith and designer Wilson Miner. What the visitors saw was also quite different: Not only had the EveryBlock team aggregated crime statistics for a number of cities, but other public information was also added to the foray including news, property foreclosures, business listings, and photographs.

EveryBlock aims to show you practically everything that's possible about the goings on near a city block, by combining data aggregated from multiple sources with custom city maps. It's built in Django, with Adrian's only concern being that it cannot correctly pluralise "octopus". Nothing else was considered, for somewhat obvious reasons.

From the projects beginnings in July 2007, it took the team 6 months to launch the public website, although Adrian says "that time includes a significant amount of effort spent on acquiring data for those initial cities". After the launch, the development team was expanded to include Joseph Kocherhans.

When EveryBlock was started, GeoDjango was still in its infancy so the team implemented what Adrian calls "fancy geographic extensions" on top of an otherwise straightforward Django codebase. To deal with the complex calculations required for some pages, high-traffic pages are served directly from memcache by nginx instead of having visitors repeatedly hit the application.

EveryBlock also optimises memory use by making extensive use of the ORM's values() function, avoiding the overhead of creating python model objects. This provides the developer with a plain Python dictionary of values which is suitable in many cases.

All of this data is presented to the user overlaid on custom city maps, built using OpenLayers and Mapnik. This solid open-source core is extended with the use of software such as the nginx web server, Monit (for keeping a watchful eye on servers), and the PostgreSQL database.

In terms of Django, EveryBlock runs against a recent subversion checkout without any patches. Unlike many projects which end up with a small repository of their own patches to the core product, Adrian is in the unique situation of being able to fix problems in Django as he discovers them. He's also got a better-than-average understanding of many of Django's internals, allowing for code that steps outside the lines of what's documented.

As EveryBlock grows and continues to add cities to their listing, very little coding is required. "Adding a new city requires adding the data, which is the really time-consuming part. The work pays off. Nobody else is crazy enough to do it!".

And the Future? The main aim right now "More cities, more data in existing cities, and more features!". And of course, it's going to be powered by Django.

• A big thank-you to Adrian Holovaty from EveryBlock for taking time out of his busy schedule to answer my questions for this week's Django Site of the Week.

Thanks

Special thanks go to Adrian Holovaty for this interview.

EveryBlock can be visited at http://www.everyblock.com/. It is also listed at DjangoSites.org.

Comments

Share Your 2c

Name

Email address

This is not published, and is only used for moderation purposes.

URL

Comment

Markdown is hot, HTML is not.

If you enter anything in this field your comment will be treated as spam

© Copyright 2008 Ross Poulton. All Rights Reserved. Django is a registered trademark of the Django Software Foundation.