Project/Code Re-Use, TurboGears, and Django
TurboGears has been making some impressive strides in both features, documentation, and possible user acquisition lately. Where it gets somewhat interesting is regarding its user-base though. The approach TG takes – building glue on top of other projects – is not new, as Subway also utilizes this, however the popularity TG has been enjoying has resulted in some interesting by-products.
First, its a bit interesting to take a look at some numbers. It’s hard to be precise, as users don’t exactly announce themselves, but many of them do usually join the mailing lists for the projects.
Django-users:
- Users: 546
- Activity: Medium
- Average messages per month (5 months+): 340
Update: Eugene pointed out that there is Django-developers as well. Many of the Django-devel members are also on the Django-users list, so adding them together wouldn’t be too accurate. Here’s the Django-devel information, along with the total traffic if you take the two combined.
Django-developers:
- Users: 323
- Activitiy: Low
- Average messages per mont (5 months+): 177
- Average messages of Django-devel+users: 517
TurboGears:
- Users: 668
- Activity: High
- Average messages per month (3 months+): 1,124
Information taken from Google Groups as of Nov. 14th, 2005
While each list is bound to pick up users that contribute and lurk who do not actually use the framework, I think this trend is significantly higher with TurboGears. I believe this is because TurboGears builds on several successful projects that are even used in fields outside of web development.
Project/Code Re-Use and Explaining the Mail Traffic
Django takes a somewhat interesting approach, especially when compared to TurboGears, in that re-use is non-existent.The reason given for the amount written from scratch does make sense to a certain extent, especially when you remember that Django was created 2 years ago (some existing projects weren’t too hot back then). It’s also interesting to note that in some cases, the syntax and decisions being applied to Django now, reflect those made quite awhile ago in other projects.
TurboGears pushes re-use to the extreme, and only resorts to writing it from scratch if there’s no clean way to integrate a similar project that does the same thing. Only the new Identity framework, and the crud stuff was written from scratch for example. But many of the largest, most heavily used pieces have large communities of their own, i.e. SQLObject, FormEncode, CherryPy, MochiKit, and setuptools (I likely missed some).
Also interesting to note, several of the top posters to the TG mailing list happen to be the project developers for the individual projects TG is comprised of. I doubt they’re heavy users (or perhaps haven’t even used TG), however they have plenty to contribute as the questions often relate to the individual parts. This creates a powerful community and further fuels development and use of the individual projects TG pulls together.
When considering the use of one framework vs the other, I think this re-use issue is definitely something to be considered. Building on other stable, established projects that are used widely and extensively leads to a more mature platform. The size of each individual community also means more people to work on solutions that benefit the whole (and even those using the parts).
For example, the TG community is starting to tackle some basic CRUD/administration type stuff. I don’t actually use TG myself, but I do use SQLObject and FormEncode which means I can plug my SQLObject classes into a TG project, and use their web based admin tool with my database. This is rather powerful and extremely useful code re-use, and in the future it’ll be even easier to do the same thing (likely using a WSGI app).
Code re-use is important, and the project re-use that TG employs leads to many benefits. Despite the reason cited on the Django page, I think Django is still ripe for re-use cases. The Django syntax is now so close to SQLObject, I wish Django would just switch to it and extended it where needed, rather than continue to re-implement the rest of it.
To be fair, Django does make it possible to use other templating languages, and Zope3 Page Templates are available (along with a Django ZPT error formatter). If part of the argument for OOP and Python is code re-use, projects that re-invent not just the wheel but the entire car aren’t exactly shining examples of what’s possible in Python.
Yeah, you’re right. To me it’s like comparing one big factory to a network of 'normal’ factories ;)
Good write up, BTW
I don’t think it is fair to compare django-users with TurboGears —- majority of discussions happen in django-devels. Django community doesn’t have clean separation between audiences in both groups. A lot of users asking questions and getting help from developers in django-devels.
You have to be careful when committing to integrating another project into your own, however. Make sure you know what you’re getting into — ideally the owners of said project are either looking towards the same direction you are, or they are at least interested in others building on top of their work. Once you’ve integrated, you’re going to be somewhat locked in. If you regret it later on, it’s going to take some time and possibly heartache to break up with her^^H^H^H the project
I’ve regretted integrating at least one module (which shall remain nameless) after realizing the developers didn’t care about nor want to integrate my patches, and that their project didn’t turn out to provide me with everything I thought it would
There are some cases where building a new wheel is better than catching a sale on an old, used wheel. It seemed like a good idea when I saw the ad in the window, but when I got it home..
(of course we’re not talking about cars, we’re talking about open source software, so there’s always the option of forking a project if need be)
As Python programmers, we’re all in this together. Any way you slice it, I’m happy to see more people using Python for Web development.
But I feel obliged to respond to the somewhat ill-founded findings of this piece.
There’s no official Django downloadable version (tarball) yet, so of course list traffic is low at this point. Honestly, I’m surprised even 546 people are signed up for the Django mailing list. These are all early adopters.
That will all change very soon, though.
Where the comparison really gets interesting, though, is real-world projects.
Django is all about actual tools that get actual stuff done — not about buzz words or academic noodling. It’s about solving real needs, real fast, intelligently and efficiently.
This is a framework that has very strong, deep roots in Real Projects in the Real World. In fact, its genesis is from real-world projects.
Look at the list of Django-powered sites. LJWorld.com. Lawrence.com. Chicagocrime.org. Grono.net. And quite a few notable, high-profile entries are going to be added to that list soon, once they’re launched.
What public-facing, high-traffic Web sites have been powered by TurboGears?
Not even the TurboGears site itself is powered by TurboGears. (Source.)
Come on, man. Ya gotta eat your own dog food.
It doesn’t really count to say “So-and-so Web site uses SQLObject” or “So-and-so Web site uses CherryPy.” Tell me which high-traffic Web sites use TurboGears. You know, the front-to-back megaframework.
That’s the important comparison to make.
(a) The dogfood argument is bad. I get annoyed when I see people waste time building internal tools and infrastructure when there are perfectly good options that work well, like static files using existing generators. People who value getting things done shouldn’t scoff at practical solutions. We all know there’s enough stuff to work on without creating additional artificial requirements.
(b) The “Maybe A works and B works, but what about A plus B? Huh, what about that?” argument is pretty silly too. All the sites you list for Django were developed by the core Django developers. Glass houses, man. Let’s not get into some sort of scaling debate, or tick off different criteria which favors one over the other.
Why do people choose one or the other? I don’t know. Maybe it’s just that TurboGears had a screencast, which brought lots of people in early, and it had a tarball to go with. That’s not exactly a feature of the framework. Maybe it has lots of tourists, because people using any of the included projects will have a tangential interest. The question then is if those tourists can be converted to contributors. In many ways that’s what ultimately will be most important to any project — how it can attract and retain quality contributors. One of the points of this article is that TG is actually getting contributions from non-users, which is an unusual but interesting development.
And even the traffic itself has some interesting aspects. A ton of the traffic on the TG list has been setuptools related, i.e., installation related. That’s not exactly a good sign. But it has also led to a lot of feedback and improvement of setuptools.
How you interpret all this depends on what you are trying to interpret. And I’m not actually sure what we are trying to interpret here.
Comparing mailing list statistics seems like a exercise in punditry to me. And as for reinventing the wheel, whilst some wheels are continually reinvented (to my exasperation), there are areas where, due to prejudice or ill-informed commentary, there aren’t decent existing solutions. Adopting some existing non-solution might give blog pundits something to talk about (“projects X and Y together at last!”), but it won’t advance either “project X” or the state of the discipline very much.
I’m not sure either that comparing mailing-list communities is sensible.
One good thing though that have arised from the fast success of Django and TurboGears is that Python web development is only bound to Zope or Twisted-web. It brought lots of people to Python (as I can see on the CherryPy community where you have users with only a PHP background and who are happy to find a tool that doesn’t force them in one specific way of doing things).
The other nice thing is that both projects seem to learn from each other. In fact AFAIK both communities do get along well and I’m a little afraid to see them being put against each other like that.
Regarding code-reuse, I agree though that it’s pretty cool because for instance when the TG community finds a problem or fixes a bug with one the of components of TG everyone benefits from it. That’s very nice.
A couple of responses to Ian’s comment:
To be precise, grono.net looks like it’s still made mostly in Java, it’s probably going to be made in django (if the site doesn’t implode).
I found Django first, and was impressed (still am), thought it seemed tight, but I didn’t like
a) builtin templating language (I like attribute based languages). I realise you’re not tied to a templating tool with Django, but it helps if the default is good.
b) use of keyword selectors in sql model. 'name__contains’. Not very pretty.
c) regex url matching. Very powerful, but uneccessarily complex, IMHO. Plus it’s not DRY, as you model url patterns in code anyway, you might as well define them there.
I did like the builtin admin interface, though, that’s sweet.
Then I came across TG, and what I liked was:
a) attribute based template language. Kid is the missing piece of the puzzle and the killer feature for me. Easily the best templating system I have come across.
b) SQLObject is a little clunky, but works better than Django’s, although this is just syntactic preference – them seem functionionaly equivalent
I didn’t like cherrypy’s url mechanism, but I prefer it to Django’s
So I’m on the TG list, and have since learned more about TG and quite like it. Havn’t played with Django as much, but first impressions last, and those initial things are what put me off.
Also, Django folks seem a little more 'Django is the best ever!’ than the TG guys, who seem a little more pragamatic 'whatever works’. Interestingly, a large number of the active TG list memebers seem to be Mac users…
I agree with Adrian, comparing mailing-list membership is stupid.
When talking about “rapid web development” (bzz) frameworks (the kind of thing you will use @work for real customers) the first things that comes to my mind are: robustness, scalability, security (oh, and licencing)... not the hype, the mailing lists members or the code-reuse of other dependencies…i meant projects ;)
I find the article a bit pointless.
Oh look! We’re turning into the Perl community ca. 10 years ago!
I tried not to make it into a “so and so is better because there’s more people on the mailing list” article. The point is not that one is better, nor that the traffic on the mailing list is in any way indicative of how “good” one is.
The point is that more people are able to help out with more issues on TG because it builds on existing communities. People that don’t even use TG can jump in and help out with the problems they are familiar with. This provides many ways to get “into” TurboGears as well as showing the viability for using many of the parts outside of TG entirely (as they frequently are).
If anything, I didn’t show enough mail list traffic to highlight this point. I didn’t show the increases in SQLObject traffic, nor the extra FormEncode traffic and patches that have come in. All the parts that compromise TG are now getting more eyes looking at the code, which as Ian pointed out is a good thing.
I’ve seen people choose one programming language over another purely because large libraries already existed in one language but not the other. They want to get actual things done and realize that rewriting libraries/frameworks/ORMs is not a good use of time and not time many have.
Django was used for comparison here because its a fairly high profile web framework that eschews code re-use. This isn’t a “TG is better than Django” post, it’s a “code re-use is good” article meant to highlight the benefits being realized by members of the communities that TG builds upon. This compares a different aspect entirely of TG and Django, in that TG’s advances frequently help out other communities.
My experience is more on the cherrypy/sqlobject side, so I’m more curious about TG. But let’s face it, Adrian’s argument is rock solid, there are some large scale implementations of Django while none so far of TG (in my opinion, regarding implementations, Django is even better than Rails).
And also obviously true, what makes TG and Django great is the same: python (with SQL and without XML).
Adrian’s argument is solid, and so is Ben’s. Django has shown success in real world applications, and TurboGears has spurred more contributions to already-existing, reusable frameworks. They are mutually exclusive points.
It’s sad to hear the “can it scale” argument coming up in Python web development, the Rails people have been fighting this as well. The main components in TG that could hamper its ability to scale have all been used in large environments, many in combination similar to TG, though not atcually TG itself.
I started playing with Django back when it got released. I thought it was great since it was using Python and it seemed like a complete web framework. But I have since stopped using it. Why? To me, Django fails in its use of databases. It stores a lot of django stuff in a database and thats not acceptable to me. That would be fine if its a standalone system, but what if your database is used by multiple systems? I don’t want some project throwing various tables up on my database.
This is why I have moved on to looking at Ruby on Rails and TurboGears. I haven’t actually played with TurboGears yet since I want to get a feel for Ruby on Rails first. However, I do plan to really try it out since I would prefer to use Python in my development.
wavydavy, I’d be very interested to hear what style of “url mechanism” you would like. I’m toying with the concept of allowing/providing multiple dispatch mechanisms with CherryPy.
jeff: Since Django was first open-sourced, we’ve eliminated four of those database tables, with the goal to remove them all. Stay tuned to Django development: It moves very quickly, improving constantly. :) A good way to stay tuned is to subscribe to the weblog’s RSS feed (http://www.djangoproject.com/rss/weblog/).
I’m just glad I don’t have to switch to Ruby no more. :P Go Django, TurboGears, Subway, and Zope3!