Groovie

It’s been a busy month for Pylons, with lots of changes for the big internal API change in 0.9. The great news for those making Pylons apps right now with 0.8.2 is how few backwards compatibility issues there are. Most of the big changes take place under the hood and compatibility objects are present to mitigate the massive breakage that I’ve seen happen in other framework upgrades.

This is going to remain a big focus on future development in Pylons too, and the API in 0.9 is solid enough that I don’t see anything but minor tweaks in the future (1.0 and beyond). Pylons based apps will be easy to maintain and upgrade thanks primarily to WSGI.

No NIH Syndrome Here!

We’ve taken some cues from the Django project that we believe make for a cleaner request-cycle. In Pylons 0.9, controllers are expected to return a response object, and for convenience a method is included that renders a template to a new response object and returns it (This will look very familiar to Djangonauts).

The command was integrated with a slightly more enhanced version of Buffet along with the Beaker Session/Caching middleware. The end-result is a powerful command that not only can render templates in any Template-Plugin compatible template language, but the rendered result can also be cached. It’s a great way to utilize template languages which might not be all that quick by themselves.

Sample controller in Pylons 0.9:

from myproj.lib.base import *

class UserController(BaseController):
    def show(self, id):
        # Cache based on id in the URL, for 30 seconds
        return render_response('/user/show.myt', cache_key=id, cache_expire=30)

    def index(self):
        # Just for fun, use Kid to render the index
        return render_response('kid', 'user.index', cache_expire=15)

Being able to easily cache any template in any template language makes it very easy to sprinkle in caching when you need to handle massive loads yet stay dynamic.

Another new feature present in the latest Routes and Pylons is resource mappings, which automatically generate routes for you with HTTP method restrictions. This makes it easy to setup controllers and their actions for specific HTTP verbs (aka, REST-ful URL’s and web services). The implementation of this feature was directly inspired by the Simply Restful Rails plug-in that was also demo’d by David Hansson in his Resources on Rails talk. Being able to discriminate valid routes based on HTTP method was brought up in the past and I’m happy to have seen an implementation that solves the issues I originally had with the idea.

The one feature still in development is to easily discern the content type requested, which was also inspired by Rails. Josh Triplett has written some code that deals with the ugly task of parsing the HTTP Accept headers, and I’m working on adding it into Paste for easy re-use by all. Combined with Routes, it’ll provide a clean and easy way to setup web applications that can serve multiple forms of content from a single action.

Conferences

Pylons was represented at EuroPython by the other lead developer of Pylons, James Gardner. For those still trying to grasp WSGI, he gave a talk on WSGI and middleware that looks like it was quite interesting (I wasn’t there unfortunately.)

Other talks involving Pylons at EuroPython:

• Dr. Rob Collins gave a presentation on Factory monitoring with Pylons, XML-RPC and SVG
• James Gardner’s full talk on Pylons, and slides of his Pylons talk
• Web Framework Shoot-out between Django, Zope, TurboGears, and Pylons (If more were covered, please let me know)

I’ll be at OSCON for those interested in chatting about Pylons, Python, or Python web frameworks later this month.

Pylons 0.9 Release

We’re currently wrapping up new features in 0.9, to make sure the resource mapping capability is present before a feature-freeze for release. Hopefully the release will be out within the next 2 weeks, if you haven’t checked out Pylons before I’d highly suggest taking a look at 0.9!

I was talking with my grandfather recently about various technology bits when he mentioned this awful trend he had started to notice on websites. Just the mention of an awful trend with the knowledge that his eyes weren’t what they used to be elicited similar anger from me, mainly, that there seems to be a significant increase in low contrast websites.

Disclaimer: I realize that this blog also commits some of these sins I’m about to rant on, and I’m in the process of changing them.

This trend is very apparent in many blogs, and some are getting so bad I want to scratch my eyes out after reading them. This isn’t the blog author’s fault, as they’re usually selecting a theme that “looks good” on a glance. Trying to read the text when the theme is in use can be a nightmare.

Consider this blog entry, it looks decent contrast-wise to me on first glance. But trying to read the text of the comments at the bottom made my eyes burn. Or the Shopify page which manages to consistently use the same color text as the background behind it. This is a pretty strict Do Not per US Section 508 Code (Web Accessibility Standards). I’m also fairly certain the W3C Accessibility standards frown on low-contrast layouts, especially if no option is given to switch to a high-contrast scheme.

This issue doesn’t just affect old people or those with handicaps. I have no color blindness or issues sorting out colors yet these themes still routinely give me headaches. The thing to keep in mind when using different different shades of the same color on top of each other is that while you might have a wonderful color-calibrated monitor, the vast majority of Internet users do not. Even those with perfect vision are typically at the mercy of the quality of their CRT/TFT and the color settings that may or may not be ‘proper’.

For the designers out there, I realize that these low contrast themes can look pretty, but please put out more high-contrast themes that look great. Now to fix-up my blog…

Ah yes, time for another release. This time its Routes 1.3.1, nothing too big to see here, thus the minor version increase. However if you were using Routes with the prefix option its highly recommended that you upgrade as there were 2 bugs specifically when used with that option that have been fixed.

Also, for those that want all their URL’s to end in a /, there’s now a append_slash option that will ensure URL’s are generated exactly how you want them to be.

Enjoy!

Finally pushed out release 0.8 of Pylons, a bunch of great features bundled up in a slick package that makes web development easy and brings re-use to new levels. I’m not going to repeat the full announcement here, but I’d suggest checking it out if you’re the slightest bit interested in a framework leveraging WSGI for re-use and ease of use.

Now that 0.8 is out, its already time to get to work on 0.9 which is going to have a slightly more stream-lined WSGI stack but other than that won’t be very different from 0.8 from a usability perspective.

Ian Bicking noted that I was considering building the next Pylons on RhubarbTart which is sort of true. Julian Krause, Ian and myself are working on WSGI and associated components that will find themselves in all of our “frameworks” and most likely quite a few others too. That’s really one of the best things about writing solid pieces of web code in a more component style vs. a hulking framework, portability is great and the re-use is fantastic.

So far, Routes has found itself embedded in at least a half-dozen different frameworks and I’ve heard some rumors its to be ported to PHP for the Zend framework as well. The WebHelpers package is also picking up users and is being used by some Djangonauts which is great as Django is generally a fairly ‘helper’ friendly framework.

Ah yes, back to the topic, Pylons has tied together these pieces of code in a very seamless fashion that make it easy to write what you want without forcing you down a path that may confine you later. On the other hand, Pylons currently doesn’t have some of the friendlier elements that frameworks like TurboGears and Django both have. Such as a pretty Admin interface, or an extensive Toolbox app, but that stuff is in the works where appropriate.

Our main desire with this release was to get solid footing for the integration of these parts, and a good architecture for building web applications. To that end, we think Pylons is a very solid performer and will adapt easily to a wide variety of needs. You’ll also get a framework that makes it a snap to plug-in new middleware whenever something striking your fancy appears on the radar, and you can keep using the parts you like best, even if you stop using Pylons.

As the date edges closer to a Pylons release, I find myself already thinking about future directions of Pylons. I’m obviously rather biased when discussing Pylons as I’m one of its creators, though I still find that thought humorous as the vast vast majority of the code that resulted in an excellent framework is not actually in Pylons, nor did James Gardner (the other Pylons dev) and myself write it.

Matt mentions in a blog entry on Python Web frameworks, maybe its just too easy to write a Python web framework. I didn’t originally see myself writing a framework as such, but it became a fairly logical conclusion to the work I was doing at the time and the framework was originally extracted (like many others) from a large production application. It really is quite easy to write a web framework, and there’s tools available that make it even easier.

WSGI has changed the point at which re-usability is possible in a Python web framework as well, the only possible result I see in the future is more frameworks (Not in the way people think though). Many would argue this is “bad”, however I think due to the different point in re-use made possible by WSGI there will actually be more collaboration and less web developer divisions despite how many ‘frameworks’ are out there (once they’re more WSGI-ish that is). Consider this analogous in some ways to how many linux distributions are out there, and the fact that people can switch between them very easily.

So what went into building Pylons?

The Basics

First, there’s a few things you’re going to need in even the most basic web framework:

• Dispatcher
• Request API/Object

That’s really the utter basics, and there’s frameworks that don’t do much more than this. Obviously many people are going to want more…. like sessions. The question that WSGI raises is why lock a session interface to a framework, when it can be re-used as WSGI middleware? While the term “WSGI Middleware” can be rather intimidating, I previously covered why WSGI is actually quite easy despite the sometimes scary or just annoying term. It’s something to be reckoned with, and there’s no excuse not to get familiar with it if you’re doing web development in Python.

As I needed something rather reliable and sturdy, I went with Myghty’s session and request API. For dispatching, I used a custom resolver utilizing Routes that dispatches to a controller and action (controller method). This results in a fairly MVC’ish style web architecture that’s rather extensible.

So check those off the above list plus sessions:

• Dispatch – Routes based
• Request API – Myghty’s HTTP request object
• Sessions – Myghty’s session interface, which is also now available as WSGI middleware

At this point, we’d have a framework that can interpret a URL, setup an easy to use request object, and call your controller. Your controller would also be able to save data using sessions. We’d have a bit of a problem distributing such a framework though, as it probably wouldn’t be very convenient to setup and install.

In the case of Pylons, so far it means I’ve written just a class to handle dispatch, locating the controllers (actually, Routes will do this), and calling them. A measly hundred or so lines of Python code.

Defaults, Structure Creation, Stand-alone Server

What really helps people get started quickly with a framework is if its easy to install, creates a basic structure of a working web application for them to get started with, and some way to run it. So our new additions:

• Web Framework Installer
• Template for starter Web Application
• Stand-alone Server

This can quickly be a fairly substantial amount of code to write, especially if you want cross-platform installation, and a system that also runs on multiple platforms. Not a problem though, Paste quickly makes the last two of these quite easy, and setuptools handles installing your framework and any requirements it may have. I should note that setuptools isn’t perfect, but it sure beats asking users to go to a half dozen sites and install various packages.

Paste is divided into 3 parts, though it can be tricky to see the relation between them. The core part of Paste contains wsgi middleware parts that will likely appeal to many, and some other basic request handling functionality. PasteScript contains the structure creation bit used in Pylons that generates the starting template. There’s also PasteDeploy which comes in useful later when deploying and running web applications made with a framework using it.

We’re looking at a decent little framework so far, now that we can install it, quickly start a new project, and run it. What’s next?

Templating and Object-Relational Mappers

Some frameworks, typically known as “full-stack” frameworks try and make more choices for you. They aim to fill your needs top-to-bottom, or at least their vision of your needs. While SQLObject is a very popular ORM, it seemed that good use of a layout and basic Python import statements would make using any ORM just as easy.

In Pylons, a models directory is provided, with some commented out suggestions for how you’d go about setting up an ORM. There’s hooks provided in the base application module that’s imported by all your controllers, so its easy to define your ORM classes and use them in your controllers.

One of the other objectives for Pylons was to try and be very ‘Pythonic’. That is, it should re-use as much of a developers Python knowledge as possible. This is used by default for templating since Myghty uses normal Python syntax for its templates in addition to providing powerful caching functionality and great re-use through components.

Not wanting to force anyone into something they couldn’t stand (template languages can be a love-hate thing), we also implemented the TurboGears Template Engines plug-in functionality. This made it very easy to let people use the template language of their choice, in a fairly uniform manner. It was also pretty minimal to implement the template language renderer as the Buffet project provided a great head-start.

Our new list is looking rather complete for a “full-stack” framework:

• Web Framework Installer
• Template for starter Web Application
• Stand-alone Server
• Dispatcher
• Request API/Object
• Sessions
• Templating
• Database Integration

Making the most of Middleware

This is one of the areas where Pylons was able to make some great leaps, with very little actual code. In several areas, thanks to the use of middleware, Pylons is able to offer features other frameworks are still working on.

To start off with, there’s the excellent EvalException middleware which provides an AJAXy exception catching system I have yet to see in any other framework. We’ve formatted it slightly to fit in nicely with Pylons, and it works like a charm. Extremely useful for those times when you want to interactively debug a web application to see what’s what, and it’s a lot quicker than putting print statements all over.

Another important bit, that a lot of frameworks skimp on is unit testing. Using Paste’s fixture middleware, its easy to test your web application. Pylons adds a few objects to the response you can test with, so you can ensure that the session was setup properly, the right template components were called with the right arguments, etc. In the future we’ll likely add some defaults to make using twill an easy option.

The best thing about all of this of course, is that these useful parts can be integrated seamlessly and re-used by other frameworks.

Taking the framework out of Framework

Given how most of these parts are definitely not unique to Pylons, nor are they intended to be, it shouldn’t be long until more frameworks start using the great modularity that Pylons is utilizing. A lot of these parts will likely become standardized to an extent, so that there’s even less barrier to switching frameworks.

At this point, Pylons becomes less of a “framework” in one sense, and more a set of defaults and structure for how a Python web application should be put together. Pylons has more features of course that I haven’t described, the WebHelpers functions are made easily available for use in templates, more Paste middleware is used for slick traceback email’s when you’re in production mode, PasteDeploy makes running your Pylons webapp easy in a variety of situations, etc.

It’s a very easy-to-extend model, with little need to put great amounts of application-y type stuff into the framework itself. It also keeps its components separate for easier testing, out of the actual framework. This means that while Pylons comes with a great set of middleware and parts set up for you, its very easy to swap in your preferred template language, your preferred ORM, a different exception handler, etc. The choice is up to you, but the defaults are set to a good starting point (also called “convention over configuration”).

Upping the Re-usability Ante

Increasing re-usability is what I’d consider the future for Python Web Development. With WSGI middleware driving re-usability, such concepts that “to use Feature X, you must use your Framework Y” just doesn’t apply. Unifying development work on excellent components that can be re-used in any WSGI-compatible framework makes Python Web Development better.

Pylons isn’t alone in aiming for this style of framework, Ian Bicking is working on a project that starts the opposite direction, with just a layout and you add sessions, templating, etc. as you need it. TurboGears is adding more Paste-compatible features that will shortly make it trivial for them to add in the EvalException middleware (assuming they haven’t already, I haven’t checked lately) and other great components. Different frameworks have different levels of re-usability, those that are built with re-usability in mind at the beginning will likely be able to adapt quicker to new demands and requirements, and take maximum advantage of the great middleware being created.

While having more people use Pylons would be great, it isn’t necessary for Pylons to become a better framework. Having more people use the WebHelpers package, or make their framework Paste-compatible, or use Routes, or Myghty’s powerful caching/session API’s all helps Pylons. It also helps any other framework using these components, and that’s what counts the most.

Here’s the final tally of Pylons features and where they came from:

• Web Framework Installer – setuptools
• Template for starter Web Application – PasteScript
• Stand-alone Server – Paste
• Dispatcher – Routes / Myghty
• Request API/Object – Myghty
• Sessions – Myghty
• Caching – Myghty
• Templating – Myghty, or any that support the TurboGears Template Plug-In
• Helper functions/AJAX – WebHelpers
• JSON – simplejson + Pylons decorator
• Global “convenience” objects – Pylons
• Database Integration – SQLObject, SQLAlchemy, anything else
• Interactive Debugging – Paste
• Traceback E-mails – Paste
• Webapp Unit Testing – Paste
• Webapp Deployment – PasteDeploy
• Webapp Distribution/Installation – setuptools

This is just the default set-up, its trivial to add more middleware, which would make this list very, very long and includes such things as OpenID Authentication, authenticated session tickets, along with other great stuff.

The Pylons Code-base: Pylons Module Reference