Web2py

web2py

Developer(s)	web2py developers
Stable release	1.69.1 / October 28, 2009 (2009-10-28)
Repository	github.com/web2py/web2py
Written in	Python
Operating system	Cross-platform
Type	Web application framework
License	GNU GPL v2.0 (with exception)
Website	homepage mailing list

Web2py is an open source web application framework. Its primary goal is to support agile development of fast, scalable, secure and portable database-driven web-based applications. Web2py is written in the Python language and is programmable in Python. Since web2py was originally designed as a teaching tool with emphasis on ease of use and deployment, it does not have any project-level configuration files.

Web2py was inspired by Ruby on Rails (RoR) framework and, as RoR, it focuses on rapid development, favors convention over configuration approach and follows Model-View-Controller (MVC) architectural pattern. But web2py is based on Python and provides a comprehensive web-based administrative interface, includes libraries to handle more protocols, and can run on the Google App Engine.

Web2py was also inspired by Django framework and, as Django, it has the ability to generate forms from database tables and it includes an extensive set of validators.

Thanks to Python, web2py is less verbose than Java-based frameworks and its syntax is much cleaner than PHP-based frameworks. This makes applications simpler to develop, easier to read and maintain.

Overview

Web2py is a full-stack framework in that it has built-in components for all major functions. Examples include:

• handling HTTP requests, HTTP responses, cookies, sessions;
• speaks multiple protocols HTML/XML, REST, ATOM and RSS, RTF and CSV, JSON, JSON-RPC and XML-RPC, AMF-RPC (Flash/Flex), and supports SOAP ;
• CRUD API;
• multiple authentication mechanisms and role based access control;
• database abstraction layer that dynamically generates SQL and runs on multiple compatible database backends;
• settable RAM, disk, and memcached-based caching for scalability;
• full internationalization;
• jQuery for Ajax and UI effects;
• automatic logging of errors with context.

Web2py helps the developer to follow sound software engineering practices such as

• the Model-View-Controller (MVC) pattern;
• self-submission of web forms;
• server-side sessions;
• safe handling of uploaded files.

Web2py has a focus on security by providing safe default mechanisms. This prevents the most common vulnerabilities.

Web2py differs from other web frameworks because it was originally designed as a teaching tool at DePaul University and thus has a very shallow learning curve. For this reason it is designed to be easy to set up (requires no installation) and provides a fully web-based development environment.

Web2py is built following the WSGI protocol, the latest Python protocol for communication between web server and web applications. It also provides handlers for CGI and the FastCGI protocols, and it includes the multi-threaded, SSL-enabled CherryPy wsgiserver.

According to agile practices, web2py has frequent releases and easy to update. But it rarely requires upgrades. Its developers have not broken backward compatibility since v1.0 in 2007, and they pledge not to break it in the future.

Some rare features

Web-based integrated development environment (IDE)

You can do all development, debugging, testing, maintenance and remote database administration without third party tools — via a web interface, which itself a web2py application.

You can manage internationalization (add language and write translations) from this IDE. Each application has an automatically generated database administrative interface, similar to Django. It also includes web-based testing and a web-based shell.

You don't have to use this IDE, applications can also be created from command line and developed with other IDEs. For example Wing IDE allows you to graphically debug a web2py application as you interact with it from your web browser, you can inspect and modify variables, make function calls etc.

Flexible views

The common Hello World program with web2py in simplest case (simple web page without template) would look like:

def hello():
    return 'Hello World'

Python based templating: web2py includes a fast, pure Python based template language

• with no indentation requirements;
• with server-side Document Object Model (DOM).

Joomla templates: you can convert Joomla 1.x templates to web2py layouts.

Generic views: if you write a controller without view, you automatically get generic view that render the variables returned by the controller. So you can develop the logic of your application without writing one line of HTML and you have a working prototype.

Strong security

Web2py has a focus on security. It never had any security issue reported. Top 10 security issues according to OWASP and what web2py does about them :

• Cross-site scripting (XSS): web2py, by default, escapes all variables rendered in the view, thus preventing XSS.
• Injection Flaws: web2py includes a database abstraction layer that makes SQL injections impossible.
• Malicious File Execution: web2py allows only exposed controllers to be executed and thus prohibits malicious file execution.
• Insecure Direct Object Reference: web2py does not exposes any internal object, moreover web2py validates all URLs thus preventing directory traversal attacks.
• Cross-site request forgery (CSRF): web2py only uses session cookies and prevents double submission of forms.
• Information Leakage and Improper Error Handling: web2py provides a built-in ticketing system. As of 2009-10-05, it is the only such web application framework.
• Broken Authentication and Session Management: web2py has a built-in mechanism for session management and uses a cookie to store the session id.
• Insecure Cryptographic Storage: web2py, by default, uses the MD5 algorithms to hash passwords. Other algorithms such as the HMAC are also available.
• Insecure Communications: web2py works with Apache and mod_ssl to provide strong encryption of communications.
• Failure to Restrict URL Access: web2py mapping URLs into function calls. web2py provides a mechanism for declaring which functions are public and which require authentication/authorization.

Ticketing system

Each web2py application comes with a ticketing system:

• If an error occurs, it is logged and a ticket is issued to the user. That allows error tracking.
• Errors and source code are accessible only to the administrator, who can search and retrieve errors by date or client-IP. No error can result in code being exposed to the users.

Portable cron

Mechanism for creating and running recurrent tasks in background. It looks for an application-specific crontab file which is in regular crontab format. Three modes of operation are available:

• Softcron: cron routines are checked after web page content has been served, does not guarantee execution precision. For unprivileged Apache CGI/WSGI installs.
• Hardcron: a cron thread gets started on web2py startup. For Windows and CherryPy/standalone web2py installs.
• Extcron: cron functions get force-called from the command line, usually from the system crontab. For Unix/Linux systems and places where the cron triggers need to be executed even if web2py is not running at the moment. Also good for CGI/WSGI installs if you have access to the system crontab.

Source code protection

Web2py can compile web applications and you can distribute them in bytecode compiled form, without source code. This helps but does not guarantee source code protection due to the existence of disassemblers and decompilers for Python bytecode (*.pyc and *.pyo files):

• Dis module of the Python standard library allows you to disassemble, but not decompile Python bytecode.
• Open source Decompyle Python disassembler and decompiler converts Python bytecode back into equivalent Python source. It accepts bytecode from any Python version between 1.5 and 2.3 inclusive. It doesn't support versions above 2.3 and hard to use.
  • Commercial decompyle service origins at Decompyle. It decompiles Python versions 1.5 up to 2.6, with some minor limitations.
• Open source UnPyc is a tool for disassembling, analyzing and decompiling Python bytecodes, with various success. UnPyc supports Python v2.5 and v2.6.

However it is quite easy to extend Python with C or C++, to achieve:

• more efficient code;
• adequate safety of intellectual property;
• reduced security exposure of private informations in the code — such as usernames and passwords.

Installation, upgrade

Web2py is based on Python, but it is not a pre-requisite to run web2py. Download from its homepage , unzip, execute are all that is needed to get started with web2py — on any operating system. Web2py comes packaged with anything you need for development including web server and database. No installation procedure, no configuration and no shell scripting required.

First steps

On Windows

After download, unzip it and click on web2py.exe. Choose a password, click the link of the administrative interface, enter same password.

To run from source type:

 python web2py.py

To run as a Windows service follow this guideline:.

On Mac OS X

After download, unzip it and click on web2py.app. Choose a password, click the link of the administrative interface, enter same password.

To run from source type:

 python web2py.py

On Unix systems, including Linux and BSD

Run from source:

 python2.5 web2py.py

Choose a password, click the link of the administrative interface, enter same password.

On Google App Engine

You must deploy the web2py engine in its source code form. So GAE runs web2py which in turns runs the web2py applications under it.

Detailed instructions about deploying web2py and running web2py applications on GAE.

Performance problem on low activity sites

GAE compiles the .py files into byte code on startup and it remains cached in GAE for less than a minute.

On low activity sites a trick for performance a cron job — under GAE or elsewhere — which will ping one of your web2py controller files in order to keep its bytecode fresh in the GAE cache .

Even better to use the task queue to hit a URL that simply returns a non-200 HTTP status code, so the task will retry indefinitely at up to 10 Hz (10 calls/sec) . You can specify how often a task is called.

Caveats

After installation, every time you run it, web2py asks you to choose a password. This password is your administrative password. If the password is left blank, the administrative interface is disabled.

The administrative interface (/admin/default/index) is only accessible via localhost (remote access available only through SSL — SSH tunnel or HTTPS) and always requires a password.

Operating systems, Python versions & implementations, virtual machines, hardwares

Web2py runs on Windows, Windows CE phones, Mac, Unix/Linux, Google App Engine, Amazon EC2, and almost any web hosting via Python 2.4 /2.5/2.6.

Python 3.x is not fully compatible with 2.x . Currently developers of web2py doesn't plan to make the framework compatible with Python 3.x .

Web2py since v1.64.0 runs unmodifiedly on Java with Jython 2.5, without any known limitation .

Web2py code runs with IronPython on .NET . Limitations:

• no csv module (so no database I/O);
• no third party database drivers (not even SQLite, so no databases at all);
• no built-in web server (unless you cripple it by removing signals and logging).

These are because csv, sqlite, signals and logging modules are not present in IronPython.

VMWare appliance: coming soon.

You can run the binary versions of web2py off a USB drive or a portable hard drive without dependencies, like Portable Python.

Web server arrangements

You can serve requests via HTTP and HTTPS with the built-in CherryPy server , with Apache , Lighttpd , Cherokee , Nginx and almost any other web server through CGI, FastCGI, WSGI, mod_proxy , and/or mod_python.

Upgrading web2py

See web2py manual, 2nd edition, page 309 for upgrading steps.

Database handling

Database abstraction layer (DAL) of web2py dynamically and transparently generates SQL queries and runs on multiple compatible database backends. You don't need to use database-specific SQL commands. You can however explicitly issue SQL statements, according to the specific database driver.

SQLite is included in Python and thus web2py, so that is the default backend. You can simply change a connection string to connect to Firebird, IBM DB2, Informix, Ingres, Microsoft SQL Server, MySQL, Oracle, PostgreSQL, and Google App Engine (GAE) with some caveats. Specialities:

• Since web2py v1.17 you can do distributed transactions with PostgreSQL v8.2 and later , because it provides API for two-phase commits. Support for Firebird and MySQL on the way.
• GAE is not a relational store, see web2py manual, 2nd edition, page 305 for guidelines and caveats.

This DAL is fast, at least comparable with SQLAlchemy and Storm .

Web2py can connect to multiple databases at once. Performs automatic transactions. Even creates and alters tables for you when required.

This is a DAL, not an ORM. An ORM maps database tables into classes and records into instances of those classes. This DAL instead maps database tables into instances of a class and records into instances of another class. It has very similar syntax to an ORM but it is faster, and can map almost any SQL expressions into DAL expressions. Usable without web2py .

Automatic database migrations

Web2py supports database migrations. You just change the definition of a table and it ALTERs the table accordingly. For example if you change a field from boolean to string, the database is altered and the values are converted from boolean to string for existing records.

In web2py migrations are automatic, but can disabled for any table. Web2py logs all migrations and migration attempts in a file so you can see what changes are made to the database.

Limitations:

• SQLite does not understand migrations well. In particular it can't alter table and change a column type. It just stores new values according to the new type.
• On GAE there is no concept of alter table so you can add columns and you can drop (they will not be dropped, just ignored). You can change some attributes of a column (like length) since they are ignored anyway. Some migrations will not work.

Applications

Ready to use applications with source code

You can find many free and ready to use web2py plugins and applications with full source code, ready to customize, with various licences.

Mostly on this list of its homepage, some of them:

• Ajax SpreadSheet: an embeddable spreadsheet.
• Chat: Ajax web chat supports multiple users and rooms.
• Cookbook: a simple appliance that allows users to post cookbook recipes. Plus a step-by-step tutorial about creating it.
• Blogs:
  • KennethDamianBlog: an easily customizable blog.
  • WordPressClone: a blog that looks identical to Wordpress.
• GrooverWiki: a wiki.
• KPAX CMS: a full CMS. Does web pages, wikis, blogs, chats, news, groups, permissions, RSS feeds.
• Podcasts: this program let you receive, watch and listen to podcasts from various sources.
• Reddish: a Reddit clone for the Google App Engine.
• QrOne CSS Designer: application to build CSS layouts.
• DamianLogAnalizer: log analyzer application that does charting.
• IsUp: you list a bunch of URLs and it probes them every 30 seconds. It maintains a database of when the URLs were down.

Three of them elsewhere:

• PyForum, a full-fledged message board system.
• T2, a web2py plugin that implements web development patterns — registration, login, logout, groups, access, attachments, comments, previews, etc. —, with extensive documentation.
• T3, a wiki application that runs everywhere, including on Google App Engine. Super-powered: it allows admin to define database tables (using DAL syntax) and to embed Python code into the source code of wiki pages.

A quick reference to the high-level web2py functionality that was added in T2 and T3: T2/T3 cheat sheet.

Websites and web applications powered by / built with web2py

Two lists about who is using it:

• Websites Powered by web2py.
• Who Uses Web2Py.

Licenses

Web2py code is released under GNU GPL v2.0 license. This license does not extend to third party libraries distributed with web2py (which are all MIT or BSD type licenses) nor does it extend to applications built with web2py.

The license of the official binaries is freeware . You can use and redistribute them but not charge for them. You are not allowed to distribute another binary version of web2py.

Applications built with web2py can be released under any license as long they do not contain web2py code. In particular they can be bytecode compiled and distributed in closed source. The admin interface provides a button to bytecode compile.

It is fine to distribute web2py (source or compiled) with your applications as long as you make it clear in the license where your application ends and web2py starts.

You are not allowed to:

• claim you built web2py;
• create a derivative of web2py in violation of the GNU GPL v2.0;
• distribute a binary version of web2py other than those officially released by Massimo DiPierro.

web2py is copyrighted by Massimo DiPierro. The web2py trademark is owned by Massimo DiPierro.

Publications

• Online documentation on its homepage, with cookbook, videos, interactive examples, interactive API reference, epydocs (complete library reference), FAQ, cheat sheet, online tools etc.
  • Cheat sheet of web2py.
• Web2py wiki — it's actually built using web2py.
• Web2py slides.

Videos

• web2py Enterprise Web Framework Tutorial.
• web2py "Shootout" video tutorial.
• web2py on the Google appengine.
• More video tutorials on Vimeo.

Printed

• Web programming with web2py, Python Magazine, June 2008.
• The official web2py manual, written by Massimo DiPierro:
  • 1st Edition: out of print. Wiley; September 16, 2008; 256 pages; ISBN 978-0-470-43232-7.
  • 2nd Edition: web2py Manual. Wiley; August 26, 2009; 341 pages; ISBN 978-0-470-59235-9. Read it online. Errata for the book.

Background

Support

Community support is available by

• joining the web2py mailing list at Google Groups;
• using the web2py knowledge base.

As of 2009-10-02, worldwide 15 companies provide commercial web2py support .

Developers

Lead developer: Massimo DiPierro (Associate Professor of Computer Science at DePaul University in Chicago).

As of 2009-10-30, homepage of web2py lists 49 "main contributors" .

Third-party softwares included in web2py

• Python-based components:
  • a fast , HTTP/1.1-compliant, multi-threaded, SSL-enabled and streaming-capable WSGI server from CherryPy;
  • fcgi.py: a FastCGI/WSGI gateway;
  • simplejson: a simple, fast, complete, correct and extensible JSON encoder and decoder;
  • markdown2: a Markdown processor, so you can write using this easy-to-read, easy-to-write plain text format, then on the fly convert it to structurally valid XHTML (or HTML);
  • PyRTF: an RTF document generator;
  • a syntax highlighter;
  • PyRSS2Gen: an RSS generator;
  • feedparser: to parse RSS and Atom feeds.
• JavaScript-based components:
  • jQuery: a lightweight JavaScript library;
  • EditArea: a free editor for source code;
  • nicEdit: a lightweight, cross platform, inline content editor.
• C-based components:
  • SQLite: a relational database;
  • memcached: a general-purpose distributed memory caching system.

History and naming

The source code for the first public version of web2py was released under GNU GPL v2.0 on 2007-09-27 by Massimo DiPierro as the Enterprise Web Framework (EWF). The name was changed twice due to name conflicts:

• EWF v1.7 was followed by Gluon v1.0.
• Gluon v1.15 was followed by web2py v1.16.

Web.py has a similar name, but they are unrelated.

References

Web2py enlisted in two source code repositories:

• latest development snapshot for Bazaar version control system on Launchpad: web2py snapshot for BZR;
• latest development snapshot for Subversion version control system on Google Code: web2py snapshot for SVN.

Freshmeat, Ohloh and Google Trends — despite their flaws — provides some valuable insights into background, composition of code, vitality and popularity of software development projects:

• web2py on Freshmeat;
• web2py on Ohloh;
• web2py on Google Trends.