Comet (programming)
Comet is a software concept that enables web servers to send data to the client program (normally a web browser) without having any need for the client to request it. It allows creation of event-driven web applications, enabling real-time interaction otherwise impossible in a browser.
Motivation
Traditionally, web pages have been delivered to the client only when the client requested it. For every client request, the browser initiates an HTTP connection to the web server, which then returns the data and the connection is closed. The drawback of this approach is that the page displayed is updated only when the user explicitly refreshes the page or moves to a new page. Since transferring entire pages takes a long time, refreshing pages introduces a long latency.
To solve this problem, Ajax can be used which allows the web browser to request only that part of the web page that has changed and update that portion accordingly. Since the overall data transferred is reduced, latency is also reduced, and overall responsiveness of the web site hosting the application increases. Further, by using asynchronous background data transfer, where the user works with partly received data as the rest of the data is being retrieved, the responsiveness of the web application can be further increased.
But this practice also suffers from the problem that the client has to request some data before it will be sent by the server. This problem becomes a major hurdle when designing applications which have to wait for some event to occur at the server side, such as some other user sending some data to the server, before it can proceed, but has no information when the event will occur.
A solution would be to design the application such that it will intermittently poll the server to find out if the event has occurred. But this is not an elegant solution as the application will waste a lot of time querying for the completion of the event, thereby directly impacting the responsiveness of the application. In addition, a lot of network bandwidth will be wasted.
A better solution would be for the server to send a message to the client when the event occurs, without the client having to ask for it. Such a client will not have to check with the server periodically; rather it can continue with other work and work on the data generated by the event when it has been pushed by the server. This is exactly what Comet sets out to achieve.
The Netscape server push browser extension is one solution, but it is not compatible with Internet Explorer, among others.
Technology
Unlike normal data transfer between web servers and browsers, connections to a server employing Comet require special constructs. The client-side application must keep working while asynchronously initiating a connection to the server. At the server, even though the request is not serviced at once, the connection remains unbroken (and unused) until the event occurs. When the event occurs, the server pushes the data generated by the event to the client over the already-established connection.
The connection can be severed after that, or may be kept open if there can be further occurrences of the event. If there are more occurrences, the server can send the event data over the connection, without the client having to request each of them explicitly.
Scalability and Reliability
![]() | This article needs attention from an expert in Science. Please add a reason or a talk parameter to this template to explain the issue with the article. |
There are some potential concerns regarding how well a web server implementing Comet can scale. Because connections are kept alive until events have occurred, it has to cope with many connections if the event occurs infrequently. The problem worsens if different connections wait for different events. Managing this large number of connections poses a considerable load on the system. But servers and other supporting applications are being developed which have better support for such conditions.
The Ajax/Comet Request Router is one mechanism for providing scalable server-side infrastructure supporting low-latency asynchronous and bidirectional Comet applications. It is used as a front-end for arbitrary web- or application-servers, to distribute client requests to a matching server, which may be a specialized program, or a general-purpose server which executes application-specific code for corresponding requests. Mort Bay Jetty and Apache 2.2 provide intrinsic support for situations where they use asynchronous network operations to reduce the overhead of managing the connections. Thus the web server logic has to deal only with sending messages back to the user once the event has occurred. Since almost all Operating Systems have an event-driven I/O subsystem, this is not a problem.
Firewalls and HTTP proxies between the browser and web server can pose further network problems. Firewalls are typically configured to drop connections that have been open for too long (as a security heuristic), and so commercial-quality frameworks tear down and recreate the "push" connection periodically. Naive intervening HTTP proxies may buffer pushed messages in chunks up to 32 or 64kb, necessitating a system for adaptive detection of intervening proxy buffer sizes.
Browsers pose a further limitation: section 8.1.4 of the HTTP 1.1 spec states that "A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy", and this recommendation is in fact followed by the most common browsers, including Internet Explorer and Firefox. Holding one connection open for HTTP streaming tends to create performance issues for AJAX applications, such as inability to kick off a new user-initiated data request while a series of images are being loaded. This is often worked around by creating a distinct hostname for push connections (even with a single physical server).[citation needed]
In clustered server environments, messages need to be routed to whichever webserver a given client is currently connected to, and guaranteed delivery may need to be ensured in the event of a webserver crash. This is typically handled by Message Oriented Middleware systems, frequently encapsulated by JMS or Spaces based Systems, frequently encapsulated by JavaSpaces.
Implementations
There are several implementations of the Comet Ajax server-push model, both commercial and non-commercial:
- Orbited is an open-source cross-language Comet server.
- Pushup is an open-source Comet server written in C++
- Virgil's One™ is a Comet implementation for streaming marketdata solutions for retail-, wholesale-, private banks and brokers
- The SmartClient Messaging module (see AJAX IRC Chat example)
- Lightstreamer. Adaptive HTTP streaming with some AJAX APIs.
- Steve and Jay McDonald’s *Fjax
- Pjax (Push technology for Ajax)
- Server-Sent Events standards proposal specified by WHATWG
- COMETd Scalable Open Source Comet Framework
- Ajax for IBM WebSphere Platform Early program Comet feature for IBM WebSphere-based Products (6.1+)
- Pushlets An open source framework for server-side java objects to push events to browser-side JavaScript, Java applet, or flash application.
- Lingr Browser-based chat community using comet for realtime message notification
- Sun Microsystems GlassFish Comet for Java EE based applications
- AjaxMessaging for Ruby on Rails Comet plugin for Ruby on Rails. A plugin to provide real time messaging (aka COMET) for Ruby on Rails using Ajax and Messaging Oriented Middleware.
- ShootingStar the comet server which is implemented by ruby and C language with use of epoll/kqueue.
See also
References
- Caplin Systems - one of the first streaming web platforms, and still probably the best-performing, now comet-compliant. Widely used by financial institutions for online trading.
- KnowNow - pioneering implementation of Comet. Link goes to a description of LiveBrowser, a feature of the KnowNow LiveServer.
- Divmod Athena: Comet implementation in Python
- DojoToolkit - Recent re-implementation
- Changing the Web Paradigm: Streaming AJAX and Comet (pdf)
- Push Technology for RIA: Two ways of introducing iPush Server to a RIA project
- Enterprise Comet: Awaken the Grizzly - an article about Comet and the usage of the Bayeux protocol
External links
- How to implement COMET with PHP
- Ajax for IBM WebSphere Platform Early program Comet feature for IBM WebSphere-based Products
- Apache Tomcat 6.0 - Advanced IO and Tomcat: Comet support allows a servlet to process IO asynchronously, receiving events when data is available for reading on the connection (rather than always using a blocking read), and writing data back on connections asynchronously (most likely responding to some event raised from some other source).
- Asteroid (a small Comet): Comet in Squeak(Smalltalk), by Diego Gomez Deck
- Caplin Systems A highly tuned and ultra-scalable bi-directional Web messaging platform that uses comet with HTTP/S tunnelling to maintain up to 30,000 concurrent connections per server for low-latency streaming - used by major financial institutions across the globe.
- COMETd: an open source scalable comet framework. A Dojo foundation sponsored project.
- Flex Enterprise Services An approach from Adobe/Macromedia for addressing the same problem using Flash.
- Full-duplex Ajax A module for Lighttpd web server
- GlassFish Comet Support: Simple example on how to write an AJAX/Comet based application using the Grizzly's Asynchronous Request Processing framework.
- HTTP-SELECT An IETF Internet Draft from 1999, which describes the HTTP asynchronous updates pattern
- ICE Technology Corp.'siPush Server - real-time push data to your RIA.
- LazyHttp, the backbone to Web 3.0
- Lightstreamer A full Comet solution that comprises a Push Server and a set of client libraries. The event-based architecture of the Server makes it able in principle to scale to tens of thousands of concurrent connections.
- Virgil's Live™ streams any data from any source to any web client and delivers real time data at low latency, high availability, through firewalls and proxy servers based on COMET.
- http://sourceforge.net/projects/comet/ a complete PHP-Comet-Server implementation, by Aresch Yavari
- ShootingStar the comet server which is implemented by ruby and C language with use of epoll/kqueue.