The Two HTTP Connection Limit Issue

From RuntimeWiki

Jump to: navigation, search

Contents

Title

"The Two HTTP Connection Limit" issue

Detailed write-up

Description

Browsers typically limit the number of concurrent HTTP connections per server to be 2(the term “per server" is used broadly here for the sake of simplicity. For example, it could be multiple physical machines sharing the same public IP.). If a browser session has two HTTP connections already, any further connection requests will have to wait until one of these two HTTP connections is finished. As a result, the browser (or the application) appears to be locked up. Though the "two connection limit" made sense years ago, it is very problematic in today's Ajax enabled "web 2.0" applications.

Why Is This Important?

  • Comet (server push): HTTP connections are request/response based and are only initiated by the client. However, many applications require Comet (server push) style communications. These applications include web-based chatting, financial trading, real-time web alerting, system monitoring, etc. In general, Comet implementations use some form of "long lived connection" for delivering server push messages. This "long lived HTTP connection" is a normal HTTP connection but the server keeps it open. Whenever a message occurs, the server uses this connection to deliver the message to the client. In doing so, the server can initiate communications to the client without having to passively wait for the client to initiate a connection first. The problem is that this long lived connection consumes one of the two connections limit. If the user opens two browser windows of the same Comet application, both connections are consumed. When both connections are consumed, even normal HTTP requests (such as opening a URL, loading an image, loading CSS, etc) are blocked. It appears to the user that the application is locked up.
  • Multi tab browsing: most browsers support multi-tab browsing. A user can open multiple browser tabs, with each tab running the same or different pages from the same server. Tab browsing itself is not an issue, but the combination of tab browsing and the two connection limit makes the problem much more likely to happen. For example:
    • If an application has a web page that issues a background Ajax request to the server and this request may take a while to finish (hence the developer wrote this as a background Ajax request), and the user happens to open this page on two different browser tabs, the "two connection" limit is reached and the application appears to be locked up;
    • If the user runs a Comet application in two tabs, the connection limit is reached and the application is locked up.
  • Portal: for a portal author, if he assembles two portlets that each consumes an HTTP connection onto the same Portal page, the entire portal is locked up.
  • Mashup: for a mashup page author, if he assembles two mashup widgets that both consume an HTTP connection onto the same page, the page will be locked up too.

Possible Solutions

  • To provide a way for a page author to indicate that a connection is going to be long lived (for example, via HTTP header);
  • To provide a way for a page author to request more concurrent connections than the default limit of 2;
  • To limit the number of concurrent HTTP connections on a per browser tab basis, instead of on a per server basis. For example, each tab has a two connection limit instead of the current two connection limit per server.

Background material that request this feature

Discussion

In this section, the contributors should express their opinions about this feature request, such as providing particular technical analysis or describing in prose why this feature is important (or not). It is recommended that each contributor create his own level-3 sub-section (e.g., === Jon Ferraiolo Comments ===).

It is OK for others to insert comments within other people's sections, but as a courtesy please make sure that it is clear which part of the section is original and which section are annotative comments. For example, perhaps annotative comments would be colorized (e.g., enclosed by <span style="color:red">JON: Here is what I think about what you are saying.</span>).

Pete Laurina's comments

I think this issue needs to be solved in collaboration with web server development. Simply increasing the limit could lead to issues with DoS attacks. Maintaining the 2 connection limit will maintain backwards compatibility and general security for older servers. Development collaboration between browser and server teams would allow for the server to send Neuberg's Meta information embedded with every response header from the server. This would allow System Administrators to have some control over system performance. The browser could then use this information to modify its connection limit on a per server basis and perhaps give a signal to the end user if multiple tabs start connections with the same server. Helping to prevent end users from seeing an unresponsive window (or at least they would know why and that it will recover shortly). If no information is returned by the server in the response header the browsers can default to the existing implementation of 2; maintaining backwards compatibility and existing security controls. This approach will also allow for scalability into the future for faster transfer rates resulting in inevitably more requests and hopefully multi-threading since a SysAdmin can adjust on demand based on their application requirements.

Jon Ferraiolo comments

Note that with OpenAjax Hub 1.1, we have talked about having an XHR manager component which would allow sustained communications (e.g., Comet-style) up and down the pipe between the client and a particular domain to be managed by the OpenAjax Hub. Therefore, it would be good if future browsers were able to low-level facilities that the Hub's JavaScript could use. For example, perhaps some form of long connection feature were available within a particular browser. It would be best if that new feature could be developed such that a future version of the Hub could use that feature when present, else fall back to the current implementation.

Douglas Crockford comments

This appears to be fixed in IE8. We should urge the other browsers to follow.

Kris Zyp's comments

I wouldn't regard IE8's new connection limit as a fix. IE8 bumped the connection limit to 6. Dealing with a scalability problem by increasing a limit should hardly be esteemed as being "fixed". The 2 connection limit is very useful limit because it helps to limit the number of connections that are used on a server, which is one of the most expensive resources on a server. We really need a way to intelligently increase the limit as needed. See XHR Connection Length Advice for proposal.

Brad Neuberg's comments

It's very important to fix this, as it will give a noticeable improvement in performance for broadband users and also open up using comet-style techniques for more websites. The short-term fix is to simply bump up the value to 6 or 8. However, what happens when several years down the line servers can handle more connections, and users have even faster connections that can handle more? Simply baking the value "2" or "6" seems as shortsighted as saying computers will only need 640K of RAM. We need a better long-term way for a server to say how many concurrent connections it can sustain, and for clients to say how many on-going connections work for them. For the server side, a META tag is probably easy enough:

<META name='server.http.connection' content='6'>

We can default this to 6 or 8 for now if not specified.

For a client, this is probably better at the OS or browser level. For example, if you are on a dial-up connection, then the OS for that network connection would say that 2 connections is the best, while for broadband 6 is better. For that future unknown time when better broadband appears, the local OS could say 12 is great, and if a remote server is running on newer hardware with great broadband it could serve up more connections using the META tag above.

This could also be specified at the HTTP level, which on second thought is probably better. What do you think?

Greg Wilkins comments

I do not believe that increasing the number of connections is a solution. I can imagine a portlet style page with 10, 20 or 30 widgets each in it's own frame and the though of each of them having a long held request on the server is a scalability nightmare. 6 connections will easily be exhausted and fails the 0,1 or infinity principle.


The key to solving this issue is that if windows/frames/tabs are going to share a resource pool, then there must be some common javascript space that they can also share so that they may communicate and collaborate to actually share those resources. Currently the frames must share 2 connections, but are isolated from each other so this sharing cannot be coordinated.

2 connections is sufficient for arbitrary communication, so long as they can be shared.


marking a request as long held is useful as a hint to the container to not pipeline other requests behind it, but it does not help share the connection. 2 long held requests will still exhaust the connection pool.

Nicole Tedesco's comments: implement a generalized resource archiving solution

(From NicoleTedesco)  A generalized resource bundling solution (images, sound files, XML and even other HTML pages) can help to minimize HTTP connection count and bandwidth consumption. For details, see my comments in the Ajax Toolkit Caching discussion.

Geoff Granum's comments

I would be thankful for the addtion simple queuing of requests. Firefox 2 and IE6/7 will all simply throw an exception.

I think this would 'solve' Greg Wilkon's issues, at least those in paragraph one, in a 'fail gracefully' kind of way.

Worring about D/DOS attacks is not really the job of the browser. If the server is not doing its own DOS checking and thread handling, that is the server's problem. If the page author ( presumably the author with knowledge of the server ) creates a situation which bogs the server down, that is the author's fault.

On that note:

- The ability for the page author to specify a desired maximum concurrent request limit for the server from which the page originated.
- A queue.
- A graceful 'server responded: too many concurrent requests from this IP address / user session' fail to queue, followed by fail to exception after n tries.

The reason for the IP address fail being that a session key can be forged easily. It might not be VALID, but it can be forged.

The reason I say that worrying about D/DOS attacks is not the job of the browser is that any half-brained code-monkey can write their own 'browser' that ignores connection limits.

That said, I agree with Brad Neuberg; if a server TELLS the browser that it is willing to accept 20 connections, then by all means, use them. I would argue that this should be a server-scoped variable which can be set on any response from the server, overriding the previous value: For instance. UserA makes a request for her home page. Meta tag declares that the server will accept 20 concurrent requests; load is light. UserA's home page is very busy; it has a hundred widgets all trying to use a request each. UserB tries to log in. The server, crying, declares that UserB can only have 4 concurrent requests. UserB has 20 widgets on his home page. UserA, in her attempt to get her 21st widget receives a new meta tag: the server wants to be nice to everyone, and only has so many connections available: only 10 connections for UserA now. UserA's queue will empty until it hits 9 items, then the next request will be made. Meanwhile, UserB's 5th request gets the meta info inviting him to increase his connection count to 10.

rektide's comments

While concurrent connection negotiation may offer some bypass and relief, the root issue is that multiple requests need an effective manner of sharing. This becomes particularly important whenever a user opens multiple tabs (unavoidable) or when a site has long lived connections (in some cases avoidable). If you wish to bypass this limit and implement server side control over connection limits theres nothing stopping you today: just use subdomains and monitor client connections. The heart of this issue is how we can make use a limited number of of the HTT-Protocol connections that are in heavy demand.

Data sharing is an answer and its mechanisms are already in place with postMessage (available in all next gen browsers pending IE8 release), however it requires developing manual collaboration systems for intelligently sharing connections, and requires supplanting existing Ajax functionality with calls to a separate master control program for connections.

I would much rather see HTTP pipelining extended to permit out of order response (turning http into a genuine multiplexing protocol ftw) or multiple clients able to listen to the same connection (allowing a dirty version of the same) or... well, I'm not sure. I like the existing hacks comet is built upon- how they've used tech as it works- and I think reorchestrating them because a user opens multiple tabs is a rotten idea, but short of simply multiplexing HTTP I'm short of good solutions.

With more specific regard to the connection limit issue, the one thing I havent heard yet here is the failure case: when a page does exceed its connection limit there MUST be well defined failure mechanisms that the client javascript MUST be able to detect. This should all be spec; the present paths of doing things like locking the browser are mostly pants answers. A way to client-side abort existing connections would also be good. Connections are currently a black box that occasionally goes Thermal-Detonator on us: developers need specs to view manage the connections the user is keeping with their site-- particularly given that it might not even be "this instance" of the site that opened the connection in the first place. Visibility would circumvent much of the cruder initialization hackery required to bootstrap the MCP in data sharing.

Phase I Voting - Vote for Your Top 5 Features

NOTE: PHASE I VOTING IS NOW OPEN. (2008-04-01) We have now changed the voting procedure. Instead of putting votes on each separate wiki page, we are asking people to cast their Phase I Votes on the following wiki page:


Phase II Voting

More about this later.

Personal tools