Server to Client notification using ASP.NET with Comet Approach

Update:
This project is now availble on GitHub here.
Check out the Wiki on how to get started here.
For a video introduction, please visit YouTube video here.

Thanks:-)

The general challenge with server to client notification in web clients is that all communication to the server is initiated by the client through http posts. In other words it is in general not possible for a server to establish a connection to a (web) client.
Different solutions can be implemented in order to address this fundamental problem:

1. Traditional ASP.NET synchronous solution

Whenever the client does an http post, the server can check if the client user has pending notifications. If this is the case, the server can add this notification to the client response.

Pros: Good scalability since no extra client-server communication is needed.
Cons: No real-time notification to clients since the user has to perform some action (actually request new aspx page) to receive the notification message. This results in an unpredictable notification time scheme. This solution has tight coupling to the existing communication model.

2. AJAX approach – rapid synchronization

The client makes synchronous callbacks to the server in some predefined interval, e.g. once every 5 seconds via JacaScript. If the server has any notifications pending to the client user, it writes this to the response stream.

Pros: Almost real time notifications (once every n seconds).
Cons: Will most likely not scale very well.

3. Comet approach – realtime server to client notification

The client makes a request to the server (through JavaScript), but does not expect any rapid answer. Instead the server saves the request and waits for any notification relevant to the specific client user to occur. When a notification occurs server side, the server responses to the client handle obtained earlier, and the client immediately processes the response stream via JavaScript and notifies the user. The client then sends a new handle to the server to wait for the next notification.

Pros: Real time notifications.
Cons: It needs to be investigated whether this can scale if there is many concurrent users. This approach is the most expensive of the three and will be difficult to test due to the many asynchronous operations.
Due to obvious scalability issues option 2 is not considered a valid path to follow (for example – 5.000 concurrent users which request notifications every 5 seconds is 1.000 requests every second – compared to the benefit of having server to client notifications this path is considered illegitimate).

This short article will explore option (3) to address the amount of uncertainty associated with this path.

The following paragraphs will focus on:

(1) a suggestion for over-all solution design for “Comet approach”
(2) a concrete description of the communication between server and client
(3) a concrete description of how the client makes the request and handles the response
(4) a concrete description of how the server handles the request

Finally I’ve made some research on scalability.

1. Overall solution design (initial suggestion)

This is a short overview of the solution design. The solution has been implemented sparsely but with some limits. Throw me a note if you want to take a lok at the code.

comet1

1 to 3: Client submits user handle

1. The client sends a user handle to notificationchecker.ashx.

2. NotificationChecker.ashx sends the handle to the messagebroker.

3. The message broker registers the user handle.

A to C: A business module N submits a notification message to user U

A. Some module submits a notification message to the message broker.

B. The message broker queues the message.

C. The message broker processes the message queue.

D to E: The message broker processes the message queue

D. The message broker checks if the user has submitted a handle. If she has, the user handle is released.

E. The message broker releases the response to the client

F to H: The client acknowledges the notification response

F. Client sends notification acknowledgement to NotificationAcknowledger.ashx.

G. NotificationAcknowledger sends acknowledgement to MessageBroker which finally removes message from repository.

H. The Message Broker sends a 200 (OK) response back to the client

Notes on transitions

In the figure above the User Handle part is marked with green, the Message parts are marked with red, and the Message Broker is marked with blue color. This illustrates that the different operations are taking place asynchronously. This chapter contains a few notes on the transition between states.

Before any messages can be sent to any client, at least one client needs to register a user handle to the message broker (steps 1 to 3).

Also, it is a prerequisite before any messaging can occur, that a message is submitted to the message broker by some business module. The message is addressed to a specific user and will receive a unique id. This is done from step A to C in the figure above.

Now the message processing begins. When a message is submitted to the message broker, the broker queues the message in the message queue. Whenever the message queue contains more than zero messages it is processed by the message broker. What the message broker does is that it dequeues the queue to retrieve a message addressed to the user U. The message broker then checks to see if the user U has submitted a handle. If not, the message is stored in the virtual repository. If the user U has submitted a handle, the user handle is released and the message is sent to the client.

When a client registers a user handle, the message broker checks to see if any message addressed to the client user exists in the virtual repository. If so, all messages to user U are queued in the message queue, and the user will receive these when the message processor dequeues them.

When a message is send to the client, the message broker will store the message temporarily in the repository. Only if the client acknowledges the messages the message broker will remove the message completely. The client must synchronously acknowledge the latest message before it sends the next user handle.  When the next user handle is send all messages in the repository are queued (as just described above), and any pending acknowledgments will be ignored (since the messages are no longer in the repository, but in the queue).

2. Communication between server and client

  • The Client sends a request to the server
    1. The endpoint is “notificationchecker.ashx”
    2. The web site takes a parameter, “userName”, which tells the server message broker which user to look for notifications for


POST /TestComet/notificationchecker.ashx?userName=mhm HTTP/1.1
Accept: */*
Accept-Language: da
Referer: http://192.168.56.159/TestComet/default.aspx
UA-CPU: x86
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB6; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; InfoPath.2)
Host: 192.168.56.159
Content-Length: 0
Proxy-Connection: Keep-Alive
Pragma: no-cache

  • If or when the server has a notification for the user in question its response contains the notification
    1. In the example below the notification is the word “test”.
    2. If necessary the notification could be communicated using JSON, XML or another standard


HTTP/1.1 200 OK
Date: Mon, 22 Jun 2009 11:05:16 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 4

test

Please note that the communication between server and client is very light weight and the network traffic therefore shouldn’t be a relevant factor.

3. Client-side handling of requests to and response from server

First, the client makes the request to the server. This is done using a small JavaScript function:

function postRequest(url) {

var xmlhttp = new ActiveXObject(“Microsoft.XMLHTTP”);

// ‘true’ specifies that it’s a async call
xmlhttp.Open(“POST”, url, true);

// Register a callback for the call
xmlhttp.onreadystatechange =
function() {
if (xmlhttp.readyState == 4) {
var response = xmlhttp.responseText;
notificationPopup.innerHTML = “<p>” + response + “</p>”;
showNotification();
setTimeout(function() { getNotifications(); }, 3000);
}
}

// Send the actual request
xmlhttp.Send();
}

Two short notes on the JavaScript:

  • The client (i.e. the browser) makes an asynchronous request to the specified url (please note that the parameter ‘true’ specifies the asynchronous communication).
  • When the readyState of the XMLHTTP object changes to 4 (completed), a function is asynchronously updating and displaying the notification to the user.
  • A timeout is set to 3 seconds before the server is asked for the next notification in order to let the user see and respond to the first notification. This could be changed depending on the required business logic (i.e. – how is the user (not the client but the physical user) going to acknowledge the message).

There is a risk for data loss or delayed messaging since the asynchronous handling of the response requires the readyState to turn 4. However, a persistent connection is established with the server which means that unless the connection is terminated by the script or the server, readyState will always be 3 (interactive) and will never reach the step (4). Also the XMLHTTP could have a buffer to fill before it makes the response available. These issues should be investigated further since they are a possible danger.

Moving on to the client-server communication, from fiddler it can be seen that the request is actually send from the client to the server:

Comet2

Also, please notice the green arrow in the left list. This indicates that no response from the server has yet been received. From a small test client I can simulate a server response:

Comet3

I’ve typed in “new server notification”, and I click the button. Now fiddler detects the new response and updates the response window with:

Comet4

And to the end user the notification would look like this:

Comet5

The image below makes it evident that the notification appears to the user right after it has been send by the business module:

Comet6

The top browser simulates a business module which sends a notification to the message broker which creates a new notification when the “Send notification” button is clicked. The bottom browser simulates a client – it receives the message send to the message broker by the top browser and immediately displays it to the user.

The showNotification() function contains:

function showNotification() {

document.getElementById(‘notificationPopup’).style.visibility = ‘visible’;
setTimeout(function() {
document.getElementById(‘notificationPopup’).style.visibility = ‘hidden’;
}, 5000);
}

The notification popup is styles with css and could be designed accordingly or integrated to the gui in some otherway.

For the notification possibility to work every page must contain the necessary message handling JavaScript.

4. Server side asynchronous handling of the request

Focusing on the asynchronous handling of the messages the server side contains:

  • notificationChecker.ashx which is maintained as the endpoint for the communication
  • AsyncTaskHandler which is a class for handling requests to notificationChecker.ashx. AsyncTaskHandler inherits IHttpAsyncHandler
  • AsyncRequestResult which inherits from IAsyncResult and is the “server ticket” for handling the communication (just as any IAsyncResult would be in normal asynchronous communication)
  • AsyncRequest which is a ‘normal’ class (without explicit inheritance) which encapsulates the asynchronous requests – this class has a method “Process” which handles the actual AsyncRequestResult (IAsyncRequest).
  • MessageBroker which processes the IAsyncResult’s by calling AsyncRequest.Process()
  • MessageRepository which is controlled by the MessageBroker and contains all messages in a queue as well as the user handles (a list of IAsyncRequest’s).

The AsyncTaskHandler is the class actually processing requests to notificationChecker.ashx. The processing looks like this:

public IAsyncResult BeginProcessRequest(HttpContext context, System.AsyncCallback cb, object extraData)
{
// Get the current user name
object data = context.Request["userName"];

// Create a result object to be used by ASP.NET
AsyncRequestResult result = new AsyncRequestResult(context, cb, data);

//TODO: a user can only have one client open!!!
MessageBroker.Value.Repository.AddUserHandle(userName, result);

// Return the AsynResult to ASP.NET
return result;
}

The user name is first obtained from the context. Hereafter the IAsyncResult object “AsyncRequestResult” is created .

Afterwards the user handle “AsyncRequestResult” is added to the repository via the MessageBroker. In the current implementation the key is the userName – this should be changed since it means that only one handle can be created per user, but it’s likely that the same user has more than one client open (which is currently not allowed).

Finally the IAsyncResult is returned to ASP.NET.

What the AddUserHandle in the repository does, is:

  • Move all pending messages in the repository for the user U to the message queue
  • Tell the MessageBroker to start processing messages if it is not already processing

Now, the MessageBroker.Process() method has this business logic:

public void Process()
{
Message message = null;
Repository.Pause = false;
while (Repository.Messages.Count > 0)
{
message = Repository.Messages.Dequeue();
if (Repository.UserHandles.ContainsKey(message.User))
{
AsyncRequestResult arr = Repository.UserHandles[message.User];

// Create a new request object to perform the main logic
AsyncRequest request = new AsyncRequest(arr);
request.MessageText = message.Text;

// Create a new worker thread to perform the work
// Notice that this will not remove a thread from the CLR ThreadPool.
ThreadStart start = new ThreadStart(request.Process);
Thread workerThread = new Thread(start);

// Start the work
workerThread.Start();

// Remove the user from the active user list
// The user will be added again when she sends a new handle
Repository.UserHandles.Remove(message.User);
// TODO: This should be rewritten – the message should not be removed from the queue before the
// user acknowledges the server response, also the user handle should not be finally
// removed before this
}
else
{
Repository.PendingMessages.Add(message.Id, message);
//TODO: timer should queue this list regularly – just to be sure not to loos messages ;-D
}
}
Repository.Pause = true;
}

This method first signals to the Repository that it has started processing (Repository.Pause = false). Afterwards it processes the whole message queue. It begins by dequeueing the message queue and thereby obtaining a message. It then checks to see if the Repository contains a handle for the user in question. If it doesn’t the message is added to the PendingMessages list. If the user has in fact submitted a user handle the AsyncRequestResult (which implements IAsyncResult) is obtained from the UserHandles in the repository. Now an AsyncRequest is created (an object which contains the .Process() method which actually sends the message to the user). A new worker thread is created with the responsibility of processing the AsyncRequest (which again contains the AsyncRequestResult). Finally the UserHandle is removed from the Repository. When the processing of the whole queue finishes the Repository is informed that the Processing has paused.

Now, in the AsyncRequestResult.Process() method it will be possible to specify the business logic for sending response to the client. For instance:

/// <summary>
/// Uses the IAsyncResult object to write
/// back the response to the caller
/// </summary>
public void Process()
{
try
{
// Write back to the client
Result.Context.Response.Write(this.MessageText);

//TODO: The result needs to be rewritten.
//at least the guid should be included, and probably some other data
//(for example a link which should automatically be followed)
}
finally
{
// Tell ASP.NET that the request is complete
Result.Complete();
}
}

The method addresses the public “Result” method which is the object instance from the AsyncRequestResult which implements IAsyncResult. Currently the method simply returns the raw message text – this part should be rewritten, cf. the comments in the code above.

When a notification has been obtained the method writes back to the client and ASP.NET is informed that the request has completed.

The solution design speaks of an acknowledge functionality also. This functionality has not been implemented yet and will thus not be presented here.

5. Scalability

[The scalability test was performed before the MessageBroker was implemented. Further investigations are needed]

It can be expected that all online users will communicate with the notification service which will be the endpoint of much communication. Thus some initial investigations on scalability have been made.

The test contains these steps:

  1. N requests have been sent from a client to a server. All requests where send from the same client. The server was placed on a VM on an external hard drive with 1 GB memory allocated.
  2. The requests have been send synchronously with no break (i.e. all requests have been send within few seconds)
  3. All results where send back when they where processed (it was determined by the server whether it wanted to prioritize handling the incoming requests first or processing the incommed requests first; it seems that the server default keeps prioritizing incoming requests before the cpu starts working on processing tasks).
  4. The results are received asynchronously by the client and the message passed from the server is written to a console window.

To measure server performance perfmon was used.

Test 1. 50 requests send to the server:

Comet7

Test 2. 200 requests send to the server:

Comet8

Tets 3. 500 requests send to the server:

Comet9

Test 4. 1000 requests send to the server:

Comet10

Test 5. 5000 requests send to the server:

Comet11

The initial conclusion is that this solution actually can scale. At least server performance doesn’t seem to suffer. However, since all requests came from the same client, Fiddler did actually crashJ

The tests above are ensuring that the server can actually process the requests. However, it does not ensure that the server can actually hold this many connections without problems. Therefore a second test has been performed where the server holds the thread for 5 seconds (Thread.Sleep(5000)) before it begins returning the responses one by one. This forces the server to hold all requests in memory.

Comet12

From the second test it can be concluded that what puts pressure on the server is actually the processing of the requests, not the ‘holding’ of the requests before the response. This is concluded since the second test reveals lower processor time than the first. This is positive for the scalability of the solution.

It should still be investigated further what this means for the server side handling on tcp-ip level so more thorough testing is definitely needed. For instance, the maximum number of open user ports is approx. 65.000 – what would happen then if 5.000 users where to submit 500 handles within a whole working day. One point being that this solution is vulnerable to the number of open clients (which is not usually an issue for web client technologies).

It is crucial to notice that this test only investigates the communication setup (multiple clients sending handles to the server and waiting for some requests). In these tests the request is simply a never changing string. If the solution is to be scalable it will be critical to make the server processing of the notifications scalable. This is further underpinned by the fact that it is the processing on the server that is the real bottleneck in this solution. In other words – a test should be schedules which includes the MessageBroker part of the solution.

Catch-up – what made this scale

Before the tests could look good I needed to adjust my test setup with several improvements:

1. Make the maximum limit of connections higher

Maximum limit (my pc)

When the number of active connections begin nearing 5.000 these errors are seen:

  • An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full
  • No connection could be made because the target machine actively refused it

Solutions:

OS runs out of available TCP “ephemeral” ports 

When the client machine is opening many TCP connections and is running Windows Server 2003, Windows XP, or any earlier version of Windows, it may run out of TCP “ephemeral” ports. In Windows Server 2003, Windows XP, and earlier versions, Windows limits the number of available ephemeral ports to approximately 5000 across the machine. It is especially common to hit this problem for applications which do not use connection pooling.

Solution: To make more ephemeral ports available, follow the directions in this KB which describe how to create the MaxUserPort  registry key:http://support.microsoft.com/kb/196271

Step-by-step following the link above:

  1. Start Registry Editor.
  2. Locate the following subkey in the registry, and then click Parameters:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

  1. On the Edit menu, click New, and then add the following registry entry:

Value Name: MaxUserPort
Value Type: DWORD
Value data: 65534
Valid Range: 5000-65534 (decimal)
Default: 0x1388 (5000 decimal)
Description: This parameter controls the maximum port number that is used when a program requests any available user port from the system. Typically , ephemeral (short-lived) ports are allocated between the values of 1024 and 5000 inclusive.

  1. Exit Registry Editor, and then restart the computer.

2. Set the ReadWriteTimeout to 10000 (or higher)

There is a risk that the server will not respond within 1 second when trying to read or write a stream. If the ReatWriteTimeout limit is made higher this will remove this issue.

3. Set the Timeout to an appropriate value

It is suggested to set the client-side timeout to 5 minutes. Also it should be ensured that when a timeout occurs, the client simply reconnects to the server. The correct timeout value should be investigated.

6. Points of interest

Comet is a technology which makes real-time server to client communication possible even for a web client.

An approach has been suggested including overall solution design and some preimplementation. This implementation is not mature and is (besides from numerous bugs) depending on

(1) acknowledge functionality to be implemented, (2) the possibility of one using having multiple clients, (3) real db implementation (currently xml file is used), (4) logging, (5) configurability, (6) specification of GUI, (7) specification of user business logic (how and when should notifications be shown etc.), (8) stability of JavaScript message processing, cf. section 3.

The scalability has been tested with the MessageBroker detached (since it was not prototyped when the scalability test was performed). The initial investigations indicate that this solution scales well. However, it should be tested what high user load means on tcp-ip level, and the MessageBroker should be included in a new test ensuring that this can scale as well. Also it should be investigated what actually is an appropriate timeout value (can the client keep its handle the whole day for instance?).

Download source code from: https://retkomma.wordpress.com/2009/09/05/source-code-for-comet-project/

8 responses to “Server to Client notification using ASP.NET with Comet Approach

  1. Very interesting approach, can you post the whole sources in order to test it?
    Thank you

  2. Sebastien Thuilliez

    Thanks for this nice article. I would also be interested in the source code in order to investigate this approach in details.
    🙂

  3. Pingback: Source code for comet project « Linguistic forms

  4. We’ve implemented a full comet solution for IIS/ASP.NET called WebSync (www.frozenmountain.com/websync). We’ve tested over 30,000 concurrent users and over 25,000 message per second using nothing but a $400 acer desktop with 3 gig of ram. It’s a great solution (ok, I’m biased, I admit), and we’re thrilled about how well it works. Check it out sometime, and let us know what you think.

  5. Hi,

    As I see you have used setTimeout to 3000 which means it will request to the server after every 3 seconds. So, bit unsatisfied with what you have written and what you have implemented.

    For the first time client has requested to server and server does not have any updates then server should sleep the client request and as soon as the there is an update send notification to server.

    Please clear.

    Thanks,
    Sunder

  6. Hi Sunder,

    I’m not going to make any changes in the code. It is a 1½ year old PoC/Test written in half a day or so – you are welcome to use it or not🙂 If you would like to extend the code or make bugfixes to it, please feel free to throw it up on GitHub.

    If you are building a real enterprise comet module, please consider more contemporary approaches like BOSH and XMPP.

    Br. Morten

  7. The world over. And carefully protected under international copyright law. france tend to get it right. The imitators do not.The material should feel soft and relatively thin to the touch. It.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s