38

I'm looking for a way to create a simple HTML table that can be updated in real-time upon a database change event; specifically a new record added.

In other words, think of it like an executive dashboard. If a sale is made and a new line is added in a database (MySQL in my case) then the web page should "refresh" the table with the new line.

I have seen some information on the new using EVENT GATEWAY but all of the examples use Coldfusion as the "pusher" and not the "consumer". I would like to have Coldfusion both update / push an event to the gateway and also consume the response.

If this can be done using a combination of AJAX and CF please let me know!

I'm really just looking to understand where to get started with real-time updating.

Thank you in advance!!

EDIT / Explanation of selected answer:

I ended up going with @bpeterson76's answer because at the moment it was easiest to implement on a small scale. I really like his Datatables suggestion, and that's what I am using to update in close to real time.

As my site gets larger though (hopefully), I'm not sure if this will be a scalable solution as every user will be hitting a "listener" page and then subsequently querying my DB. My query is relatively simple, but I'm still worried about performance in the future.

In my opinion though, as HTML5 starts to become the web standard, the Web Sockets method suggested by @iKnowKungFoo is most likely the best approach. Comet with long polling is also a great idea, but it's a little cumbersome to implement / also seems to have some scaling issues.

So, let's hope web users start to adopt more modern browsers that support HTML5, because Web Sockets is a relatively easy and scalable way to get close to real time.

If you feel that I made the wrong decision please leave a comment.

Finally, here is some source code for it all:

Javascript:

note, this is a very simple implementation. It's only looking to see if the number of records in the current datatable has changed and if so update the table and throw an alert. The production code is much longer and more involved. This is just showing a simple way of getting a close to real-time update.

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.js"></script>
<script type="text/javascript" charset="utf-8">

var originalNumberOfRecsInDatatable = 0;
var oTable;

var setChecker = setInterval(checkIfNewRecordHasBeenAdded,5000); //5 second intervals

function checkIfNewRecordHasBeenAdded() {

        //json object to post to CFM page
        var postData = {
        numberOfRecords:  originalNumberOfRecsInDatatable 
        };

        var ajaxResponse = $.ajax({
        type: "post",
        url: "./tabs/checkIfNewItemIsAvailable.cfm",
        contentType: "application/json",
        data: JSON.stringify( postData )
        })

        // When the response comes back, if update is available
        //then re-draw the datatable and throw an alert to the user
        ajaxResponse.then(
        function( apiResponse ){

         var obj = jQuery.parseJSON(apiResponse);

         if (obj.isUpdateAvail == "Yes")
         {              
            oTable = $('#MY_DATATABLE_ID').dataTable();
            oTable.fnDraw(false);

            originalNumberOfRecsInDatatable = obj.recordcount;

            alert('A new line has been added!');
         }

        }
        );

    }
</script>

Coldfusion:

<cfset requestBody = toString( getHttpRequestData().content ) />

<!--- Double-check to make sure it's a JSON value. --->
<cfif isJSON( requestBody )>

<cfset deserializedResult = deserializeJSON( requestBody )>

<cfset numberOFRecords = #deserializedResult.originalNumberOfRecsInDatatable#>


<cfquery  name="qCount" datasource="#Application.DBdsn#" username="#Application.DBusername#" password="#Application.DBpw#">
    SELECT COUNT(ID) as total
    FROM myTable
</cfquery>

<cfif #qCount.total# neq #variables.originalNumberOfRecsInDatatable#>
    {"isUpdateAvail": "Yes", "recordcount": <cfoutput>#qCount.total#</cfoutput>}
<cfelse>
    {"isUpdateAvail": "No"}
</cfif>


</cfif>
3
  • 1
    The obvious solution sounds like a MySQL Trigger but that would probably also require a User Defined Function in order to call the Web Service. This might be over-kill. Do you have access to the code that makes the CREATE, UPDATE and DELETE database calls? If so you could simply add and additional call in at that point to make the notification of the change to additional systems. Let me know if this is the case and I'll also submit an answer.
    – leggetter
    Jun 25, 2011 at 14:10
  • @leggetter, yes, this is the case. Basically, I have a Web Service updating the DB, which could also act as a notifier. The issue I am facing is how do I notify / respond to the notification?
    – AngeloS
    Jun 27, 2011 at 1:40
  • Ok, I've submitted an answer explaining how I would trigger a push notification event and use a real-time push solution. Triggers can be tricky and aren't always the simplest solution.
    – leggetter
    Jul 5, 2011 at 14:30

8 Answers 8

8

This isn't too difficult. The simple way would be to add via .append:

$( '#table > tbody:last').append('<tr id="id"><td>stuff</td></tr>');

Adding elements real-time isn't entirely possible. You'd have to run an Ajax query that updates in a loop to "catch" the change. So, not totally real-time, but very, very close to it. Your user really wouldn't notice the difference, though your server's load might.

But if you're going to get more involved, I'd suggest looking at DataTables. It gives you quite a few new features, including sorting, paging, filtering, limiting, searching, and ajax loading. From there, you could either add an element via ajax and refresh the table view, or simply append on via its API. I've been using DataTables in my app for some time now and they've been consistently cited as the number 1 feature that makes the immense amount of data usable.

--Edit --

Because it isn't obvious, to update the DataTable you call set your Datatables call to a variable:

var oTable = $('#selector').dataTable();

Then run this to do the update:

  oTable.fnDraw(false);

UPDATE -- 5 years later, Feb 2016: This is much more possible today than it was in 2011. New Javascript frameworks such as Backbone.js can connect directly to the database and trigger changes on UI elements including tables on change, update, or delete of data....it's one of these framework's primary benefits. Additionally, UI's can be fed real-time updates via socket connections to a web service, which can also then be caught and acted upon. While the technique described here still works, there are far more "live" ways of doing things today.

5
  • I like the Data Tables approach. Is there any way to set up AJAX to "listen" for a server side event? The code I am writing to insert the record could also function as a notifier, I just need to know how to notify and how to listen. Is that an AJAX function or server-side function?
    – AngeloS
    Jun 24, 2011 at 18:12
  • I think I'd run an Ajax query in a setInterval loop, which is basically just javascript's way of firing a function every X period of time. The Ajax would page Cold Fusion, which would hit the database and return a JSON string of results. To me, the easiest AJAX to understand has been Jquery's .ajax. So, build a function that does an ajax call, in the success set your results, and enclose all with setInterval. Easy Peasy.... Jun 24, 2011 at 19:25
  • That makes total sense. Thank you! .. in your opinion, would this be a scalable solution since we would be hitting my DB / a CF service multiple times per minute per each user?
    – AngeloS
    Jun 27, 2011 at 1:57
  • @angelo, that depends on your circumstances. Database size, query complexity, indexing, joins, and server specifics are all going to factor into the results. When I attempt things like this, I put in debug code that reports time started, time ended, etc for queries, then try things until I get the best result. Jun 27, 2011 at 3:32
  • Gotcha.. most likely going to go with this solution, just testing how it is going to work. Also, you're missing a "$" in front of ('#selector')
    – AngeloS
    Jun 27, 2011 at 21:58
7

You can use SSE (Server Sent Events) a feature in HTML5.

Server-Sent Events (SSE) is a standard describing how servers can initiate data transmission towards clients once an initial client connection has been established. They are commonly used to send message updates or continuous data streams to a browser client and designed to enhance native, cross-browser streaming through a JavaScript API called EventSource, through which a client requests a particular URL in order to receive an event stream.

heres a simple example

http://www.w3schools.com/html/html5_serversentevents.asp

1
6

In MS SQL, you can attach a trigger to a table insert/delete/update event that can fire a stored proc to invoke a web service. If the web service is CF-based, you can, in turn, invoke a messaging service using event gateways. Anything listening to the gateway can be notified to refresh its contents. That said, you'd have to see if MySQL supports triggers and accessing web services via stored procedures. You'd also have to have some sort of component in your web app that's listening to the messaging gateway. It's easy to do in Adobe Flex applications, but I'm not sure if there are comparable components accessible in JavaScript.

While this answer does not come close to directly addressing your question, perhaps it will give you some ideas as to how to solve the problem using db triggers and CF messaging gateways.

M. McConnell

1
  • Yeah, I saw a bunch of Flex examples, which lead my to this question in the first place, but i like your idea about firing a SP that calls a web service. I'm going to keep searching for the right messaging service though as that is what is vexing me the most at the moment.
    – AngeloS
    Jun 27, 2011 at 1:53
3

With "current" technologies, I think long polling with Ajax is your only choice. However, if you can use HTML5, you should take a look at WebSockets which gives you the functionality you want.

http://net.tutsplus.com/tutorials/javascript-ajax/start-using-html5-websockets-today/

WebSockets is a technique for two-way communication over one (TCP) socket, a type of PUSH technology. At the moment, it’s still being standardized by the W3C; however, the latest versions of Chrome and Safari have support for WebSockets.

http://html5demos.com/web-socket

2
  • 1
    This is the idea I love the most, but what I'm worried about are my users using older browsers that do not support HTML5. Should this be a valid concern?
    – AngeloS
    Jun 27, 2011 at 1:48
  • Yes, it's a valid concern. However ColdFusion 10's new CFWEBSOCKET tag enables the HTML5 functionality on an HTML5 capable browser, otherwise it falls back to a Flash widget to provide the push functionality. Without CF10, you'll have to write your own fallback or use something like this: code.google.com/p/jquery-graceful-websocket. May 10, 2013 at 15:21
3

Check out AJAX long polling. Place to start Comet

1
  • 2
    Thank you for your response. I did some research on long-polling, but in the examples I have seen (like this SO link question) seem to say that it's not highly scalable due to multiple connections etc. Would you happen to have a scalable example?
    – AngeloS
    Jun 27, 2011 at 1:43
1

No, you can't have any db code execute server side code. But you could write a service to poll the db periodically to see if a new record has been added then notify the code you have that needs pseudo real-time updates.

3
  • Gotcha.. do you have any examples of how to notify the code that needs real-time updates? Would this be done via AJAX or directly through ColdFusion?
    – AngeloS
    Jun 24, 2011 at 18:10
  • @AngeloS I would recommend using Ajax. I am not a ColdFusion programmer so I can't give you examples ... meh. However, the idea is straightforward: make periodic ajax requests via javascript to some service that makes the check against the db. Once you discover that a record has been added, act accordingly. Jun 24, 2011 at 18:12
  • This is incorrect. For example in postgres, you can use triggers to publish events over the notify API. I started a project using this idea a few days ago github.com/cameron-martin/live_query Jul 7, 2014 at 22:50
0

The browser can receive real-time updates via BOSH connection to Jabber/XMPP server. All bits and pieces can be found in this book http://professionalxmpp.com/ which I highly recommend. If you can anyhow send XMPP message upon record addition in your DB, then it is relatively easy to build the dashboard you want. You need strophe.js, Jabber/XMPP server (e.g. ejabberd), http server for proxying http-bind requests. All the details can be found in the book. A must read which I strongly believe will solve your problem.

0

The way I would achieve the notification is after the database update has been successfully committed I would publish an event that would tell any listening systems or even web pages that the change has occurred. I've detailed one way of doing this using an e-commerce solution in a recent blog post. The blog post shows how to trigger the event in ASP.NET but the same thing can easily be done in any other language since ultimately the trigger is performed via a REST API call.

The solution in this blog post uses Pusher but there's not reason why you couldn't install your own real-time server or use a Message Queue to communication between your app and the realtime server, which would then push the notification to the web page or client application.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.