1

My ASP Classic application are fetching over 10.000 rows via a triple inner join. After this, it is going through each line, calculating some stuff and putting data in a multidimensional array with 17 coloumns.

All this are being fetched through a jQuery AJAX when pushing the search button.

Logically this takes a while, 5+ minutes actually.

It all works fine. BUT... While doing this, the whole system freezes, not only for the user doing the calculations, but also for all the other users using the system in other ways. How can I optimize this?!

A small relevant code snippet:

dim userArray()

sql = "select first.*, second.fullname, second.info, third.inputs from first inner join second on first.userid = second.id inner join third on first.info = third.id where convert(varchar, first.period, 112) between '20020115' and '20120115' order by second.fullname, first.userid"
set rs = conn.execute(sql)

if rs.eof then
  response.write("Nothing were found in the period")
else
  counter = 0
  redim userArray(17, 1000)
  do until rs.eof
    response.flush
    counter = counter + 1

  ' A LOT of calculations and putting in array...

    rs.movenext
  loop

  for i = 1 to counter
    ' Doing all the response.writes to the user...
  next
end if
6
  • 1
    What system do you have? Is the database on the same system? I bet the system is crap and if the database is on the same system, the disc system pathetic - and overlaoded. THis results in the OS becoming totally unresponsive.
    – TomTom
    Jan 20, 2012 at 9:53
  • The OS is Windows Server 2008 R2 running the latest IIS and SQL-Server, and yeah. The DB and IIS are on the same server. I dont think that the OS is unresponsive, because I can log on to the server via remote desktop meanwhile.. Therefore it must be the IIS-server, because the DB are still responding to other applications when my localhost are connecting to it..
    – MicBehrens
    Jan 20, 2012 at 10:06
  • While the query is running, open a connection to the DB and exec sp_lock. Has your query locked tables that the other users need access to?
    – Journey
    Jan 20, 2012 at 10:11
  • If doing a sp_who meanwhile the query is running, I get a cmd: "select", status: "suspended" on the user the web-application uses.. Does this mean noone on the webuser can do a SELECT command while it is suspended?
    – MicBehrens
    Jan 20, 2012 at 10:21
  • In general others can SELECT from the same tables as you are, but no one may update them while you're reading from them. (Though you can change this with query hints and transaction isolation levels.)
    – Journey
    Jan 20, 2012 at 10:29

3 Answers 3

1

Lets analyse this whilst also bearing mind the SQL has an ORDER BY Clause:-

do until rs.eof 
  response.flush 
  counter = counter + 1 

  ' A LOT of calculations and putting in array... 

  rs.movenext 
loop

Note the Response.Flush, first thing I would do is get rid of that. You will probably need to increase the ASP Response Buffering Limit (in IIS manager). Flush is sending the generated content so far to the client, it waits for the client to acknowledge receipt of all the packets sent before it completes. That is where I'm gonna guess 90% of the 5+ minutes is being spent.

Now "A LOT calculations". VBScript is not know for its peformance. This code may well take some time. In some cases some calculations can be done much better by SQL than in script so that is one option. Another would be to build some COM compiled component to do complex work (although some accounting needs to made for marshalling which can wipe out benefits). However it may be unavoidable that you need to do these calcs in VBScript.

Now rs.movenext. This loop means you hold the connection and rowset open for pretty much all the time the processing is required. That is while the servers is sending bytes over the network to the client and while VBScript is crunching numbers. A much better approach would be suck up all the rowset quickly and disconnect from the DB, then crunch numbers and finally dump the buffer to the client.

Consider using a disconnected recordset (you specify a client side static cursor) or even the simple GetRows method of the recordset object that dumps the whole rowset into a 2-dimensional array. This will mean that you maintain locks on the various tables for the smallest time possible.

4
  • How high would you recommend I set the "Response Buffering Limit" to in IIS Manager -> Site -> ASP? Currently it is set to 4194304
    – MicBehrens
    Jan 20, 2012 at 13:08
  • 1
    @erizias: The purposes of this setting is protect servers that host many third-party sites from any one site gobbling up all the resources. For servers that you control I recommend you set it to something really big like 2GB. Jan 20, 2012 at 13:19
  • The server IS going to host a lot of sites.. but all of the sites are my own systems.. The current setting, is that in kB or B? :)
    – MicBehrens
    Jan 20, 2012 at 13:25
  • Ah it's in bytes.. so I need to set it to 2097152000 to get 2gb :)
    – MicBehrens
    Jan 20, 2012 at 13:29
1

I see you already use response.flush() to flush data to the browser intermittedly during the process, but if you're using AJAX, the call must first complete before your AJAX callback function is called, so I think response.flush is not going to be of any use there. You might try calling the AJAX url directly, and put in a response.write() in the loop to see what happens (and what the speed is)

To get even more information, you could add a timer before the query, after the query and inside the loop and response.write the time that has passed since starting the script. This will give you a very good idea where the delay is happening: http://www.codefixer.com/codesnippets/vbscript_timer_function.asp

You say the machine freezes, is that the client PC with the browser, or the server where IIS runs? If a bucketload of data is being sent I have seen browsers hang and not update until it's done.

Try to add WITH NOLOCK after each table name in your query to select without locking the database for writing. This might give you some data that was overwritten during the execution of your query, but that's usually not a problem.

Also, indexes. Try changing the clustered index on the fields you use in your WHERE statement, or add some regular indexes if you need your clustered index. Optimizing your indexes will speed things up considerably if you haven't done so.

HTH,

Erik

1

I'd refactor that code like this:

sql = "select first.*, second.fullname, second.info, third.inputs from first inner join second on first.userid = second.id inner join third on first.info = third.id where convert(varchar, first.period, 112) between '20020115' and '20120115' order by second.fullname, first.userid"
Set rs = conn.Execute(sql)
If NOT rs.EOF Then
    aRecords = rs.GetRows() ' retrieve your records and assign them to an array '
End If
rs.Close ' record set is now released, so it shouldn't lock up your database anymore

If IsArray(aRecords) Then ' just a small sanity check '
    iCount = UBound(aRecords, 2)
    For iCounter = 0 To iCount
        ' Your calculations and your Response.Writes '
    Next
    Erase aRecords
Else
    ' no result found '
End If


Update

You can assign the records to variables in your For loop, e.g.

id = aRecords(0, iCounter)

Then you only need to refer to id when ever you need it. You're right though in that if what you're selecting is dynamic (i.e. column's positions can shift), then this approach can produce problems for when you're trying to assign records to variables further along the line. To assign fullname from your second table, for instance, you'd have to know how many columns are being retrieved from first.*.

3
  • 1
    This is pretty much the code @AnthonyWJones was referring to in his last recommendation. I have never used getRows() before, so this is great! Thanks :)
    – MicBehrens
    Jan 20, 2012 at 13:18
  • I found 1 problem doing this.. now i need to do fx aRecords(0, iCounter) to get the ID of the rows where I before just did rs("id") .. is there anyway to fix this? It is dynamic how many coloumns that in the table and how many of them I actually need..
    – MicBehrens
    Jan 20, 2012 at 14:03
  • I just figured out how to do a function to do all this ! :) I just updated the answer showing the result, but I dont have the rights to edit, so its queued :/
    – MicBehrens
    Jan 20, 2012 at 15:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.