Outline · [ Standard ] · Linear+ Webserver with Ajax and XML

[DonGato]

I sugest another approach:

Don't use templates. Just serve an HTML and use the JS to fecth the data as per AJAX code.

[Fuxie - DK]

Do you mean to build both templates into eMule.exe or simply to have ONE look (as if there was only one templste)???

[muleteer]

Whichever method we choose, I think we should also look seriously at supporting partial rendering of large pages. Break up the page into manageable chunks, transfer each chunk as soon as it is ready, and on the remote side render in the browser whatever is received instead of waiting until all the data is received before rendering.

Eg. this forum supports partial rendering. When I open a long thread, a few posts are shown first, then a few more, then more, until finally the page is done.

[DonGato]

Do you mean to build both templates into eMule.exe or simply to have ONE look (as if there was only one templste)???

No, I mean to remove templates and just have static HTML content, but just realized we offer translations (that I never used) and that would be a problem.

[Darklord]

I sugest another approach:

Don't use templates. Just serve an HTML and use the JS to fecth the data as per AJAX code.

Its good practice to allow usage to non ajax adopters.

I think a static usage should be kept.

But it was me who sugested the xml+xslt combo, so....

No, I mean to remove templates and just have static HTML content, but just realized we offer translations (that I never used) and that would be a problem.

Yep, this was a problem i found.

See Implementation Notes for possible solutions.

Tried to find other implementation cases that used i18n and l10n but still havent found anything, looking on a possibility.

[Aw3]

The thing is that the main slow down is not because of data reloading, it is because of data preparation. I think we can make it 10 times faster just by rewriting a couple of places in WS...

That is also an advantage of the ajax way, since a lot less needs to be prepared and sent.

You need to prepare anyway, as you approach can work for some information field, but the lists are changing all the time, thus the whole list should be resend.

I believe the biggest cpu sink its the search and replace operations (correct me if im wrong).

The replace as its done now will search all over the document for a given tag to substitute, even if theres only one ocorrence and EVEN if there isnt that tag.

Actually no, search & replace is not the bottleneck. And it is not that slow as you think, because every page is split into several templates and you don't process the whole page every time.

[Darklord]

The thing is that the main slow down is not because of data reloading, it is because of data preparation. I think we can make it 10 times faster just by rewriting a couple of places in WS...

That is also an advantage of the ajax way, since a lot less needs to be prepared and sent.

You need to prepare anyway, as you approach can work for some information field, but the lists are changing all the time, thus the whole list should be resend.

Tue, but there is not getting around that, i think.

I believe the biggest cpu sink its the search and replace operations (correct me if im wrong).

The replace as its done now will search all over the document for a given tag to substitute, even if theres only one ocorrence and EVEN if there isnt that tag.

Actually no, search & replace is not the bottleneck. And it is not that slow as you think, because every page is split into several templates and you don't process the whole page every time.

Thats the point, although its split into templates, they are assembled (some, at least) BEFORE the substitutions; as those are added, the bigger the string to check gets; and the problem i pointed of checking the whole string remains.

[Darklord]

Fuxie's client is our test bench for WS performance. It is an extreme scenario with older hardware, so processing time is significant.
Its not just the number of files in his downloads; his upload queue size is about 7000. We seldom dare to expand his upload queue in the WS.

Now that i have the XML response for the upload queue, it would be fun to see how long it takes.

Here is the new client.

The request is http://[webserver url]/?ses=[valid user session]&channel=xml&request=getUpQueue (be careful, the url is case sensitive).

Also added the debug lines to the log, to check the metrics.

Dont mind the size sent, gzip isnt used.

[Darklord]

Added upload list to xml requests, still havent changed the template.

Here is the new client.

The request url is http://[webserver url]/?ses=[valid user session]&channel=xml&request=getUpList (case sensitive).

[Fuxie - DK]

Please explain IN DETAIL what we're supposed to test?

[Darklord]

Please explain IN DETAIL what we're supposed to test?

Ok.

1. Download the binary package.
   
2. Access the webserver through the browser as usual, authenticating, and click on Transfer.
   
3. Look at the url on the location box. If you acess it on a local copy, with the standard port, it should be something like "http://localhost:4711/?ses=[a number]&w=transfer"
   
4. There, "localhost:4711" is the webserver url i talked about before, and the number [a number] is the valid session id, that needs to be passed to every request.
   
5. To make xml requests, edit the url on the location box to the one given, substituting [webserver url] with the url of your webserver, and [valid session id] with the number found above. The urls are case sensitive, so beware.
   
6. There are three accepted xml requests with the latest executable:
   • a stats request, using the url:
     
     
     CODE

     
     http://[webserver url]/?ses=[valid user session]&channel=xml&request=getStats
     

   • a upload queue request, using the url:
     
     
     CODE

     
     http://[webserver url]/?ses=[valid user session]&channel=xml&request=getUpQueue
     

     
     
     
   • a upload list request, using the url:
     
     
     CODE

     
     http://[webserver url]/?ses=[valid user session]&channel=xml&request=getUpList
     

7. Its possible to check request times and sizes on the debug log, it uses the same notation of other requests, but marking them as a XML request.
   
8. All sizes are unzipped, it isnt sent compressed yet.
   
9. Im looking for problems with the construction of the xml; both IE and Firefox will complain if a problem got through
   
10. one possible problem may be with names as it uses Latin-1 encoding as of yet, and there isnt a function to escape foreign characters.
    
11. Using the template, the webserver client does a stats request every minute, to update the stats bar

[Fuxie - DK]

Please explain IN DETAIL what we're supposed to test?

[*] Download the binary package.

You're using an old template.. There are some incorrections on the Login-Page..

CODE

http://[webserver url]/?ses=[valid user session]&channel=xml&request=getStats

[*] a upload queue request, using the url:

CODE

http://[webserver url]/?ses=[valid user session]&channel=xml&request=getUpQueue

[*] a upload list request, using the url:

CODE

http://[webserver url]/?ses=[valid user session]&channel=xml&request=getUpList

When trying these three URLs all I get in MSIE is the display of a XML file.. Can't use that for much...

[Darklord]

You're using an old template.. There are some incorrections on the Login-Page..

Ya, havent updated that one, only the executable.

When trying these three URLs all I get in MSIE is the display of a XML file.. Can't use that for much...

Actualy thats the point It isnt supposed to be processed by humans. But it has a lot of uses, as you can see when letting the webserver open (check the time, and the debug log), and with something im working on.

What you can do is check the request times on the debug log and see if the preparation times arent too big, like you did before.

[Fuxie - DK]

The main problem from my PoV isn't the process time in eMule.exe... It's the time it takes for MSIE to prepare the page AFTER the page has been generated and sent to MSIE...

Transfer Page is the main problem, and the way I see it, that hasn't been dealt with at all yet..

[Darklord]

The main problem from my PoV isn't the process time in eMule.exe... It's the time it takes for MSIE to prepare the page AFTER the page has been generated and sent to MSIE...

Transfer Page is the main problem, and the way I see it, that hasn't been dealt with at all yet..

Yep, and that is also one of the strong points of the ajax way, since it doesnt reload the entire page, just the changing bits.

And no, only the stats bar is updated that way yet, like i said, there isnt any update to the template.

The XML requests are an essential step for it (the transmition of the data in a way the browser can understand), but can be of a lot of value by themselves, as i will show... soon.