Web interface to a Python script, with "streaming text"

Question

I have a Python command-line script that can take some time to run (from 5 seconds to 5 minutes), depending on the checks it does. The script uses many command-line arguments, some mandatory, most optional.

I'd like to "webify" this script, making it possible to access it from a Web browser, but without changing the core python script, so by launching it from something else (like another Python script for instance)

I've used mod_python with Apache before so it's not too much of a problem to build a Python script that generates a Web page with a form and a Submit button and run my core python script from it.

The problem is with the output of the core python script. I don't want to wait until it has completed to display its output in a DIV (or in a FRAME). As soon as it generates a line, I want that line to be displayed on the Web interface, a kind of text streaming in the DIV.

I know how to do this from a Javascript contained in a HTML page or as an external Javascript file, but not from the output of another program.

Any idea how I can achieve this? I prefer to keep it all Python+Apache but if something else is really required (PHP, AJAX, Javascript, ...), I can live with it.

---

As suggested by whatnick below, I tried to redirect the output of the core python script in a temporary file and display this file using the AJAX Logfile Trailer & Viewer code I found on the Web.

It sort of works but there is still some buffering, as the lines appear in the "log tail page" in several chunks of lines, not lines by lines. This is most probably due to the fact the AJAX script reloads the log file using a timer, every X milliseconds (configurable in the script, used for a Javascript timer). Even if I lower it, it still not fast enough for my core script, which can sometimes output several lines very fast.

Answer 1

You can log the output from your lengthy script to a file and use a javascript timer based code to pull, parse and display content of the log periodically. If pulling the entire log is too much, you can also delegate to a smaller/faster server side python script to parse the log and pass on the tail of the log to the calling javascript. This is how router logs etc. in embedded devices is displayed.

Good luck. Here is a small tutorial on Javascript Timers.

Answer 2

For IE and Firefox, if you stream your response, it will start rendering before the full page is provided. You can thus stream a log of what is going on in the middle of an HTML file. Once the process is completed, you render the rest of the page as normal.

You will have to go through some convolutions to get a normal Python template engine to work with that since they tend to do everything in one shot.

Chrome does not seem to like this very well (I expect that is a webkit thing so you can probably discount Safari from this trick as well) and I am not sure about Opera.