Apache Rivet

As with any tool, it's always nice to see something work, so let's create a small "Hello World" page.

Assuming you have Apache configured correctly, create a file called hello.rvt where Apache can find it, with the following content:

<?
puts "Hello World"
?>

If you then access it with your browser, you should see a blank page with the text "Hello World" (without the quotes) on it.

In another simple example, we dynamically generate a table:

<? puts "<table>\n"
for {set i 1} { $i <= 8 } {incr i} {
    puts "<tr>\n"
    for {set j 1} {$j <= 8} {incr j} {
        set num [ expr $i * $j * 4 - 1]
        puts [ format "<td bgcolor=\"%02x%02x%02x\" > $num $num $num </td>\n" \
		   $num $num $num ]
    }
    puts "</tr>\n"
}
puts "</table>\n" ?>

If you read the code, you can see that this is pure Tcl. We could take the same code, run it outside of Rivet, and it would generate the same HTML!

The result should look something like this:

Here, we demonstrate how to access variables set by GET or POST operations.

Given an HTML form like the following:

     <form action="vars.rvt">
      <table>
	<tbody>
	  <tr>
	    <td><b>Title:</b></td>
	    <td><input name="title"></td>
	  </tr>
	  <tr>
	    <td><b>Salary:</b></td>
	    <td><input name="salary"></td>
	  </tr>
	  <tr>
	    <td><b>Boss:</b></td>
	    <td><input name="boss"></td></tr>
	  <tr>
	    <td><b>Skills:</b></td>
	    <td>
	      <select name="skills" multiple="multiple">
		<option>c</option>
		<option>java</option>
		<option>Tcl</option>
		<option>Perl</option>
	      </select>
	    </td>
	  </tr>
	  <tr>
	    <td><input type="submit"></td>
	  </tr>
	</tbody>
      </table>
    </form>

We can use this Rivet script to get the variable values:

<?
set errlist {}
if { [var exists title] } {
    set title [var get title]
} else {
    set errlist "You need to enter a title"
}

if { [var exists salary] } {
    set salary [var get salary]
    if { ! [string is digit $salary] } {
	lappend errlist "Salary must be a number"
    }
} else {
    lappend errlist "You need to enter a salary"
}

if { [var exists boss] } {
    set boss [var get boss]
} else {
    set boss "Mr. Burns"
}

if { [var exists skills] } {
    set skills [var list skills]
} else {
    lappend errlist "You need to enter some skills"
}

if { [llength $errlist] != 0 } {
    foreach err $errlist {
	puts "<b> $err </b>"
    }
} else {
    puts "Thanks for the information!"
    ?>
    <table>
      <tbody>
	<tr>
	  <td><b>Title:</b></td>
	  <td><? puts $title ?></td>
	</tr>
	<tr>
	  <td><b>Boss:</b></td>
	  <td><? puts $boss ?></td>
	</tr>
	<tr>
	  <td><b>Salary:</b></td>
	  <td><? puts $salary ?></td>
	</tr>
	<tr>
	  <td><b>Skills:</b></td>
	  <td><? puts $skills ?></td>
	</tr>
      </tbody>
    </table>
    <?
}
?>

The first statement checks to make sure that the boss variable has been passed to the script, and then does something with that information. If it's not present, an error is added to the list of errors.

In the second block of code, the variable salary is fetched, with one more error check - because it's a number, it needs to be composed of digits.

The boss variable isn't required to have been sent - we set it to "Mr. Burns" if it isn't among the information we received.

The last bit of variable handing code is a bit trickier. Because skills is a listbox, and can potentially have multiple values, we opt to receive them as a list, so that at some point, we could iterate over them.

The script then checks to make sure that errlist is empty and outputting a thankyou message. If errlist is not empty, the list of errors it contains is printed.

The upload command endows Rivet with an interface to access files transferred over http as parts of a multipart form. The following HTML in one file, say, upload.html creates a form with a text input entry. By clicking the file chooser button the file browser shows up and the user selects the file to be uploaded (the file path will appear in the text input). In order to make sure you're uploading the whole file you must combine the action of the enctype and method attributes of the <form...> tag in the way shown in the example. Failure to do so would result in the client sending only the file's path, rather than the actual contents.

<form action="foo.rvt" enctype="multipart/form-data" method="post">
<input type="file" name="MyUpload"></input>
<input type="submit" value="Send File"></input>
</form>

In the script invoked by the form (upload.rvt) upload ?argument ...? commands can be used to manipulate the various files uploaded.

<?
upload save MyUpload /tmp/uploadfiles/file1
puts "Saved file [upload filename MyUpload] \
	([upload size MyUpload] bytes) to server"
?>

Don't forget that the apache server must have write access to the directory where files are being created. The Rivet Apache directives have a substantial impact on the upload process, you have to carefully read the docs in order to set the appropriate directives values that would match your requirements.

It is also important to understand that some upload commands are effective only when used in a mutually exclusive way. Apache stores the data in temporary files which are read by the upload save ?upload name? ?filename? or by the upload data ?upload name? command. Subsequent calls to these 2 commands using the same ?upload name? argument will return no data on the second call. Likewise upload channel ?upload name? will return a Tcl file channel that you can use in regular Tcl scripts only if you haven't already read the data, for example with a call to the upload data ?upload name? command.

In general setting up a data file to be sent over http is as easy as determining the file's URI and letting Apache's standard download mechanism do all that is needed. If this approach fits your design all you have to do is to keep the downloadable files somewhere within Apache's DocumentRoot (or in any of the directories that you can configure and register using the Alias definitions or the Virtual Hosts mechanism).

When a client sends a request for a file Apache takes care of determining the filetype, sends appropriate headers to the client and then the file content. If the client is a browser capable of displaying the content of the file a representation is shown in the browser's window. When the browser hasn't a valid builtin method or plugin registered for the file's content the typical download dialog pops up asking for directions from the user.

Rivet can help if you have more sofisticated needs. For instance you may be developing an application that uses webpages to collect input data that have to be passed on to scripts or programs. In this case the content is generated on demand and a real file representing the data doesn't exist on the server. In other circumstances you may need to dynamically inhibit the download of a specific file and hide it away, even to those clients that might have already saved the URI to the file in their bookmarks. Your scripts may expunge from the pages every link to the file (your pages are dynamic, aren't they?) and move the file out of way, but it looks like a cumbersome solution.

Putting Tcl and Rivet in charge of the whole download mechanism helps in building cleaner and safer approaches to the download problem.

In this example a procedure checks for the existence of a parameter passed in by the browser. The parameter is the name (without extension) of a pdf file. Pdf files are stored in a directory whose path is in the pdf_repository variable.

# Code example for the transmission of a pdf file. 

if {[var exists pdfname]} {
    set pdfname [var get pdfname]

# let's build the full path to the pdf file. The 'pdf_repository'
# directory must be readable by the apache children

    set pdf_full_path [file join $pdf_repository ${pdfname}.pdf]
    if {[file exists $pdf_full_path]} {

# Before the file is sent we inform the client about the file type and
# file name. The client can be proposed a filename different from the
# original one. In this case, this is the point where a new file name
# must be generated.

	headers type			"application/pdf"
	headers add Content-Disposition "attachment; filename=${pdfname}.pdf"
	headers add Content-Description "PDF Document"

# The pdf is read and stored in a Tcl variable. The file handle is
# configured for a binary read: we are just shipping raw data to a
# client. The following 4 lines of code can be replaced by any code
# that is able to retrieve the data to be sent from any data source
# (e.g. database, external program, other Tcl code)

	set paper	    [open $pdf_full_path r]
	fconfigure	    $paper -translation binary
	set pdf		    [read $paper]
	close $paper

# Now we got the data: let's tell the client how many bytes we are
# about to send (useful for the download progress bar of a dialog box)

	headers add Content-Length  [string length $pdf]

# Let's send the actual file content

	puts $pdf
    } else {
	source pdf_not_found_error.rvt
    }
} else {
    source parameter_not_defined_error.rvt
}

Before the pdf is sent the procedure sets the Content-Type, Content-Disposition, Content-Description and Content-Length headers to inform the client about the file type, name and size. Notice that in order to set the Content-Type header Rivet uses a specialiezed form of the headers command. Headers must be sent before data gets sent down the output channel. Messing with this prescription causes an error to be raised (in fact the protocol itself is been violated)

More information about the meaning of the mime headers in the http context can be found at http://www.w3.org/Protocols/rfc2616/rfc2616.html

The headers command is crucial for generating XML messages that have to be understood by JavaScript code used in Ajax applications.

Ajax is a web programming technique that heavily relies on the abilty of JavaScript (and other scriping languages like VBScript) to manipulate dynamically the HTML structure of a page. In modern browsers JavaScript code is enabled to send GET or POST requests to the webserver. These requests can ask the server to run scripts (for example Rivet Tcl scripts) that build responses and send them back to the browser. JavaScript code reads asynchronously these responses, elaborates their content and accordingly modifies the page on display. Ajax helps to build web applications that are more responsive and flexible. Instead of going through the cycle of request-generation-transfer of a page, Ajax allows the programmer to request and transmit only the essential data thus matching the general requirement of separation between data and user interface (and saving the server from sending over the same html code and graphics every time a page is refreshed)

In Ajax applications the communication between client and server is controlled by an instance of a specialized object: the non-IE world uses the XMLHttpRequest class to create it, whereas IE uses the ActiveXObject class. Through an instance of that class a POST or GET request is sent to the server, which in turn responds with a message that is stored as a string in a property of the communication object called 'returnedText'. Although this sort of communication channel doesn't imply that the messages being transmitted through it have a specific protocol, it's become widely customary to use XML as the format for such messages. A number of XML specification are being used for this, among which XML-RPC and SOAP are worth to be quoted. Anyway, you can invent your own protocol (either based on XML or anything else), but one has to be aware of the fact that if the http headers are properly set and the message returned to the client is a well formed XML fragment, also the property XMLResponse is set with a reference to an object that stores the response as an XML document. This object is an instance of a class that implements the XML W3C DOM tree methods and properties, thus enabling the scripts to read and manipulate the data embedded in the XML message with a structured and standard interface.

In this example a Rivet script initializes an array with the essential data regarding a few of the major composers of the european music. This array plays the role of a database which, in a real case, might store large tables with thousands records and more complete and extended information. The script is designed to send back to the client two types of responses: a catalog of the composers or a single record of a composer.

# The database array contains xml fragments representing the
# results of queries to a database. Many databases are now able
# to produce the results of a query in XML. 
#
#array unset composer
#
set	composer(1)	"<composer>\n"
append  composer(1)     "    <first_name>Claudio</first_name>\n"
append  composer(1)	"    <last_name>Monteverdi</last_name>\n"
append	composer(1)	"    <lifespan>1567-1643</lifespan>\n"
append	composer(1)	"    <era>Renaissance/Baroque</era>\n"
append	composer(1)	"    <key>1</key>\n"
append	composer(1)	"</composer>\n"

set	composer(2)	"<composer>\n"
append  composer(2)     "    <first_name>Johann Sebastian</first_name>\n"
append  composer(2)	"    <last_name>Bach</last_name>\n"
append	composer(2)	"    <lifespan>1685-1750</lifespan>\n"
append	composer(2)	"    <era>Baroque</era>\n"
append	composer(2)	"    <key>2</key>\n"
append	composer(2)	"</composer>\n"

set	composer(3)	"<composer>\n"
append  composer(3)     "    <first_name>Ludwig</first_name>\n"
append  composer(3)	"    <last_name>van Beethoven</last_name>\n"
append	composer(3)	"    <lifespan>1770-1827</lifespan>\n"
append	composer(3)	"    <era>Romantic</era>\n"
append	composer(3)	"    <key>3</key>\n"
append	composer(3)	"</composer>\n"

set	composer(4)	"<composer>\n"
append  composer(4)     "    <first_name>Wolfgang Amadaeus</first_name>\n"
append  composer(4)	"    <last_name>Mozart</last_name>\n"
append	composer(4)	"    <lifespan>1756-1791</lifespan>\n"
append	composer(4)	"    <era>Classical</era>\n"
append	composer(4)	"    <key>4</key>\n"
append	composer(4)	"</composer>\n"

set	composer(5)	"<composer>\n"
append  composer(5)     "    <first_name>Robert</first_name>\n"
append  composer(5)	"    <last_name>Schumann</last_name>\n"
append	composer(5)	"    <lifespan>1810-1856</lifespan>\n"
append	composer(5)	"    <era>Romantic</era>\n"
append	composer(5)	"    <key>5</key>\n"
append	composer(5)	"</composer>\n"

# we use the 'load' argument in order to determine the type of query
#
# load=catalog:		    we have to return a list of the names in the database
# load=composer&res_id=<id>: the script is supposed to return the record
#			    having <id> as record id

if {[var exists load]} {

# the xml declaration is common to every message (error messages included)

    set xml "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>\n"
    switch [var get load] {
	catalog {
	    append xml "<catalog>\n"
	    foreach nm [array names composer] {
	    	if {[regexp {<last_name>(.+)</last_name>}   $composer($nm) m last_name] && \
		    [regexp {<first_name>(.+)</first_name>} $composer($nm) m first_name]} {
	            append xml "    <composer key='$nm'>$first_name $last_name</composer>\n"
		}
	    }
	    append xml "</catalog>"
	}
	composer {
	    if {[var exists rec_id]} {
		set rec_id [var get rec_id]
		if {[info exists composer($rec_id)]} {
		    append xml $composer($rec_id)
		}
	    }
	}
    }

# we have to tell the client this is an XML message. Failing to do so
# would result in an XMLResponse property set to null

    headers type "text/xml"
    headers add Content-Length [string length $xml]
    puts $xml
}

For sake of brevity the JavaScript and HTML will not listed here. They can be downloaded (along with the Tcl script) stored in the rivet-ajax.tar.gz archive. By simply opening this tar archive in a directory accessible by your apache server and pointing your browser to the rivetService.html page you should see a page with a drop-down list. Every time a different name is picked from the list a new query is sent and logged in the apache access.log file, even though the html is never reloaded.