Freeport Translation Software

This software is both a a gateway for the Plexus server, and a CGI for other servers (Such as Apache) that will read Freeport style menu files used by the National Capital Freenet and send HTML to the browser. It was originally written by Michael Richardson, with updates by Russell McOrmond. Matthew Darwin re-wrote it as an Apache module.

While this software could be of general use for any Freenet system migrating from Freeport to the WEB, this particular project is centered around the National Capital Freenet here in Ottawa.

There are a number of different ways to configure this software:

Using Plexus, and the translator as a Gateway

Plexus

freeport_gateway.pl

This is the PLEXUS gateway that reads the Freeport menu files and outputs HTML to the browser.

freeport.pl

These are the common routines usedby each translator.

pattern.pl

This is a gateway that is used to support 'p' commands when a pattern is given. (Currently not supported/tested)

Configuration in local.conf and plexus.conf

The sections referencing freeport.pl are required to configure the freeport_gateway.pl script. In plexus.conf, the various variables are set such as where the 'information providers' (/ip/) files are (What to put in front of variable filenames for 'p',etc), as well as the root directory (where the files for /freeport/ are actually stored on the filesystem.

Using Apache with the translator configured as a CGI

Apache

This server can be configured to have references to /freeport/ set to be sent to a CGI that will return the translated code.
In srm.conf:
ScriptAlias /freeport/ /home/www/floraweb/cgi-bin/freeport_cgi/


freeport_cgi

This is the CGI wrapper for the routines in freeport_inc.pl. If given a full filename on the command line, it will output the translated HTML code to STDOUT.

freeport.pl

These are the common routines usedby each translator.

Using Apache with the translator configured to do offline batch translation

Apache with an updated mod_mime.c which contains the 'ForceType' config option.

This server can be configured to have all files in a specific directory automatically be returned as a specific mime-type. In this case we wish ALL files in the directory to be returned as text/html regardless of their filenames.

<DIRECTORY /home/www/data/flora/free>
DefaultType text/html
ForceType text/html
...
</Directory>


freeport_batch

This is a wrapper for the routines in freeport.pl. It will read a file listing all the filenames (Created by a directory walking utility), read these menu files from one directory, and generate translated HTML files into a different directory. On the NCF there is a file called /freenet/rootdir/config/menu_attributes.dat that can be used for this purpose.

freeport.pl

These are the common routines usedby each translator.

This batch translator option has many advantages over a C based translation built into the Apache server, which was another consideration to moving away from the expensive PLEXUS implementation.

• Perl is much easier to maintain. The developer does not need to worry about pointers, allocating memory, or any of those things that can cause problems even for the experienced C developer. PERL was designed to be used in textual-translations such as the Freeport translator.
• No changes to the actual WEB server are required. The WEB server just serves standard HTML files, and therefore this solution can make use of all the optimization features of that WEB server.
• The batch translation option has the advantage of not being server dependent - any WEB server that can serve HTML documents can be used.

Disadvantages:

• A batch translation might not be perfectly up to date with the menu file. This problem can be solved by incorporating the translator into the MMS system, as well as documenting the use of a shell command to translate for menu maintainers still using the shell
• A copy of each menu file is kept in .html file format. Since any translator built into a server may wish to have a cache, this is not a real consideration as the cache would take up a similar amount of space, and also add to the complexity of the system.

As an Apache module

In May 1998, Matthew re-wrote freeport.pl as a perl 5 module (NCF::FreePort::Menu.pm). He added a bunch of functionality, removed some stuff (the header and footer) and cleaned it up. It's also OO now.

The two pieces you need to get things to run as an Apache module are:

• NCF::FreePort::Menu.pm
• NCF::FreePort::ApacheMenu.pm

It works in Apache 1.3.x with mod_perl on Linux.

While many other sites previously ran this software, there are only two that we know of at the moment:

• The National Capital FreeNet Web Technology Center, part of the Davin Community Site [Apache module version]
• National Capital FreeNet (not actively used). [plexus version]

• Update batch translator
  • Create exceptions list for patterns of files that need not be translated.
  • Create 'substitution' list to allow a .html file to be used instead of the translated menu file for specific files. This allows the WEB menu design people to set up special cases for the translator by substituting their own file

With the Freeport menu converter, we have the ability to interpret the Freeport menu files in any way we wish. Often you will read a menu via the WEB, and there will be "Can't convert" messages in places where you might not expect them, or where you feel that the translator might be more intelligent. Alternatively, the Freeport translator might be translating a menu in a way that is felt to be inappropriate.

In order to make things more intelligent, we need to be indicated a few things:

• The menu file needs to be grabbed and brought into the Test menu so that it can be used as an example of the menu structure. It should be specified to us as the URL of the file itself (EG: http://www.ncf.carleton.ca/ip/menu.main is the URL of the main menu)
• A precise description of exactly what you feel the menu translator should be doing, and/or what you feel it is doing incorrectly.

Once we have this information, we will then be in a position to translate this into code that the Freeport to WEB translator would run.

Please post any bug reports or suggestions to the comnet-www@flora.ottawa.on.ca mailing list, or by posting directly to the support newsgroup.

• Handling of User-personal files
  %p sed -e 's/aa000/$USER/g' file | p
The IP's should use '%P WWWURL http://path-to-origional-file', since a UserID is a bit redundant to have displayed and 'aa000' is fine to display to an outside user.
• Handle 'approved' list of scripts/binaries that output text and could be sent to WEB (freenet/subs/groups.pl , freenet/subs/contact.sh, etc ?). These would be handled with the writing of a small shell wrapper script that would make the program into a CGI. This script could then just be placed into the CGI-BIN directory and referenced in the Freeport file as:
  %P WWWURL http://www.ncf.carleton.ca/cgi-bin/groupslist

  File: cgi-bin/groupslist
  

  #!/bin/csh
  
  echo "Content-Type: text/html"
  echo ""
  echo "<HTML><HEAD>"
  echo "<TITLE>Freenet Operation Groups</TITLE>"
  echo "</HEAD><BODY><PRE>"
  /freenet/rootdir/menus/freenet/subs/groups.pl
  echo "</PRE><P></BODY></HTML>"
  

• document owner, location, and last modified time stored within HTML comments. (Aug 5, 1995 - RWM)
• properly handles 'order of operation' problem by assuming that an externally representable command (nr, telnet, gopher, lynx, etc) is the best choice of multiple %p or %P lines: (Aug 6, 1995 - RWM)
  

  %e nr before ACCESS - this would fail
  %p nr flora.comnet-www
  %P BBGUEST echo You must be a registered user to enter this group ; prtc
  
  %e ACCESS before NR - this would pass
  %P BBGUEST echo You must be a registered user to enter this group ; prtc
  %p nr flora.comnet-www
  

• server strips 'prtc' commands from '%p %P' lines before continuing processing. (Aug 8, 1995 - RWM)
• proper TITLE set for the document: (Aug 9, 1995 - RWM)
  Ian Says:
> The first line in <<< xxxx >>> is the title. If you don't find it,
> the first non-blank %l or %L line is possibly the title.
> Do your best. The style guide is a guide;
> it's not the Law...


  Problem: With Lynx it displays the title at the top right hand corner. Do we wish to 'strip' that text from the MENU, or have it displayed twice for the LYNX client. Under Netscape/etc it puts the title in the title bar, and having the text also in the BODY of the HTML file looks great.

• Created new $freeport'ip_dir variable that is prepended to the URL's within the 'information providers' file area, rather than having the '/freeport/rootdir/menus/...' that was previously used for the URL. (Nov 12, 1995 - RWM)
• Created new $freeport'badfreeport variable that contains the URL of where to send people when they click on the "Can't convert" link.(Nov 28, 1995 - RWM)
• Moved software into freeport.pl (Common routines) and two wrappers: freeport_cgi (For a CGI based translation) and freeport_gateway.pl (For the Plexus gateway) (Dec 4, 1995 - RWM)
• Updated freeport_cgi to now be used as file translator as well. It will accept a filename on the command line and will output the translation of that menu file to STDOUT. (Jan 26, 1996 - RWM)
• Fixed bugs with '%p bbtelnet' where the menu text and telnet address were flipped. Fixed bug where multiple gopher references in the same file would conflict (Variables not being cleared) (Jan 26, 1996 - RWM)
• Created first version of Batch translator (Jan 26, 1996 - RWM)
• Wrote modified mod_mime.c for Apache (1.0.3) which allowed for forcing the MIME type - modified BATCH translator to not add in the '.html' for the translated files. (March 27, 1996 - RWM)
• Patched freeport.pl to 'collapse' URL's that include './' or '../' directory references in them (See discussion in comnet-www group)
• Added option to automatically add in link to NCF home page on translated pages. [Done for Apache module version only]
• Added option to automatically add an 'up' link to translated pages (Use /freenet/rootdir/config/menu_attributes.dat) [Done for Apache module version only]

Last update: $Date: 1998/11/27 02:06:32 $