The MDPP Installation and Startup Guide

All older distributions are now obsolete.

Requirements: VMS

The VMS libraries contain code compiled using the new FORTRAN compiler. This compiler is only supported on VMS versions 5.4 and above. To use MDPP linked using the libraries you must have the new Math RunTime library that ships with the V6.0 compiler installed (not the compiler itself: you may not have a license for it). To use the MDPP on lower versions of VMS will require full re-compilation of all sources.

Requirements: True64 UNIX (OSF/1)

OSF/1 uses shared libraries and these must be available for the pre-linked program to work. In particular, the compiled code support library, libota.so, must be available as well as the FORTRAN shared library libUfor.so. These libraries are available on the OSF/1 installation CD-ROM and should be loaded from it in the event that there are problems starting the program. Obviously, to re-compile or re-link the program, a FORTRAN compiler must be available (with it's license).

Requirements: MacOSX

FORTRAN is a licensed product for most UNIX platforms. For the Mac under OSX we have used the Absoft F90 compiler (use v9.2 or newer) which you will need to buy should you want to modify the code: MDPP will not compile with g77 and I doubt that there is any rationale to try to make it do so.

MacOSX is BSD UNIX, but it is hidden from the user in most routine use. To run MDPP you must "unmask" it by installing both the "BSD" option and X11 from the OSX installation CD. Since the Mac does not use X11 as its display you need to start the X server before you try to run MDPP and display an image: the icon is located by default in the "/Applications/Utilities" directory. When the X server starts it opens an "xterm" window. While it is not necessary, it makes things easier to run MDPP from the xterm rather than another terminal window. I have modified the definition for the xterm in the X11 "Customize Application Menu" to be
xterm -rightbar -sb +sk -sl 5000
This gives you a useful window that can be scrolled, which the default window can't. But with these modest actions MDPP behaves identically to other UNIXes and delivers excellent performance. Note that the MDPP image distributed on the web is built for the latest version of MacOSX on the G4 and will run on G5. It is easy to recompile for G5, but these images do not run on the G4 boxes.

Requirements: Other UNIX

I don't have access to other UNIX boxes, so I can't comment directly on the difficulty of getting MDPP running. I doubt that Linux would be a problem since the Absoft compilers run identically on OSX and Linux (and Windows too): I'd anticipate a clean compile and link on Linux from the sources. I'd also be surprised if a SUN posed a problem, since the starting point for the OSX version was the obsolete SUN port of MDPP and Macs have the same floating point and "endian" formats. If you want to try a port, contact me to be sure that the version of the source on-line is indeed the most up-to-date.

Requirements: Running MDPP on a remote machine

You can, of course, telnet or ssh into another machine (e.g. mdpp-server.uni.edu) and run MDPP using the machine you are logged in to (e.g a laptop) as an X server to view the images. If you do this you need to do two things:

(1) You must authorize the remote machine to put X-windows on the local machine (your laptop) using the xhost command on your laptop as follows:
my-laptop$ xhost mdpp-server.uni.edu

(2) Before you start MDPP on the remote machine you must direct the X11 windows on that machine to your laptop:
mdpp-server$ setenv DISPLAY my-laptop.uni.edu:0.0

Now you can start MDPP and when you display an image it will appear on your laptop.

There is a minor problem when using the X-server on a MacOSX box in that the server gets confused about the root window. This is easy to solve by ensuring that the xterm window is open on the desktop (rather than being in the dock, or deleted). Easiest, is to telnet or ssh to the remote machine from an xterm, which guarantees that X is awake and functioning when you need it to field incoming images.

The first task: Downloading and expanding the tarballs

The basic instructions are HERE. The distribution files should be unloaded using the tar utility, with the command '$VMSTAR -X' (VMS:decompress first) or 'gtar -xz' (UNIX). Issue this command from within the directory where you want to root the MDPP directory tree.

Defining Required Components and Decompressing the Archives

The distribution components will create a small directory-trees that may or may not contain compressed files. These files are of two types: BACKUP (.BCK) savesets and tar archives (.tar). If you are running UNIX only, you can safely delete any compressed BACKUP savesets. VMS users should not delete the tar archive since common files are stored in them: wait until you have identified all the pieces. Decompression of archives has been done with LZDCMP (for files like *.BCK-Z: VMS) and GZIP (for files like *.tar-gz). Yes we plan to elimiate the use of LZDCMP compression: gzip is better and is cross-platform. On UNIX systems rename the files to standard form: e.g. file.tar-gz to file.gtz before attempting to untar with "gtar". Once decompressed, use BACKUP or tar/VMSTAR to restore the directories.

Continuing the Installation

The next task is gaining access to the on-line documentation. Start a WWW browser (e.g. Safari, Netscape, IE ..) and open the Installation page. If you are opening it locally, use the file MPP:[HTMLDOC]INSTALLATION.HTML for VMS or $MPP/htmldoc/installation.html for UNIX. Note that this requires MPP to be defined to point to the location of the root of the MDPP files.

Reminder on Customization

The separate components of MDPP are bulky: delete pieces you don't want. Use TAILOR.COM to eliminate pieces you do not need (for VMS), or remove the directories that are not needed with a command like: 'rm -R ogen' (UNIX).

Step-by-Step Installation of the Software

1. IF AND ONLY IF you have a previous version of MDPP then: Rename the root directory [MDPP] of the old version to a new directory, e.g. [MDPPOLD]. The command will look like:

   REN DISK:[000000]MDPP.DIR DISK:[000000]MDPPOLD.DIR

   where "DISK:" is the name of your disk volume. You may need help from the System Manager to do this. This old directory can be deleted when the new version of MDP is running (remember to save the old [MDPP.LOCAL] area if you have made your own local changes to MDP).

2. Create a new directory using the command:

   $ cre/dir [.mdpp]

   Set your current directory to this new directory and copy each of the distribution components into it. Use GUNZIP to decompress the ".gtz" files then use VMSTAR to unload the contents.

3. The file "DISK:[MDPP]AA_README.1ST" is obsolete, but you may want to read it either directly on line, or print the file AA_README_1ST.PS (PostScript). Most of the information that used to be in that file is now here. But its still a good read!

4. Use LZDCMP and GZIP to decompress the files in the directory tree VMSTAR generates for you.

5. Use BACKUP and VMSTAR to re-create the directory-trees in each of the sub-directories.

6. Edit and then execute the file MDPP.COM in the [GENERAL] directory. This sets up the pointers to the directories and the symbols to access utility programs, etc. This is not an essential step at this point, but it may significantly simplify getting started. If you make a mistake you can alter MDPP.COM and then re-execute it. The key step at this point is to create the logical MPP: This is essential so as to perform the next step. You do need to customize the MDPP.COM file for your site. This file defines SYMBOLS and LOGICALS for various devices to allow printing of pictures, etc. The file MDPP.COM should contain sufficient information to permit the correct changes to be made. Remember to set the PL0: definition correctly if you want to use ReGIS plotting.

7. Use the command TAILOR.COM to eliminate MDPP components not needed at your site: e.g. Alpha, Ultrix or VAX/VMS code may not be required.

8. The BOOKREADER documentation (optional).
   (**WARNING** The BOOKREADER docs are obsolete and are nolonger updated or maintained. Please ensure that you can use the WWW-distributed documentation for future releases.) These can be installed using the command file MPP:[ON-LINE]MDPP-BOOK.COM. This command file adds the MDPP documentation to the system-wide documentation on the VMS machine. To install these on an on-going basis, execute the command procedure above in the SYSTARTUP file: ask the system manager for help. [WARNING: The logical MPP must be defined BEFORE executing the procedure or it will fail and may prematurely terminate the system startup job.]
   

9. NOTE that you will probably need to relink the MDPP executable task. However, BEFORE re-linking read the section on customizing MDPP to your site. Use the command file CONFIG.COM found in the directory [GENERAL] to create the command file that you will use to re-link.

MDPP Directory Structure

MDPP
 |
 +--- GENERAL	The VMS/UNIX programs/executables. 
 | 
 +--- LIB	The VMS/UNIX Libraries and some .COM files
 |     |---- RGB	The color tables
 |     |---- MENU	Some examples of user menus
 |     +---- PROC	Some examples of procedures
 | 
 +--- DRIVERS	The Driver libraries
 |     |---- DECW	The DECwindows driver: VMS
 |     |---- UNIX	The UNIX driver (MacOSX right now)
 | 
 +--- DOC		The text docs
 +--- HTMLDOC		The docs in HTML
 | 
 +--- EXAMPLES		Some examples.
	+---- HELICES
	|      +------- STRAIGHTEN
	+---- MOVIE
	+---- PLANE_LAYER_FILTERING
	+---- PLANE_LAYER_RECONSTRUCTION

Differences between VMS and UNIX

Obviously there will be small differences between the VMS and UNIX versions. Mostly, these are listed in the docs. Please look at the docs for RD, SV and IO for the file processing differences.

Getting Started with MDPP

.IS   OUTPUT  FOR  A  CRT (1=NO):> x	
.ENTER NAME OF OUTPUT FILE (A20):> out/O/I/S	
.ENTER NAME OF INPUT  FILE (A20):> inp

If the output device is not a CRT then enter `1'; otherwise enter a <cr>. The output file name is entered in response to the next prompt; the default is `SYS$OUTPUT:'. Output processing is controlled by the switches `/O', `/I' and `/S'. Using the `/I[NPUT_LOG]' switch causes an input log to be generated containing all commands and data entered to MDP. This logfile, called `INPUT.LOG', should be able to be re-submitted to MDP to re-create the current session. Note that it is only useful to use this switch if you are logging a terminal session. The `/O[UTPUT_LOG]' switch controls whether an output log, called `OUTPUT.LOG', is to be generated. There is no point in requesting an output log if a general output file is being created. If a general output file is request then the `/S[POOL]' switch controls the spooling of this output file. If the file is to be spooled then append `/S' to the file name (e.g. `MYFILE.OUT/S'). If just `/S' is entered then a default filename is assigned (MDPPSPL.DMP) and the output is spooled at the end of the run and the file deleted. The input file is to be entered; the default is `SYS$INPUT:'. If an input file name is entered input will come from that file. The default extension is `.INP'. If an input file is entered then MDPP will ask if MACRO processing is to be done (see the write-up for the MACRO and LOOP processors). The prompt is:-

.ENTER `1' FOR MACRO PROCESSING: m

Entering `1' will cause the MACRO processor to be invoked and an ex panded input file will be generated.

.FILES OPENED SUCCESSFULLY

indicates that processing, if requested, is complete and all further input will be from the designated input file. For the beginer, this will be the terminal.

This tedious input can be simplified considerably and the input all placed on a single line in a comma-separated list, as follows:-

$MDPT x,out,inp,sw (for VMS)

mcgcr0-14> mdp x,out,inp,sw (for UNIX)

As before, `x' is either "1" for the `dumb terminal' mode or blank for the SMG screen managed output., and `sw' is either "1" to activate the MACRO processor (only if you have an input file please) or blank to skip MACRO processing.

The program will now clear the screen and be ready to start work. The actual response will depend on the manner that it has been started.

The ":>" or "Command:>" prompts indicate that the program is ready to receive input from the terminal and the user, by now in a frenzy of excitment, will comply.

Use Of The MDPP

Commands are read in the FORTRAN format (A2,2I2). Spaces are therefore significant. The 2I2 allows two numbers to be read in which provide extra information for the commands: the user is advised to respect the formating as failure to do so will lead to disaster.

Description of the Interactive Commands

In some of the simpler subroutines errors are not explicitly indicated by an error message but the expected output is suppressed (very rare in this version of MDPP). All commands should terminate with a confirmatory message: if this message is missing and the console simply types the prompt asking for the next command, something has probably gone wrong. It's up to the bright intelligent user to spot his silly mistake.

Basic Program Structure & Philosophy

Data involved in processing steps is read from a permanent disk file into a disk or a core buffer area depending on its size, the availability of core in the program and the nature of the processing to be done. Once the data are in the program buffer (the so-called current file) the user can perform operations on these data using the MDPP's commands. At the end of each step (command) in the analysis the data is left stored in the buffer. When the user is finished, the modified data can be written back out to disk. Output is very flexible: the user can write out data after each and every step, after a small series of steps, at the end of the entire chain of processing operations, or not at all. (Note that this is different from some other packages which return the data to disk after each step. Some other packages retain the original data used at each step: MDPP does not, unless the data has been saved by the user explicitly). In the MDPP data read from a disk file is transformed by each command and then returned to the buffer, ready for the next step.

(Generally speaking it is not necessary for the user to pay particular attention to the particular type of buffer being used in a calculation. There are exceptions however when the arrays being used are `stacks' of pictures or data, and when data arrays are in core. This is sometimes important because some of the routines will work only on data held in core: usually however, these are output processes or those designed for use on processed data.)

To reiterate, analysis of the data is performed by a series of subroutines which are called in the order required by the user, the main program acting as a `shunting yard' passing the data between each of the subroutines which actually do the work. The order in which the subroutines are called is controlled by the user who generates a so-called `command list'. The command list consists of a series of two-letter commands, which specify the operation (subroutine) the user wishes to utilize, followed by a series of data lines, should these be needed.

The computational work done by each subroutine is called a job-step. Each job-step is, up to a point, independent of those steps preceding it and those which follow. The programs have been written to permit a reasonable amount of flexibility in organizing the analysis but the user should bear in mind that the order is not really variable if sensible results are to be obtained.

Memory Management: VMS

Memory Management: MacOSX

Image Output

Output from `DV' is directed to the terminal, or to a display device. The command causes a file to be output to the device designated by the line `Display type' in the file `MDPP.DFL' (see below for a description of this file). At NYU we set this to `1' which designates a video display device. If you have a video display you must have (or write) a driver for it which interfaces MDPP with the device. (See above.)

'DV 3' puts up a very crude picture on the screen of any terminal. `DV' also uses ReGIS to plot images on the screen of a VT125 or VT240 series terminal; `DV 1' is quite quick; `DV 2' is slow, but the quality is surprisingly good (these two programs were provided by Dr. Michael Rademacher in Albany, NY).

Interacting with the Display

Many programs (but not all) which require the cursor position to be marked inside the image window allow you to double-click inside the window (or hit <Cntl-E>, VAX only). This causes a sub-window to be created with an enlargement of the area within it. This is to allow the user to position the cursor to within a sample point in the image: this is generally impossible to do on the main display since the MDP images of large areas are displayed at a resolution too low to permit cursor positioning at the single pixel level. This sub-window automatically extinguishes itself if a mouse key is clicked, or the cursor is moved outside the displayed area. A subwindow is not displayed if the magnification of the original image is already sufficient to allow single-pixel positioning.

The Mouse Button Order

Plotting

Communications

Defaults (or: What am I going to do NOW!!?)

The MDPP is a complex program. The documentation is idiosyncratic and there are going to be many times when figuring out what to do will be well-nigh impossible. As an aid to the confused (and to make life easier for the cognoscenti) MDPP tries to have acceptable defaults available for all the inputs requested by the various commands. The correct thing to do when faced with a confusing situation, is to just hit <cr> and see what MDPP thinks the right answers are. Now obviously this is not going to work all the time. So if you are confused and you have not saved the last step, back off from the command by hitting <cntl-Z> (VMS) or <cntl-D> (UNIX) to exit to the command prompt, save the file, and then restart the command. Now you can play: if you get the input arguments wrong just re-read the file and try again. If the file is saved from the previous step, there is really nothing that can get seriously screwed up..

Getting HELP

Using the Examples

For the most part, MDPP deals best with the problems it was designed to tackle, EM images and structural biology. Some of the tools here are unique and have never been successfully reproduced on other integrated packages (the helix stuff is an example, shadowed particle processing is another). We are very happy with the 2-D filtering tools as well and the 3-D reconstructions from tilted views of 2-D crystals.

However, if you are looking for a package to do general image processing of color images, or morphing, etc. MDPP is not the tool for you. There are a few tools, but sophisticated stuff isn't there, and never will be, probably. If you want to slice and dice 3-D images at high speed, you need a SGI and something like AVS as your IP package, not MDPP. My guess is that the tools for analyzing radiological data are probably all there, but we do not have the experience or expertise to recommend MDPP in this application.

Getting started is tough. But the only way to come to grips with the programs is to use them. Play around! Read in one of the example files (e.g. DONNA.BMD, or MANDRIL.BMD), get maximums and minimums (MM), generate noise images (GN), display them on the workstation (DV), slice their values (SL), add noise to DONNA (SU), smooth the result (SO), compute a total image power (TP), renormalize (NR), draw some contours (PL), warp the image (WA), etc, etc... Look thought the "Menu of commands" and try some things you like the look of (maybe they will work [our fault], maybe not [your fault] :-). But don't get scared off and if you have a major problem, feel free to call or E-mail for help.

Finally, when you have built up a bit of confidence, look at the examples in the area MPP:[EXAMPLES]. This are contains complete sets of data and command lists to allow you to see the MDPP in action for several of the major problem areas where the MDPP works well. Don't screw up the files in that area!   Rather, copy them over to your own space and play with them. Change the command lists, use the commands interactively, fiddle. Report the bugs if you find any! Don't accept stuff you do not understand, and don't accept anomalies in the processing (that is how we have found some key errors).

Finally, good luck! We have fun making MDPP (for the most part that is) and think it is a really great product. Your use of it can make it better. Keep in touch!

The Mandatory Disclaimer In tiny print