Tobyfit linux install
From Tobyfit
Contents |
Tobyfit Linux install
The current release (R307 or later) adds support for running jobs on the localhost, but this set up is more complex than for Windows systems. This is due to the range of different compliers and MPI libraries for Linux along with the 32/64bit systems. If you already have grid access from your Linux system you do not need to enable the localhost option, and it defaults to being unavailable. The section on "Support for localhost on Linux" below gives some details. If you have any problems please email me.
Install with NGS grid access
Getting grid access
If you wish to use the grid enabled features of Tobyfit, i.e. the ability to use grid resources for high performance on large problems, you need a grid certificate to identify yourself and then obtain permission to access grid resources.
The detailed steps needed to get an e-Science certificate and then apply for a NGS account are described on the NGS website How-to-join. The first step is the application for an e-Science certificate which may take a day or two. An email will inform you when your certificate is available and how to load it into your browser (note you must use the same browser that you used to request the certificate).
Users at STFC Rutherford Appleton Laboratory can obtain an e-Science certificate and use it to access the local scarf cluster without needing NGS registration. Alternatively it is possible to use the "Single Sign On" style certificates which can be generated from an STFC federal ID and password. Users with access to other Globus enabled resources should get an appropriate certificate for themselves for that resource.
With a UK e-Science certificate you can apply for access to the hosts forming the National grid Service. Applications may take one to two weeks to be approved.
The tool Certificate Wizard is a convenient way of converting a UK e-Science certificate into the form most often used for grid access and setting up your machine to recognise e-Science certificates. This should be run on the machine that you requested your certificate from. It is best to first backup your certificate from the browser, as described in the How-to-join instructions above. You must use a short password (8 characters or less) for the certificate export, due to American export restrictions on the software, but you can (and should) use a better one later on - at the end of the wizard process there is the chance to reset the pass phrase to a longer value.
If a certificate is obtained on a machine other than the Linux host you wish to install Tobyfit on, then you can export the certificate from the browser on that machine and copy it to Linux host. You can then use the certificate wizard on the Linux host to create the .globus directory.
SRB access
NGS no longer support the Storage Resource Broker (SRB), and Tobyfit has been updated not to use this method for file transfer. Hence this is no longer required.
Installing Tobyfit GUI
The latest zip of the GUI is available from here - in the zip file tf_gui_distv2.0.zip. This version has been tested with Matlab V7.1 to R2009a and does not need SRB support.
To install the Tobyfit package the zip file should be unpacked into a temporary directory, e.g. "/tmp". Then run Matlab, version 7.1 or later, and go to the install directory and run the script "setup". The script first checks if the grid .globus directory have been created, if not only jobs using the local host can be run. A check is still made for SRB access, but this method is now deprecated.
The script will ask for an installation path (with the default of the user's home directory, "~/") and unpack all necessary Matlab files into this location using the path "e-science/Tobyfitproject", e.g. "~/e-science/TobyFitProject". You will be asked if you want to add this directory, which contains the startup command "tobyfit.m", to the default path. If you say "no" you will need to be in the given directory to start tobyfit. The static Java class path needs to be updated to work with Tobyfit and the setup script can either edit the system "classpath.txt" or create a copy of this file in the install directory. If a shared version of Matlab has been installed on the Linux host by the system administrator it is usual that a user cannot change the common classpath.txt. In the latter case Matlab must be started from the install directory since only then will it read the correct classpath file.
Installing Tobyfit server
Each grid resource that the user wishes to use must have a suitable version of the Tobyfit server code installed. Binary versions are available for the main NGS nodes, along with the RAL machines scarf and mountainview01. Details of how to do this are on the Tobyfit server installation page.
Support for localhost under Linux
Since Linux systems have a greater variety of compilers and supported MPI libraries than Windows, it is not practical to provide a single pre-complied library and executable of the server code in the installation. Hence it is necessary to download and build the server code on the Linux system after installing the GUI code if support for jobs on the localhost is required. If the user is happy to only use grid systems from Linux this section can be ignored.
Tobyfit server uses MPI for parallel processing and an MPI library must be installed to use.
On Linux [OpenMPI] is assumed to be the supported MPI library for the localhost option.
If this is not already available on the Linux host it should be installed and built.
On the Isis machine "mordred" a version of OpenMPI is already installed in /usr/local/bin and scripts should pick this up.
On the new Isis machine "melehan" a system version of OpenMPI is not currently installed, but a version is available in /home/rff93/openmpi.
While other MPI versions can be used, some editing of scripts will be necessary.
On either of these Isis machines, the easiest way to install the server code is to take a copy from ~rff93/e-science/TOBYFIT_V2.2,
e.g. cp -r ~rff93/e-science/TOBYFIT_V2.2 ~/e-science/. You still need to edit the Resources.properties file as mentioned
below.
Otherwise you will need to download and build the server code.
The Tobyfit server build has been tested with Intel Fortran versions 9 and 10 but the code should compile with any standard Fortran compiler that the MPI library is built to support. When building OpenMPI a default compiler is selected, accessed by the mpif90 command which should be added to the PATH if not already present.
The server code can be built in the same directory structure as the GUI, under ~/e-science/TOBYFIT_V2.2. Alternatively it can be built else where and a soft link made so that it appears as ~/e-science/TOBYFIT_V2.2. e.g. to build in place do:
cd ~/e-science ;
tar xzf /path/tobyfit_server.tgz ;
cd TOBYFIT_V2.2 ;
make
In addition to building the server code on the Linux system it is necessary to edit the file Resources.properties which on Linux should found in the directory ~/e-science/GaptkComputeToolbox/lib. In the section for the localhost options it is necessary to comment out the Windows defaults (using the "#" character) and uncomment the Linux values. The localhost option for Linux will not be offered until this change is made and the GUI restarted.
It may also be necessary to edit the shell script ~/e-science/TobyfitProject/bin/linux_mpi_wrapper.sh
to set the correct paths for the executable and library files on your system.
Testing the Install
Start Matlab in the directory in which Tobyfit was installed and run the command "tobyfit". If all is OK this should then offer a Window asking for your grid certificate passphrase. You can ignore the Java warning messages that appear in the Matlab console window and just enter your grid password/pass phrase which you used when saving the user certificate. A "proxy successfully created" window should then appear which you click OK on before the main GUI appears.
To run a simple test case, assuming that you have set up the localhost option:
- From the "File" menu, select "Load TFT"
- In the file browser that this gives, go into the "demo" directory and select "demo.tft".
- In the directory browser that is produced, just say "OK" to the highlighted "demo" directory.
- A text box will ask if the data should be sent to remote hosts. Say "yes" if you have access to remote Globus hosts.
- A final text box will ask if you want to load history information. Say "no".
- At this point you should see some data appear in the GUI. These are the numerical values and file names read from the TFT file. You can step through the separate tabs to see all the values.
- To test running a local job on the Windows machine, select the "simulate" tab and click on "Do simulate".
- A new Job submission window will appear and you should select "manual", "local" resource, 1 or 2 processors and them "OK".
- After a short pause an entry will appear in the job table of the main GUI with the Status "active".
- Press the "Query" button to update the status which should soon change to "completed".
- Select the results by clicking the "select" box on the new row. The details of the simulation should appear in the test area.
