Interface FlowTracer with Accelerator

They are:

With a binary interface capable of supporting multiple users. The jobs are injected directly into the memory of the Accelerator server, thus preserving safely the credentials (user, osgroup) of each job. This method bypasses the policy layers implemented as part of the nc run command, which may be a disadvantage in some cases.
With a scripted interface limited to just one user, namely the owner of the FlowTracer project. This mode is similar to the one used to interface to LSF or SGE. The advantage of this approach is that the same nc run interface is used to submit the jobs, so all jobs are processed by all policy layer implemented as part of nc run.

Using the Binary Interface

FlowTracer provides an optimized implementation in C++ for the interface to Accelerator.

As discussed in Connect to a Third Party Batch Processing System, a Batch Processing System agent (BPS agent) is a vovtasker that connects to your FlowTracer vovserver and dispatches jobs to a backend Batch Processing System. You can start a BPS agent vovtasker from the command line by using the -V option of the vovtasker command, or you can define a BPS agent to Accelerator in taskers.tcl so that the BPS agent will be started when you start the FlowTracer project.

Start a BPS Agent from Command Line

First, decide which Accelerator to use to send jobs to which Accelerator project. Usually the Accelerator queue is called 'vnc', but you can find the available ones by asking your administrator, or by using:

% vovproject list -all -l | grep vnc

The command line syntax to start a BPS agent is:

% vovtasker -V qname@host[:port]

where port is optional and can be omitted if the port number of the Accelerator server is the default one for the queue name.

For example, an Accelerator server is running with the default name vnc, and is running on host alpaca with the default port 6271. You can connect a BPS agent to the FlowTracer project and send jobs to this Accelerator server by:

% vovtasker -V vnc@alpaca                # or
% vovtasker -V vnc@alpaca:6271

You can use other options of vovtasker to specify other attributes of your BPS agent. Among all the attributes, the default value of "capacity" for BPS agent is 100 while default for normal tasker is equal to the number of tasker machine's CPUs. You can override this default using the -T option. Usually you want to have a large capacity, because you want to give the BPS the ability to schedule jobs in the most effective manner. Capacities of 200 or 300 are typical.

Define a BPS Agent in taskers.tcl

In the taskers.tcl configuration file, you use vtk_tasker_nc to define a BPS agent to Accelerator. The syntax for it can be found in vtk_tasker_define.

After you define the BPS agent in taskers.tcl, it will be started when you restart your FlowTracer project. You can start it without restarting your project by using the command

% vovtaskermgr start name-of-BPS-agent

Unless you give it a different name, the name of the BPS agent will be NC. See instructions for The taskers.tcl File for details.

Examples

Here is an example, where in the context of FlowTracer project test1@alpaca, we now start a BPS agent which interfaces to FlowTracer vnc@alpaca:

% vovtasker -V vnc@alpaca -a NC -r "se_license spice_license"

Here's what the tasker monitors look like:

Here is the architecture diagram in this case:

The above example is a FlowTracer project named test1@alpaca. There are two normal "direct" taskers cheetah and bison already connected to it. The Accelerator server vnc@alpaca has 3 taskers, alpaca, pluto and comet. A BPS agent named NC is started in the context of test1@alpaca, which connects it to Accelerator vnc@alpaca. The taskers of vnc@alpaca are represented as BPS taskers prefixed with the name of the BPS agent in test1@alpaca for easy monitoring and management.

Handling Resources

The server managing the FlowTracer project uses its resource maps to determine which vovtasker should receive each job. When a job requests a floating resource, for example a license, that request needs to be passed through the BPS agent to Accelerator, so the job will not be started when the license is unavailable.

There are several methods you can use so that the jobs will run correctly in Accelerator and be dispatched through the BPS agent.

If the number of resources is small, you can declare them using the -resources option of vtk_tasker_nc, or the -r option with the -V option of vovtasker on its command line.
When the jobs in your FlowTracer project need many different resources from Accelerator, you can use the ncconfig.tcl file with the -F option of vovtasker for the BPS agent.
You can start the BPS agent vovtasker
- with the vovtaskermgr command
- automatically with a liveness script
- manually from the command line

Here is an example of the interface configuration file between FlowTracer and Accelerator.

# Interface control file for NC dispatcher (-V) vovtasker for an FT project
#parray NCTASKER; # uncomment for troubleshooting

# Integer, controls debug messages
set NCTASKER(flag,debug) 0

# int, whether to show the farm tasker LEDs in the FT GUI
set NCTASKER(flag,show,taskers) 0

# BPS agent offers resources of this type from NC, e.g License:PrimeTime-SI
set NCTASKER(tasker,resources,resmap,types,export)  {License}
# set NCTASKER(tasker,resources,resmap,types,export) {Priority}

# These are offered by the BPS agent in addition to the exported res from NC
set NCTASKER(tasker,resources)       "NC linux sun7"

# set NCTASKER(job,resources,start) ""

# Resources to delete from jobs dispatched to NC
# set NCTASKER(job,resources,del)   ""

# Resources to add to jobs dispatched to NC
# set NCTASKER(job,resources,add)   ""

# set NCTASKER(job,group)           ""

You can place this file in any convenient directory, but it is recommended to place it in the .swd of the FlowTracer project. This directory must be accessible to the FlowTracer project anyway, and the live_start_nctasker.tcl script expects it to be there.

When using the interface file with BPS taskers defined in taskers.tcl there is a shortcut that causes the file to be searched for first in the .swd of the FT project, and then in $VOVDIR/local.

To setup your FlowTracer project so that an Accelerator BPS agent can be managed with the vovtaskermgr command, use the -nccfg option of vtk_tasker_nc, for example:

vtk_tasker_nc -name NCbps -nccfg path-to-ncconfig.tcl

Note: If you use the special value - (minus sign) for path-to-ncconfig.tcl, the search mechanism described above will be invoked. Whether by search or path specification, the config file must be found or the BPS vovtasker will not be started.

To manually start a BPS agent vovtasker from the command line with the interface configuration file, use:

% vovtasker -V qname@host[:port] -F path-to-ncconfig.tcl

To setup your FlowTracer project so that an Accelerator BPS agent will start automatically whenever jobs need to be run in your FlowTracer project, set up the ncconfig.tcl file in your FlowTracer project's .swd directory. This is the only place where the standard liveness script looks for the interface file.

Also, copy the example liveness script to the tasks subdirectory of your FlowTracer project's .swd configuration directory. You may need to make the tasks directory there.

%  cp $VOVDIR/etc/liveness/live_start_nctasker.tcl `vovserverdir -p tasks`

To test whether it is working, retrace some jobs from your FlowTracer project. You may need to modify the jobs so that they request the resource 'NC' that is offered by the BPS dispatcher. They should become SCHEDULED (light blue color). A new vovtasker should appear, then the job will become dispatched (cream color) and then RUNNING (yellow), and finally VALID or FAILED.

Check the resources offered by the BPS dispatcher tasker using the monitors, the browser, or by

% vovtaskermgr show -resources

The log file for the BPS dispatcher is in the usual place for tasker logs, in the directory .swd/logs/taskers/the-tasker-name.

Using the Scripted Interface

These taskers can be declared in the same way as the binary interface, except that you have to use the option -singleUser 1 in vtk_tasker_nc. The script $VOVDIR/etc/tasker_scripts/taskersVNC.tcl provides an interface to Accelerator.