Re: [alma-sw-ssr] Draft v2.0 of Pipeline and Offline Requirements

[Barry Clark <bclark@aoc.nrao.edu>]

> 
> The subject of the Section 2 on Pipeline Requirements is referred to as the
> "Pipeline".  This may be implemented as disparate
> tools or programs, or as separate packages provided by different groups,
> or as a single package, as long as it fulfills the requirements.
> 
> The subject of Section 3 on Offline Data Reduction Requirements is referred to
> as the "Package" or "Offline Package".  This may be implemented as disparate
> tools or programs, or as separate packages provided by different groups but
> integrated into a single suite, or a single package.
> 

Perhaps instead:  

The "Package" or "Offline Package" is a set of tools or programs, believed 
adequate for ALMA reductions, and used by ALMA staff for reductions upon
which the behavior of the system will be judged.  It may consist of packages
provided by different groups, with transitions provided to integrate them
into a single suite.  The requirements will state that the Package will be
available for installation on the observer's own computer systems.  The 
requirements on the Package are set forth in Section 3.

A "Pipeline" is a set of operations, implemented by the underlying Package,
which takes a concise description of the way these operations are to be
performed and accesses ALMA data, either from the ALMA archive or from 
local files, and produces a desired data product.  (For purposes of software
requirements, the alternate definitions as a machine or set of machines,
or as the supervisory process that invokes these operations are less useful.)
There are several Pipelines essential to the efficient operation of ALMA.

\bullet The Calibration Pipeline operates in quasi real time, looks at only
calibrator observations, and produces one or more of the following data 
products (depending on the type of observation and type of calibrator), and 
places the results in a location where they can be accessed both by the 
real-time system and by other reduction proceedures:  1) an antenna pointing 
offset for all antennas. 2).  Tsys for all antennas as a function of time.  
3.) Sideband ratios for all antennas.  4.)  Antenna based flux calibration
(TSYSJY) from a flux calibrator.  5.)  Antenna based bandpass calibration.
6.) Antenna based polarization leakage terms (with the usual indeterminate
offsets from a single observation).  7.) Antenna based IF phase differences
(from a strongly polarized calibrator).  8.) Antenna based phase calibration
(with noise and atmospheric rms).

\bullet The Science Pipeline will process most science data.  It's data product
is an image cube.  This product will in many cases be adequate to achieve the 
observer's science goals.  It may access ALMA data from several observing
sessions and even from observations not the observer's own.  It is intended
to produce the best image possible without the intervention of an expert
observer.  The Science Pipeline will include a data calibration phase, that
may, in fact, run somewhat asynchronously with the image making phase; this
should not be confused with the Calibration Pipeline above.

\bullet The Quick Look Pipeline will process data from only one observing
session, and will comprise a subset of the operations of the Science Pipeline.
It will be sufficiently limited in its processing to produce results in a
time short compared to the length of a typical observing session.  It's
data products (images) will usually be available while the session is still
in progress.

The requirements for these pipelines are set forth in Section 2.