RE: [alma-sw-ssr] Draft v2.0 of Pipeline and Offline Requirements

["Tim Cornwell" <tcornwel@nrao.edu>]

Robert wrote:
> 
> I tried to reply to some of Tim's comments in order to clarify a few
> points. 
> 
> 
> Tim Cornwell wrote:
> > 0. A point concerning scope. AIPS++ is about 150 FTE-years. AIPS is
> > probably about the same. The ESO Data flow system is about 300 FTE-years
> > (I believe). I would guess from some communications that for the items
> > described in this requirements document, the ALMA computing division
> > has between 40 and 60 FTE-years (depending on how one counts various
> > things). I would counsel that you spend that effort wisely. I think the
> > current draft overspends by a large factor.
> 
> Remember that these requirements will be used as input to a re-use
> analysis, and that the ALMA FTE's should  be used only to help fill in
> the remaining gaps, and build a pipeline.

I took that into account. Even if you build on top of another package,
the current draft is too expensive. I know that the process of
requirements/reuse analysis/costing will reveal this but I think
a reality check now is possible and useful.
>  
> > 1. A general comment is that data reduction splits into strategy
> > and tactics. The tactics come from the basic physics but the
> > strategy comes from experience. I think the document is mostly
> > fine on tactics but is a little too specific about some strategies.
> > The items on the calibration pipeline seem to me to fit in this
> > category. For example, 2.1-R3 is a strategy that may or may not
> > work in all situations.
> 
> The main motivation here is to feed back the results to the dynamic
> scheduling and data acquisition processes. We believe e.g. that if these
> tests do not work, the data cannot be calibrated, and we switch to
> another less demanding activity. This is a first guess strategy based on
> experience with existing mm-wave arrays. 

That's not the only point. The design of C++ library + high level
scripting allows one to put detailed and well-known tactics into
the library (via e.g. a measurement model for a telescope), and
defer strategies for implementation in the scripting language.
There are packages and systems that don't have this property
and therefore it is worth specifying.

> 
> > 4. There are some prescriptive implementation details that should
> > be removed (e.g. 3.0-R7 "using the fastest algorithm", also
> > the Appendix of Barry Clark's input parameters).
> 
> For the quick look speed matters of course (as the name says).

The fastest algorithm may require excessive disk space or have
low precision or only powers of two or whatever.... The point is
that calling out "fastest" as being the most important factor
is not necessary.

Tim