> (MasatoshiOhishi)
> General Comments:
> 
> In the ITU (International Telecommunication Union), we discuss possible
> frequency allocations for Satellite Operators in the 40 GHz range and
> around 94 GHz.  I think we need to consider how to monitor and mitigate
> interferences due to space-to-earth direction transmissions from satellites.
> It is necessary to make real-time monitoring to avoid unwanted emissions
> from satellites.
> 

An addition requirement (6.4.1.-R3) has been added in that sense.

 \item {\red The Pipeline shall be able to detect interference,
e.g. due to communication satellites operating at ar near the observed
frequencies, flag the data accordingly, and trigger alarms if
necessary. \reqpri{3}}

> ---005--------------------------------------------------------------------------
> (RayPlante)
> Hi Robert,
> 
> Unfortunately, my first comment is a about requirement that is (I think)
> missing, so I feel compelled to break from the requested format
> in Part 1.  Part 2 refers to existing requirements.  I hope my discussion
> is not too lengthy to be helpful.  Your committee's response need not be
> as detailed.
> 
> 
> 1. User Access to Software
> 
> I feel an explicit requirement is necessary about the usefulness of
> data in the archive to end users (astronomers); something like,
> 
>   "Astronomers shall have [free?] access to software necessary for
>   processing all data products available from the archive."
> 
> This may seem obvious (hopefully, I didn't overlook such a statement). It
> is certainly implied by other requirements in sections 1, 3, and 4;
> however, I think it's important to spell this out as I think it affects
> how these other requirements are met (and the cost of doing so).  Some of
> my other comments below hinge on an implicit assumption of this nature.
> 
> You probably should go even further.  Here are some possible riders:
> 
>   *  "In particular, the astronomers should have access to all software
>      necessary for duplicating or redoing the [reversible] processing
>      carried out by the pipeline"
>   *  "...on their local computing platforms"
>   *  the data reduction scripts are locally runnable
> 
> To the extent that these are true, they should be expressed
> explicitly.  The extent to which they need not be true, that should be
> stated as well--for example,
> 
>   *  restriction: limited to data products in the Observational
>      Archive.
>   *  possible exceptions: Atmospheric Model, Quick Look operations
>   *  products made available by astronomers (e.g. final images)
>      resulting from local processing are exempt from this
>      requirement.
>   *  some projects may require high performance computing to perform
>      the processing; performance requirements (execution time, memory,
>      disk space, etc.) should not be considered when meeting this
>      requirement for such projects.
>

 You should look into the requirements for off-line data
processing. Look in particular OL-1.1-R5, OL-1.2-R8. 

The pipeline will also be runnable at places where the archives are
kept; these are known as "Regional Support Centers", and are foreseen by the
ALMA project.


> 
> ---007--------------------------------------------------------------------------
> (TimCornwell)
> Apologies for being late. I hope that you can consider my suggestions. I
> appreciate being asked to review these requirements. I found it helpful
> for our own work here on end-to-end processing of data from NRAO
> telescopes. I have a few global concerns:
> 
> -        The distinction between the various pipelines is insufficiently
> motivated.
 

We have tried to clarify the distinctions. The Science Calibration and
Science Imaging pipeline were merged into a single "Science
Pipeline". The real-time calibration and the quick-look pipeline have
obvious inter-relations, and could therefore be merged into a single
"real-time pipeline". However, we kept the distinction, in order to
emphasize which operations are necessary for the system running, and
which are only performed for monitoring. 

The distinction is more an architecture issue than a requirement
issue. In fact the Analysis group has taken the view of integrating
the (real-time) Calibration Pipeline in the Instrument Operation
Subsystem, not the Data Reduction subsystem.

> -        The goal of processing all standard modes should be qualified
> by the attachment of quality measures.
>
 

Accepted. An addtional requirement (6.5-R5) has been added for that purpose.


> -        There is a tendency to be overly prescriptive in describing how
> the reduction is to be done. I think that our best practices will evolve
> as ALMA comes online and the requirements should reflect this.
> 

We have added a number of "provisions" to make it clear that only
the current best-practices are described in this document. On another
hand, the ALMA pipeline should be developed during the coming 
years, and the SSR document should be as prescriptive as possible,
in order to be useful for the developers.

> 
> ---009--------------------------------------------------------------------------
> (WimBrouw)
>         Comments on Pipeline part of ALMA SSR and Use cases --  version 4
> 
> Introduction
> 
> Add in the introduction that the Science calibration and Imaging will be a
> sub-set of the off-line data reduction package on the one hand; and that
> the real-time calibration will be available in the off-line data procesing
> package.
>

This would imply that the Pipeline be built with the Off-Line package.
While this may certainly be true, it is felt that this is not a
requirement for ALMA software. The requirement is that all
operations done by the Pipeline can also be done off-line.
 
> Add that pipeline processing will have parallel streams, with a mixture of
> synchronous and asynchronous operation
> 

This is an implementation issue.

> Question (in principle for Use cases): will the measured WVR be archived? If
> not, calibration based on WVR should not be included in off-line package.
> 

Yes, the WVR data will be archived. See Req. 7.2-R2.

> ---012--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 26, 3.6.1 "NOte" --> "Note"
> 
> "Occurrence" is misspelled as "occurence" everywhere in this section
> 
> 3.6.1.1 "make results available to the dynamic scheduler" -- this assumes
> a certain division of the system into items that include a "dynamic scheduler."
> Since things like preliminary phase rms will be used to adjust target and
> calibrator dwell and cycle times, the requirement as expressed would not
> address this use. In general, it would be better if the authors indicated
> how they want quantities used, rather than to which (as yet undefined) "part"
> of the system they should be sent.
>

Accepted. We have replaced "Dynamic Scheduler" by "dynamic scheduling
system".  This part of the system is described in another part of the
SSR document; the list on required inputs to the dynamic scheduling
algorithm is given in Req. 4.0-R3. Some of those input parameters are
supplied by the Pipeline, which we have indicated in the Pipeline
requirements. This should be cross-checked with the dynamic scheduling
requirements.

> Are baseline and holography calibrations really "(quasi) real time"? Robert
> Lucas indicated to the software group that baseline calibrations might be
> deferrable to a time after scientific observations corresponding to the (new)
> antenna configuration had been completed--in some cases.

Accepted. Baseline and holographies are not quasi-real time operations
-- except possibly in some cases, as high-frequency work, where the
dynamic scheduling may have to take into account the baseline
quality. As suggested by another reviewer, we could consider such
operations as "plug-ins". The Requirements 6.3.4-R2 to R5 have been
rephrased accordingly.

> 
> Again, "make results available to the sequencer" doesn't say what these
> results are to be used for. "Sequencer" is not a meaningful concept at the
> requirements stage.
> 

Accepted. "Sequencer" has been replaced by the very general term
"real-time control system". Some parameters derived by the Pipeline
(e.g.  pointing offsets) shall actually be used by the control system
in order to update the instrument parameters.

> In the same vein, requiring Quick Look Operations to be done during and after
> an "observing session" doesn't give developers the information they need to
> make these operations useful. Who is to look at the results? When? What kinds
> of corrective actions should be made possible by these results?
>

The QL Pipeline is there to allow the ALMA staff to monitor all
observational/astronomical informations that can provide a valuable
information to detect any misbehaviour of the instrument. 

The results are also provided to the observer (PI) by means of a
web page (see 4.0-R9 and UC 4.5.2).

 
> ---013--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p.26 3.6.1.1  Real-time Calibration Operations
>    In Telescope / Array Calibrations, although I could be wrong, do we really
> need to reduce the holography data ? See also subsection 3.6.3.
>

See comment 12.
 
> ---014--------------------------------------------------------------------------
> (TimCornwell)
> Page 26, 3.6.1.1 Pipeline operations
> 
> The distinction between Science Calibration and Science Imaging is too
> fine to be worth building in at this level. While calibration and
> imaging used to be considered as separate items in a waterfall model, a
> more modern view is that the two are intertwined via the concept of
> self-calibration. 

Agreed. The two sections are now merged.

> I am similarly dubious about the distinction between
> real-time calibration and quick look operations. I'd recommend the
> distinction be between Real-time and Post-observing Processing.
> 

Cf comment 007. The difference here is in the motivation (real-time
calibration is for automatic detection of problems, quick look is for
humans), the data being processed (well known calibrators vs science
objects) ...

> ---015--------------------------------------------------------------------------
> (WimBrouw)
> p26 3.6.1.1: Most of the following details in this section are valid for all
> types of operations. Iso saying 'relates to interferometric', why not have
> an indicator to exclude the few items that are not valid for single dish
> or light-bucket operations.
> 

Agreed.

> ---016--------------------------------------------------------------------------
> (WimBrouw)
> p26 3.6.1.1 RT cal operations
> - Some of these are 'real' RT, i.e. they have to be done before observing can
> proceed, since errors will remove information (like focus; delay; offset
> pointing); others are not (holographics; baseline tests)

Agreed.

> - The Atmospheric model is RT; some of the astronomical ones could be
> (atmospheric calibration idf that is the one based on the atmospheric model);
> others are definitely not (bandpass calibration; prelimin phase and ampl cal)


> - Is the 'preliminary phase rms' purely based on atmospheric information? If
> not, what is its purpose for the dynamic scheduler (since the phase noise
> will depend on baseline; ampl; atmosphere, ..

The phase rms are computed from the calibrator measurements. They are
used to estimate the seeing, and are an input for the dynamic
scheduler.

> - Can it be indicated which one are 'await result RT', and which are not?

The list given in 3.6.1.1 is mainly to give a overall view of
each group of operations; the precise definitions are given in the 
requirement list.

> - Indicate that this processing should include, but not be limited to the
> list given

Agreed. 

> - Should results be made available to sequencer (and/or scheduler) or should
> they be archived, and be available through archive?
>

This is an implementation issue. 
 
> ---017--------------------------------------------------------------------------
> (WimBrouw)
> p26
> 3.6.1.1 QL operations
> - Occurrence: after observing session, and during observing session at tbd
> intervals
> - Monitoring-- done through archive access of cal data?
> - Quick calib -- apply should be before resample/integration; also what is the
> calibration done here -- oh, I see apply only; not do. Maybe some terminology
> to indicate active and passive use of the verb 'to calibrate' should be
> used. Maybe use here 'correction' rather than 'calibration'
> - Quick imaging: rather than 'no deconvolution'. I would think 'subset of
> data' is more important (specified beforehand e.g. for spectral line to
> integrate channels with expected result
> - Display tools: display 'information about' current observations. Display of
> monitoring information and non-imaged data (e.g. baseline-time amplitude
> images) and statistical extractions are more important than 'images'
> (especially for non-SNR=100 single sources)
> 

The list given in 3.6.1.1 is only to give a overall view of each group
of operations; the actual definitions are given in the requirement
list. In order to avoid ambiguities, the list in 3.6.1.1 has been
simplified.

> ---018--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 27, Science Imaging Operations
> 
> Accessing the archive for previous observations, producing and archiving
> deconvolved images are expensive to do "after completion of observing session."
> My understanding from the SSR discussions was that these would be done,
> *at most* (and this is what is implied by requirement 3.1-R13) 
> after breakpoints and (of course) when
>  the project (or at least observations of the target) is
> complete. This requirement is also in conflict with Use Case 4.5.3 on p. 73.

Accepted: 

After discussion in the Granada Meeting we have decided to separate
occasions when full science data processing is performed, from the
break points and the end of sessions. The PI will specify in the
Observing Tool when Monitor Points are requested: after each
scheduling block, after each end of session (provided a minimum
observing time is reached), or at break points only. The necessary
computing resources will be evaluated by the Observing Tool and this
evaluation will be available to the reviewers.



> 
> 3.6.1.3 It seems as though "instrumental" calibrations were called "telescope"
> calibrations in 3.6.1.1. The change in terminology is confusing.
> 

Agreed.

> Although bandpass calibrations do not require a time interpolation, and
> can therefore be "immediately derived and stored, to be applied to all
> following observations," 3.6.1.1 says it's "derived or applied" after
> completion of observing sessions. This seems inconsistent to me, and is
> another example of how the requirements can get confused when they contain
> implementation considerations instead of real requirements.

Agreed.

> 
> ---019--------------------------------------------------------------------------
> (TadafumiTakata)
> p27 3.6.1.1 Pipeline Operation
>            Science Imaging Operation
> Comment
> Accurate positional calibration may be needed in producing
> deconvolved images, which are most popular data for general
> archive users. They have to include coordinate information
> like FITS WCS headers which can be interpreted by image
> browsers to be provided to ALMA users etc.
> 
> (Please ignore this comment if it is already included in the
> pointing and/or instrument calibration before producing science
> images.)

Agreed.

> 
> ---020--------------------------------------------------------------------------
> (TimCornwell)
> Page 27, 3.6.1.3 Calibration
> 
> I don't think that one can draw a line between items requiring
> interpolation and those not. All items must benefit from interpolation
> (or modeling in time and space). Another important and related factor is
> quality control: one needs access to a time-series to allow discovery of
> errant values. The key point here is whether one must wait for all
> relevant values before making a prediction of calibration values at some
> point in time and space.
> 

The Science Pipeline should use the best technique available, including
checks of any time-variation of all calibration parameters. However, this
would make no sense for the Quick-Look pipeline, which should use the
simplest method.

> ---021--------------------------------------------------------------------------
> (WimBrouw)
> p27 3.6.1.1 Science cal operations (science correction(?))
> - Why after completion of session? Why not also require option during
> session. I could think of a program; and hence an observing session,
> observing many individual fields as snapshots for getting to know some
> parameters of a series of 'sources'. Run it in between
> - derive and/or apply: bandpass (not a curve according to other places but
> fixed; a session could have many bands and places in bands; and hence many
> bandpasses
> - what is 'final' phase and ampl calibration: cannot eb precise enough ijn
> pipeline with 'limited or no deconvolution and selfcal). Or is 'final' as far
> as the pipeline is concerned?
> - flux scale is that derived here?
>
 
The list given in 3.6.1.1 is only to give a overall view of each group
of operations; the actual definitions are given in the requirement
list. In order to avoid ambiguities, the list in 3.6.1.1 has been
simplified.

 
> ---022--------------------------------------------------------------------------
> (WimBrouw)
> p 27
> 3.6.1.1 Science imaging
> - should 'produce temperature-calibrated visibilities' be in previous step?
> (btw why day 'uv tables',is implementation
> - is 'deconvolved' true here? If so; there should be an extensive
> (self-)calibration step in scince calib operations to be able to derive that.
>

See comment 21.
 
> ---023--------------------------------------------------------------------------
> (WimBrouw)
> p27
> 3.6.1.2
> - astronomical 'source': a project can have many sources (list) or a large
> (mosaiced) field. Mention that source here is not the normal single, limited
> source.
> 

Rephrased.

> ---024--------------------------------------------------------------------------
> (WimBrouw)
> p27
> 3.6.1.3
> - instrumental: indicate which ones (as mentioned above) are part of the
> RT-wait set; and which ones are not
> - no time interpolation: I am not sure it is a good idea to give these
> examples as 'time-invariant' calibrations. There are quite a few schemes
> where it is easily possible to have to interpolate e.g. bandpass (filter
> poles can be very T sensitive; if receivers are going to apply diurnal
> Doppler); bandpass could also have a phase error. I would leave it
> open. Also: bandpass and pol are non time-critical:
> Also, indicate listsinclude but are not limited to)


See comment 20.


> 
> ---025--------------------------------------------------------------------------
> (WimBrouw)
> p28. Finally ... average of observed...' I.e. leave out 'the', and indicate
> that the average will be done outside the pipeline operations (there could
> be contaminated channels e.g.)
> 

The pipeline shall produce a pseudo-continuum measurement, using 
astronomers' inputs. The final average is to be done by the astronomers,
off-line.


> ---026--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 29, 3.6.2-R2 What does "data-driven" mean here?
> 

It means that the Pipeline should run automatically as soon as new
data is acquired,  
and that operations to be performed should be determined by the
nature and purpose of the data, as indicated by header information
associated with it.

> 3.6.2-R3 The meaning of, and need for, "templates" isn't clear. Isn't this
> more of a design issue for the people doing the observing preparation tools?

Agreed. The second part of the requirement has been removed.

> 
> 6.2-R4 I understand a "pipeline" as something that, once started, runs
> automatically. In contrast, "tools" are usually instruments for
> doing some task that one wants to direct interactively.
> "Automatic flagging tools" sounds like a contradiction to me.
> I think you mean that certain data should be flagged automatically.

Rephrased.

> 
> 6.2-R6 This requirement is unreasonable, in my opinion. You can't repeat a
> series of previous operations and you can't resort to a copy of the dataset
> at an intermediate state. "Sufficient recording..." then means that results
> of every step have to be saved -- but not as a "copy of the dataset". Maybe
> I misunderstand, but I think that a straightforward interpretation of what's
> written here will produce a very heavy and expensive system. I think the
> kinds of "steps" that can be reversed and redone should be spelled out in more
> detail.
>

This requirement has been removed.
 
> 6.2-R9 I don't understand. Doesn't the quantity to be calibrated determine
> whether it's baseline- or antenna-based? (Maybe this is a stupid question,
> but the Aperture Synthesis Summer School isn't until after the deadline for
> comments.)
> 

In some cases, antenna-based calibration may fail, in which case 
baseline-based calibration is to be used.

> ---027--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 29 3.6.2  Pipeline general requirements
> In 6.2-R4, the degree of interference should be added as a condition to
> flag data.
>

Agreed.

 
> ---028--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.29 6.2-R3
>      'through readable and comprehensible data reduction scripts'
>      Sounds good but I think there are additional points, namely: a)
>      if observing scripts exists it would be better to use the same
>      control language for both observing and reduction scripts, and b)
>      in order not to be too dependent on a specific pipeline engine
>      the script language must be independent of any specific
>      processing system (ref. Reduction Blocks for the VLT).  That is
>      the script should provide the flow control and specify the
>      reduction tasks to be executed while the engine just should
>      execute these tasks as best it can.  Further, there is a
>      verification issues (like for observing scripts) if users are
>      allowed to change them.
> 

The second part of the requirement has been removed, in order to
be less prescriptive.

> ---029--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.29 6.2-R6
>      'Sufficient recording ... shall be carried out so that any step
>      can be reversed' is a very general requirements.  Some processing
>      is difficult to reverse and often it is simpler to save some
>      intermediate results.  Recording all reduction steps is clearly
>      needed but already covered by 6.2-R5.
> 

The requirement has been removed.

> ---030--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.29 3.6.2 Pipeline General Requirements
>      I am missing some requirement on that the engine used for pipeline
>      processing should be well separated from the ALMA system i.e.  it
>      should be possible to replace the pipeline engine without any
>      significant change in the ALMA software.  Although this partly is
>      a design concern since one would not like to have the ALMA
>      software depend on an alien, uncontrolled package, it is also a
>      science concern.  One would like to enable anybody to contribute
>      useful pipeline modules to ALMA and not limiting them to software
>      written in a specific system.  Further, with a lifetime of
>      decades for ALMA it is likely that it will outlast one or several
>      pipeline engines.
> 

We believe this is a fair design concern, based on experience.

We propose to add the general requirement:

The choice of a specific data processing package to provide the main
pipeline functionality should not impose unnecessary constraints on
the rest of the ALMA software system, in particular on the manner in
which the data is stored and handled, as in the future new or better
functionalities may be provided by newly developed data processing
systems.


> ---031--------------------------------------------------------------------------
> (RayPlante)
> p. 29, 3.6.2, 6.2-R6:  (Reversability of processing)
> 
> I think the general aim of this requirement is necessary: we want the
> ability to remove or substitute any part of the processing.  In
> particular, we want to be able redo it with small modifications to the
> parameters.  However, the wording of this requirement concerns me when
> I consider its strict application to complex processing; some
> clarification may be helpful.
>

The requirement 6.2-R6 has been removed.
 
> This requirement places a corresponding requirement on the off-line
> software (assuming point 1 above), the data formats used, and on what
> products must be stored in the archive.  Should one be able to back up
> an arbitrary number of steps in processing ("with out resorting to a
> copy at an itermediate state")?  What constitutes a step?  Suppose
> target source has been self-calibrated with multiple loops; does
> reversability mean that we have to keep the generated gain table and
> image models for each loop?  Taken to its extreme, this requirement
> could be pretty costly:
> 
>   *  if all datasets required for step-wise reversability, then the
>      archive must organize and label these products in a way that they
>      make sense to the user.
>   *  the data formats must retain sufficient information for backing
>      up.
>   *  for every processing step implemented, its reversal must also be
>      implemented.  The cost is greater if this capability must be
>      available to the end astronomer (see 1 above).
> 
> If we only mean to apply this requirement to the extent already
> supported by current packages, we may be okay here.  If we only
> require needing to make one step backward at any given step in the
> chain, we might be okay.
> 
> The cost is of particular concern if it's an unnecessary one.  Thus,
> I'm curious as to the intent of the clause "without...resorting to a
> copy of the dataset at the intermediate step."  Redoing processing by
> going back to an intermediate product is often the easiest way to
> "back out" (as opposed to reversing multiple steps in turn).  If the
> processing script is available (which is required) and the assumption
> from 1. (above) is true, then this is straight forward.
> 
> This requirement could be clarified in the context of the definition
> of ranked data products in the archive.  For example:
> 
>    Level 0 -- raw data: has no/certain/all real-time calibratation
>       operations applied
>    Level 1 -- calibrated data: has science calibration applied
>    Level 2 -- deconvolve images:  has Science imaging operations
>       applied
>    Level 3 -- final, user-supplied (locally processed) data products
> 
> Then, say, if one starts with the data products at a given level and
> apply the scripts that produce the next level stepwise, it should be
> then possible step backwards at any point in the chain.
> 
> By the way, defining ranked data products will help the user
> understand what they need from the archive.  They can easily choose
> what level of processing they want to accept and know what products
> they, therefore, need to retrieve.
> 
> ---032--------------------------------------------------------------------------
> (RayPlante)
> p. 29, 3.6.2, 6.2-R8:
> "Sequencer" -- you probably need an entry in the Glossary for
> this.  I was a little unclear about what it does.  According
> to my pdf reader, this is the first use of this term in the document;
> although, its function may be made clear in the Use Cases (which I did
> not study as well).
> 

"Sequencer" has been replaced by the very general term "real-time
control system". 


> ---033--------------------------------------------------------------------------
> (TadafumiTakata)
> p29 6.2-R1.1
> Comment
>  Is there any need to have a feed-back (like result images and
> so on) from this operation ? If so, some tools for helping the
> process to reflect the result to ALMA archive should be necessary.
> 

The concern is that if one allows PIs to contribute the results
of those special projects into the Archive, then ALMA has no control
whatsoever over the quality of the data. It is understood that, to be
usable for archival research (VO), the quality of the data has to be
evaluated in a well defined way.

> ---034--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, Pipeline General Requirements
> 
> I think that the goal of being able to process all data from the array
> in standard modes is too ambitious, certainly so for a telescope yet to
> be built. For the EVLA, VLBA, and GBT pipelines, we are planning to
> attach a quality measure to pipeline results. This adds an important
> level of qualification. A division might be:
> 
>             Meets all quantified scientific goals
> 
>             Meets some quantified scientific goals
> 
>             Meets none of the quantified scientific goals
> 
>   A priority 2 requirement could be added that the pipeline must process
>   all data from all standard modes to the highest level of quality. The
> _ priority 1 requirement is to process all standard modes and attach a
>   quality measure.
> 

Excellent suggestion. A requirement (6.5-R5) has been added in that sense.

> The requirement that the pipeline not constitute a bottleneck is a fine
> sentiment but I'm not sure who it is directed to? - TAC, operations?

Computing engineers, I guess.

> 
> ---035--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, 6.2-R2:
> 
> Some default parameters (cellsize, field of view, calibration methods,
> etc) should come from a standards database that is under change control.

We believe that those parameters should be provided at proposal
preparation phase, or defaulted at this occasion (so that the user may
check their values if needed).

> 
> ---036--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, 6.2-R3:
> 
> The second sentence is an unnecessary implementation detail. For NRAO
> pipelines, we are generating scripts from production rules encapsulated
> as make files. This is arguably superior to templates. In any event,
> it's not necessary to state how the scripts will be generated.
> 

Agreed. The second sentence has been removed.

> ---037--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, 6.2-R6:
> 
> Redoing is not the same as reversing. I'd recommend removing the
> reversing part. To redo, one simply needs checkpoints. To reverse, one
> needs much more. I think this requirement is close to requiring the
> ability to undo operations, which is very expensive.
> 

The requirement 6.2-R6 has been removed.

> ---038--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, 6.2-R9:
> 
> I think this requirement is not necessary. It is true that antenna-based
> calibration is better than baseline calibration for effects that really
> are antenna-based. However, it almost certainly is true that physical
> modeling of antenna phases by e.g. time and space parameterized phase
> screen is superior to antenna-based calibration.  Perhaps the
> requirement should state that "Best practices" must be following in
> calibration?
> 

Requirement rephrased.


> ---039--------------------------------------------------------------------------
> (WimBrouw)
> p29 6.2-R1.1 This holds for science data; I would suggest that the pipeline
> always handles 'active calibrater' data
>

The calibration pipeline should handle these data in the standard
way (pointing, focusing, checking phase stability) in order to ensure
that the requested observing conditions are met.
 
> ---040--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R3: shall operate through 'automaticly generated ' readable ... Drop the
> second part (implementation)
>

Agreed.
 
> ---041--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R4: 'step' is undefined; say: the pipeline shall include automatic flagging
> of data. Do not use 'discard' (bypass? not use).
>

Requirement rephrased.
 
> ---042--------------------------------------------------------------------------
> (WimBrouw)
> p29
> add: R4.1: Flagging should be multi-level to enable selctive re-use of
> automatically flagged data; or the automatic flagging must be reversable in
> the off-line stage using identical algorithms
>

It is already mentioned in this requirement that the flagging must be
reversible at the off-line stage.
 
> ---043--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R5: is 'output' archiving (should be)?
> 

Yes.

> ---044--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R6: yes; but add : '.. reversed and redone, taking into account the order of
> operations and the fact that some of them are non-commutative, and have to be
> 'done and redone' ...'
> 

6.2-R6 has been removed, because it was felt it was not a requirement
for the pipeline operations.


> ---045--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R7 and R2 are mutually exclusive. Maybe R7 could be rephrased slightly?
>

Rephrased. 
 
> ---046--------------------------------------------------------------------------
> (WimBrouw)
> p29
> 6.2-R9: talking about Ampl and phase only?
> R9: could you give one or more examples were baseline calibration (of what)
> is required?
>

In some cases, antenna-based calibration may fail, in which case
baseline-based calibration should be used. Baseline-based calibration
is required only for effects that are baseline-based:
e.g. baseline based amplitude calibration is needed when
atmospheric decorrelation during the basic integration time is important.

 
> ---047--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 30, 3.6.3.1 "Astronomical Calibration: Atmospheric Model" -- this
> terminology is inconsistent with that of 3.6.1.1, where "Atmospheric Model"
> and "Astronomical Calibration" are listed as separate categories.
>

Title of 3.6.3.1 changed.
 
> ---048--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 30 3.6.3 Real-time Calibration Operations
>   In 6.3-R1 it says, "The real-time Calibration Operations shall be activated
> AFTER each observations."  However in page 26, in subsection 3.6.1.1,
> it says, "Real-time Calibration Operations occur (quasi) real time."
> I feel these are inconsistent.
>

They are not inconsistent, because the term "obervation" is used 
according to its definition given in the SSR document, p. 10: it
actually represents a short integration period.
 
> ---049--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.30 6.3-R1
>      '... after each observation' It sounds too strong for me.  I
>      would have expected that only after calibration observations
>      the calibration Operations are done. Normally, calibrations
>      would be done after each science observation which then would
>      trigger the calibration operation but the current requirement
>      is stronger.
> 

If the last observation is a science one, then the calibration
pipeline has nothing to do. If it is a calibration observation, then
several operations have to be performed. The requirement has been
modified for clarity.



> ---050--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.30 6.3.2-R2
>      'convert the raw data into temperatures, or, alternatively, store
>      the conversion factor' There seems no reason to give an
>      alternative -  either one or the other.  I would prefer the
>      latter since this would store the raw, acquired data with the
>      factor and not apply it.
>

Requirement removed.
 
> ---051--------------------------------------------------------------------------
> (RayPlante)
> p. 30, 3.6.3, 6.3-R2:
> "Whenever the results ... allows to identify" ==>
>      "Whenever the results ... allow one to identify"
> 
> ---052--------------------------------------------------------------------------
> (TadafumiTakata)
> p30 6.3.1-R3
> Comment
>  The resultant parameters of model calculation should be referred
> by users (including astronomers) via "The Data Extractor Tool" or so
> for evaluating how the data are affected by this parameters.
> (I think it is already included in enviromental information extraction.)
>

The results of the atmospheric model are to be archived, and should 
thus be accessible later on.
 
> ---053--------------------------------------------------------------------------
> (TimCornwell)
> Page 30, 6.3-R2:
> 
>   I think this should be priority 2. It's going to be very hard to do from
>   the real-time calibration. An equivalent goal for the post-observing
> _ processing is possible and should be priority 1.
>

Agreed.


> ---054--------------------------------------------------------------------------
> (TimCornwell)
> Page 30, 6.3.2-R3:
> 
> This seems too prescriptive. What is described is the current best
> practice for mm arrays. This may not be the best practice for ALMA. Does
> one want to bind ALMA to work this way?
>

Requirement rephrased.
 
> ---055--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3-R2 '...after?? ..' RT calib is necessary before some data can be
> taken (atmospheric model). Certainly an observing session should also end
> with one.
> If you do flag; multi-level flagging is even more important.
>


For flagging: see requirement 2.3-R2
 
> ---056--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3.1-R1 ...The prediction will be based on measured atmospheric data,
>     including, but not limited to:
>

Rephrased.

 
> ---057--------------------------------------------------------------------------
> (WimBrouw)
> p30
> R1: what is 'line-of-sight'?: on a per antenna basis (what I would assume,
> certainly for higher frequencies and longer available baselines); and does it
> assume some isotropy and/or weighting over the HPBW FOV?
>

Agreed: this is done for each antenna independently (though some
consistency between neighbouring antenna could be checked). There is
no planned method to measure variations of atmospheric properties
inside the antenna primary beam (may be infrared sensors could be
used?). However most of the absorption occurs in the low atmospheric
layers where all line of sights inside the primary beam see the
atmosphere averaged over the same near-field antenna pattern.
 
> ---058--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3.1-R3 already covered in R1? If not, what is different?
>

R1 is a general requirement that a powerful atmospheric modelling
be available. R2 and R3 are indicating two crucial uses of the
atmopheric model.
 
> ---059--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3.2-R1: 'store the results'? or 'archive the results'
> 

The verb 'archived' may suggest that the Pipeline writes the
results listed in this requirement directly in the Archive. 
This is actually an implementation issue. What matters here
is that those quantities are available for further Pipeline
operations -- they could thus be written in the Archive at 
a latter stage, together with several other values. We
therefore prefer the term 'stored', which is more general.

> ---060--------------------------------------------------------------------------
> (WimBrouw)
> p30
> R1.*: a time-constant indication is necessary here. It is a large variety of
> items (and again the list ois probably not complete over life-time of
> instrument). I think 1 and 3 are per observation normally (or even
> longer). R2 could be fast varying?? (probably depend on length of observation
> as well); R4: is that in bore-direction (is really long-term to determine
> correctly ). The polarised antena pattern is something measured annually
> maybe.
> 

The software should be able to reduce all those observations. The
frequency at which such calibrations are performed is an opertaion
issue, which will largely depend on the observing modes, and on the
expertise acquired using the ALMA. Hence, we prefer not to give such
timescales in the software requirements. 

> ---061--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3.2-R2: alternative is incorrect. Tsys should always be stored
>     (=archived). The alternative is the deferment of the conversion, not the
>     storage.

Agreed. This requirement has been removed. 

> 
> ---062--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 31, 6.3.2-R3.4 "...passed to the Dynamic Scheduler..." Again, it is
> difficult to understand how this data is to be used. Scheduling decisions are
> typically made on the timescale of an SB (nominally 15 minutes-1/2 hour, in
> order to take account also of the time needed to bring a new receiver band
> to a ready state). So what's needed here? Averages? Instantaneous values?
> Predictions of how these values will evolve over the next 1/2 hour?
> 

Cf. comment 12.

> ---063--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 31, 6.3.4-R2, How are the baseline calibrations to be used by the
> "Sequencer"?
>

The "sequencer" ( = the real-time control system, cf comment 12) shall
use the antenna locations to compute the delays to be applied on each
baseline by the correlator.
 
> ---064--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 31 3.6.3.4 Telescope / Array Calibration
>   In 6.3.4-R4, do we really need to reduce the holography data real-time ?
> 

No, not real time. Cf comment 12.

> ---065--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.31 6.3.3-R1
>      '... and pass the results to' For me 'pass' suggests something
>      active i.e. the pipeline explicitly sends the information.
>      I would prefer 'made available' only.
> 

Agreed.


> ---066--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.31 6.3.3-R2
>      Same comment as for 6.3.2-R2
> 

Same answer.

> ---067--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.31 6.3.4-R1
>      'must be passed or ...' Same comment as for 6.3.3-R1
>

Agreed.
 
> ---068--------------------------------------------------------------------------
> (TimCornwell)
> Page 31, 6.3.4-R2:
> 
> "baselines" is jargon. I'd change this to "antenna locations".
>

Agreed.

 
> ---069--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.2-R3.3 : How can you have different parameters for the correction
> that is done on a fast basis before integration over samples?
>

The idea is to choose the best conversion coefficient when the
observation has been repeated using a few values. This may not be necessary.
 
> ---070--------------------------------------------------------------------------
> (WimBrouw)
> p31
> R3.4: what do you gain by doing this operation per baseline??
>

It is useful to do amplitude calibration per baseline in case of
amplitude decorrelation.

 
> ---071--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.3: Drop; or indicate again the earlier comment
>

6.3.3-R2 is removed.
 
> ---072--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.4-R1: why explicitly made available to sequencer (earlier comment). Let
>     sequencer determine what it needs: just archive/log it.
>

Cf comment 12 on dynamic scheduling. 
 
> ---073--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.4-R1.1: indicate (or in a more general comment) if it handles on offset
>     pointing here (as I would assume).
>

Rephrased.
 
> ---074--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.4-R4: should piepline handle this? Why?
> Mybe make it more generic:
> ' The pipeline should be able to accomodate plug-ins for the handling of
> special observations like holography; absolute pointing model generations;
> baseline determination etc'
>

Excellent suggestion. Requirement rephrased. 
 
> ---075--------------------------------------------------------------------------
> (JoeSchwarz)
> p.32, 6.3.4-R5 Is the derivation of the primary beam properties from planets,
> etc., really a "real-time" operation?
>

No, just like holography or absolute pointing models, these are
telescope calibration operations to be performed regularly, but not
to be reduced in "real-time". The requirements have been rephrased
for clarity.
 
> ---076--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 32, 6.3.4-R6 "...passed or made available to the Sequencer..." Once again,
> we need to know how these results are to be used, and on what timescale. Even
> if we accept that there will be a "Sequencer" in the system, we have no
> "Sequencer Requirements" chapter (and I hope we don't write one!), so we need
> to know *what* is to be done with pointing, focus, skydip, etc. Are they to
> be used to correct the control system's pointing model so that the antenna
> points where it's supposed to, or rather to correct the data already taken,
> or both...?
> 

Cf comment 12.

> ---077--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.32 6.3.4-R6
>      'must be passed or ...' Same comment as for 6.3.3-R1
> 

Agreed.

> ---078--------------------------------------------------------------------------
> (WimBrouw)
> p32
> 6.3.4-R5: drop the 'and aperture efficiency'
>     Add 'squint' to the list; or jusdt make it 'generic beam parameters like
>     ...'
>     Should this also be a plug-in? I assume this is not done on a daily
>     basis.
>

Requirement rephrased.

 
> ---079--------------------------------------------------------------------------
> (WimBrouw)
> p32
> R6: drop
>

Requirement rephrased.

 
> ---080--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 33, 6.4-R2 This seems too vague to me. How much data should be made
> available to the PI's over the Internet? When? In near real-time? Suppose the
> PI's aren't available or are asleep? It's entirely unclear how this data is
> to be used, e.g., whether we're talking about letting the PI play operator
> from his/her institute in Europe or America or just check on what an image
> looks like from time to time...
>

This was  introduced last year after the previous review.  The way it
should work is explained in 4.0-R9 and Use Case 4.5.2.
 
> ---081--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.33 6.4-R1
>      'shall be activated automatically after each ...' That is after
>      each observation which may be too much.  I would weaken the
>      statement and not make it mandatory but configurable.
>

Requirement rephrased.
 
> ---082--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.33 6.4-R2
>      '..., via the Internet' It sounds nice but it may give a
>      bandwidth problem if lots of people tries to get Quick Look
>      data like images.
>

Cf comment 80.
 
> ---083--------------------------------------------------------------------------
> (WimBrouw)
> p33:
>  6.4.1-R3.4: integrated? Since the seesion will contain 'simple scans' in
> frequency and/or position (mosaicing..) difficult to define exactly
> here. Just say 'indication of above over seesion' or so
> 

Rephrased.

> r3.5: noise per pointing?
> 

Added R3.6: for mosaics, it should be possible to monitor all quantities 
(R3.1 to R3.5) per pointing center

> ---084--------------------------------------------------------------------------
> (WimBrouw)
> p33
> 3.6.4.2/3: See it as an indicator of what should be done at the
> minimum. A good list. Maybe add to R3: 'TAKING INTO ACCOUNT FLAGGING'
>

It is implicit in the definition of 'flagging' that all data affected
are not taken into account in subsequent operations, unless
specifically mentioned.

 
> ---085--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 34, 6.4.2-R4 "Mosaic and self-calibration projects shall be supported."
> We need much more information than this to understand this requirement, and
> hence to know whether we have fulfilled it! What kind of Quick Look Operations
> are appropriate here?
> 

Requirement rephrased.

> ---086--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 34 3.6.4.3 Data Processing: Single Dish Data
>   In 6.4.3-R1.1, "on/off " could be replaced by "position switch".
>

Agreed.

 
> ---087--------------------------------------------------------------------------
> (TimCornwell)
> Page 34, 6.4.2-R3:
> 
> The mention of the Fourier transform is too prescriptive, and is
> unnecessary. It is too prescriptive because it is entirely possible that
> we might end up using some efficient linear algebra technique to make
> images quickly. It would be better to require something like along the
> lines that any image be available in a time comparable to a typical
> observing block.
>

Agreed. This requirement is rewritten accordingly.

 
> ---088--------------------------------------------------------------------------
> (TimCornwell)
> Page 34, 6.4.2-R4:
> 
> More explanation is needed: does the Quick Look pipeline just process an
> observing block or all relevant observing blocks? E.g. in a mosaicing
> experiment, does the QL pipeline just image the last patch of the sky
> observed? This is also a problem for self-calibration.

The QL pipeline deals with all data acquired during the current
session. For a mosaic, it should probably use a quick-and-dirty method
to produce an image, the best processing being deferred to the Science
Pipeline and/or to the off-line stage.

> 
> ---089--------------------------------------------------------------------------
> (TimCornwell)
> Page 34, 6.4.2-R5.
> 
> This is too prescriptive and probably wrong: one is better off doing
> this in the visibility plane. I'd make this a best practice requirement.
> 

Agreed. See comment 91.

> ---090--------------------------------------------------------------------------
> (WimBrouw)
> P34
> 6.4.2-R4: mosaic yes; self-calibration not as stated (see R3; and many
> earlier remarks about 'no or limited'. Should be limited to make it a
> non-bottleneck
> 

Agreed.

> ---091--------------------------------------------------------------------------
> (WimBrouw)
> P34
> R5: Compare to 'clean-beam'?? I do not get this at all. Maybe state: For
> ...the pipeline shall use the data to produce an estimate of the seeing.'
> (and pointing??)
> 

The idea was to check the size of the source with that of the clean
beam, to check the seeing effects. This requirement has been rephrased 
to be less prescriptive: the pipeline shall be able to make any valuable 
test on the data taken on a point-like source.

> ---092--------------------------------------------------------------------------
> (WimBrouw)
> p34
>  6.3.4-R1: a question: no statement about 'supported modes' is made for
> synthesis. Switching, OTF etc could be there as well. Should be added
> somewhere?
> 

There are actually less observing modes for aperture synthesis than
with a single-dish antenna: single-field imaging, mosaicing, OTF
mosaicing, snapshots. All modes can actually be calibrated in a
similar way, the imaging processing being different. Requirements
6.4.2-R3 and R4 have been modified to make it more explicit.


> ---093--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 36, 6.5-R1, R2 The term "session" is used too imprecisely here. Isn't it
> possible that some data preceding and following a session will be relevant?
> If, for example, a baseline calibration was done by the project that was
> executing 5 minutes before the current one, do we really want to repeat it?
> I imagine that there is other calibration data coming from outside the
> "session" that could be useful, too.
>

Several calibrations are to be done by the ALMA staff, and results
used by all subsequent projects. This includes baseline, pointing model,
primary beam measurements, etc. The results of such observations are
to be used by the real-time control system, to update antenna
parameters.

For 6.5-R1.1/6.5-R1.2, 6.5-R2, only very recent data is probably usable
(same receiver tuning needed). 
 
> I would also change "shall find in the Archive all data..." to "shall use all
> data...". Where this data comes from (whether it's in the Archive, or cached
> in memory, or whatever) is irrelevant from a requirements point of view.
> 

Agreed.


> ---094--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 36 3.6.5.2 Single Dish Data
>   In 6.5.2-R1.1, "on/off " could be replaced by "position switch".
>

Agreed.
 
> ---095--------------------------------------------------------------------------
> (TimCornwell)
> Page 36, 6.5-R2:
> 
> Can and should the Science Calibration Pipeline make use of historical
> observations which are not part of the project?
>


Yes: I think all observations of known sources identified as
calibration observations should be usable by all projects. In case a
particular project is observing a famous calibrator e.g. NRAO150 to
search for specific lines at high frequency resolution, then these
observations should be identified as target observations and will not
be considered by the system as calibrations.

I think thge main interest is for flux monitoring of calibrators,
anyway, and that should be handled by an observatory project using all
calibrations, not by each project directly.
 
> ---096--------------------------------------------------------------------------
> (WimBrouw)
> p36
> 6.5.1-R1: The way the bandpass and time-variations are given here does
> not indicate that the requirement could be for multiple bandpasses (depending
> on type of session) and time-ferquency variations.
>

Agreed ther may be several bandpass calibrations in a session.
 
> ---097--------------------------------------------------------------------------
> (WimBrouw)
> p36
> R2: should the pipeline 'observe' such a source if none available??
>

No. It is the responsibility of ALMA to make sure that the observing
modes are consistent: The observing procedure should be intelligent
enough to ensure that a project is not done without checking that e.g.
flux and a bandpass calibration are performed. The Pipeline will not
go back and observe.
 
> ---098--------------------------------------------------------------------------
> (WimBrouw)
> p36
> R3: why not drop R3.1 and make it just amplitude and phase corrected for the
> appropriate frequency ('scaling' could be to simple depending on how the
> delay is done as a mixture of time-delay and phase rotation??
> 

Rephrased.

> ---099--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6-R1 "subproject" is not defined anywhere in this document.
> 

Changed to 'project'.

> ---100--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6-R4 Whether the Internet is an appropriate delivery medium for
> Science Imaging results regardless of their size seems to me debatable. If
> a PI has been waiting for six months for a project to complete (because,
> for example, it requires different antenna configurations), why can't he/she
> wait a few more days for the delivery of a DVD? I think that this decision
> should be made in view of a) how urgent the project is (a basis for coordinated
> or follow-up observations?); b) how much data is involved (Alma will produce
> 180 Tb/year, so it's not impossible that we might occasionally have datasets
> that are significant fractions of a Terabyte); and c) what the real capacity,
> cost and reliability of the Internet is in the Alma epoch.
>

Agreed. We have rephrased the requirement, the data should be in
the archive and accessible to the PI (in fact I do not think we have
required that the data are `delivered' to the PI).
 
> ---101--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6.1-R1, R2 "...find in the Archive..." Again, where it comes from
> isn't relevant.
>

Agreed.

 
> ---102--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6.1-R3, R4 If the flux scales are different, what is to be done?
> What are the possible consequences of "direct comparison of the redundant
> data"?
>

In both cases, if the data are not compatible, the quality flag should
be down.

 
> ---103--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6.1-R7 I suggest that having "several [deconvolution] algorithms
> running in parallel" is better done as an offline, rather than an online
> task. If it *must* be done as part of the production pipeline processing,
> I don't see how it can be met as a "priority 1" requirement, which, as Page
> 12 of this document defines it, means:
> 
> "Must be there for Interim Science period, when the system is commissioned
> to produce meaningful science results."
> 

Requirement rephrased: having a deconvolution algorithm available is
in Priority 1. Having several algorithms running in parallel is in
priority 3.

> ---104--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 38 3.6.6.1 Interferometric Data
>   In 6.6.1-R7, it would be worthy to add a new CLEAN algorithm,
> the Wavelet-CLEAN, developed by Japanese group. And it would be
> very important how we judge the image quality.  Which is the best ???
>

We should add this, shouldn't we? but not be restrictive to a fixed
list of algotithms.
 
> ---105--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.38 6.6-R4
>      '..., via the Internet' Same comment as for 6.4-R2

see comment 100.

> 
> ---106--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 3.6.6 Science Imaging Operations
> 
> There is a potential problem in all pipeline processed images: the
> provenance of each scientific result must be knowable and
> straightforward. If a virtual observatory is to make use of ALMA
> results, there must be an ALMA standard product that was produced in
> some standard way without strange choices for e.g. cellsize, field of
> view made by the observer. By this logic, one is forced to produce at
> least two results: the "standard product", and that required by the
> observer. In many cases, the observer may just ask for the standard
> product. This whole aspect of the scientific imaging pipeline must be
> clarified. It affects a large number of the requirements following. My
> specific recommendation would be that two products are produced: the
> standard product defined by ALMA, and the observer's product, defined by
> the observer.
> 

The idea of having two products is reasonable, provided the
non-standard products are not required too often for large images (in
order not to increase the archive size needlessly. Note that I have no
idea how to define the standard product in terms of frequency
sampling: using the whole available correlator output at all times
would boots up the data rate! Here I think we have to trust the
observer, as well as with the mapped area. So what remains are the
cell size and the mapping algorithm. We have control over those as we
provide them in the pipeline.

My guess is that we should take your option whenever the user-provided
parameters (e.g. cell size, algorithm) are not regarded as `safe'
(e.g. may degrade the fidelity more that we like) by the pipeline. 

> Another question that must be resolved is whether results are processed
> only before insertion into the archive or also on exit from the archive
> (e.g. triggered if best practices have changed). This is a hard question
> to answer. This is discussed in requirements below but I think it needs
> to be thought through a bit more.

I think we should have this possibility. This is mentioned in the
Introduction and in the Archive section.

> 
> Finally, is the observer free to use the pipeline repeatedly or is the
> processing limited to that specified in the observing setup? If the
> former, how is time allocated? I think the latter is therefore
> preferred.

These are operational issues. The idea was that at regional centers
the pipeline may be run to reprocess data to the current standards of
data reduction. Experience seems to show that reprocessing data one
year after observations can be done at a fraction of the time (and
cost) of the first processing. But I agree this may be a problem for
large projects.  


> 
> ---107--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 6.6-R2
> 
> In many cases, the final product could be a linear mosaic instead of a
> deconvolved image. I'd therefore remove the second sentence.
> 

In all cases, a deconvolution will be applied to the data, so we prefer
keeping the current phrasing.

> ---108--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 6.6-R3
> 
> Can relevant non-proprietary data from other projects be included?
> 

This is a policy decision obviously. My position would be yes,
provided that the availability of such data was checked before proposing;
then the reviewers and/or project management may decide... 

> ---109--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 6.6.1-R3
> 
> Phase centers and polarization frames must also be checked.
>

This is included in the general term "instrument setup and properties".
 
> ---110--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 6.6.1-R6, 6.6.1-R7.
> 
> If ALMA works as well as expected, the extra images requested here will
> be unnecessary: for example, the various weightings should be less
> divergent than for the VLA. Similarly for the deconvolved images. Also
> there is a combinatorial explosion possible (weighting x deconvolution x
> other parameters). I'd recommend that the Standard Product be just one
> image.
>

Yes, but the pipeline should have the possibility, if explicitely 
required, to use different weighting schemes.
 
> ---111--------------------------------------------------------------------------
> (WimBrouw)
> p38
> 6.6.1-R5: There is no 'continuum measurement'
> 

Requirement rephrased: the Pipeline shall extract visibilities of a
PSEUDO-continuum, corresponding to the sum of the spectral channels.

> ---112--------------------------------------------------------------------------
> (WimBrouw)
> p38
> R2: 'compatible'?
>

Yes
 
> ---113--------------------------------------------------------------------------
> (WimBrouw)
> p38
> R5: 'appropriate'? I would think that the data is used to produce the required
> image cube; not the other way around.

Yes 

> 
> ---114--------------------------------------------------------------------------
> (TimCornwell)
> Page 39, 6.6.1-R9
> 
> This is really a call for more development of automated methods for
> identifying and removing continuum. In the absence of any known method,
> one cannot require that it be used!
> 

Requirement rephrased.

> ---115--------------------------------------------------------------------------
> (WimBrouw)
> p39
> 6.6.1-R7.3: Add: 'model fitting and data subtraction'

Yes

> 
> ---116--------------------------------------------------------------------------
> (WimBrouw)
> p39
> R8: do not specify both the domains. Why? I would think that image-plane
> subtraction is a non-option in general.
> 

Yes

From - Mon Jul  8 17:02:05 2002
Return-Path: <gueth@iram.fr>
Received: from iram.fr (pctcp96.iram.fr [193.48.252.226])
          by iraux2.iram.fr (8.9.1/jtpda-5.3.2) with ESMTP id QAA12367
          for <lucas@iram.fr>; Mon, 8 Jul 2002 16:42:59 +0200 (METDST)
Sender: gueth@iram.fr
Message-ID: <3D29A4C0.B7BD90A8@iram.fr>
Date: Mon, 08 Jul 2002 16:42:08 +0200
From: Frederic Gueth <gueth@iram.fr>
Organization: IRAM
X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.4.2-2 i686)
X-Accept-Language: en
MIME-Version: 1.0
To: Robert Lucas <lucas@iram.fr>
Subject: comments pipeline
Content-Type: multipart/mixed;
 boundary="------------0A08B4311B6BF3FC3D30D931"
X-Mozilla-Status: 8001
X-Mozilla-Status2: 00000000
X-UIDL: 8d0884d6f70e1a279df17307162f6505

This is a multi-part message in MIME format.
--------------0A08B4311B6BF3FC3D30D931
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

 
--------------0A08B4311B6BF3FC3D30D931
Content-Type: text/plain; charset=us-ascii;
 name="pipelineComments.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="pipelineComments.txt"


> (MasatoshiOhishi)
> General Comments:
> 
> In the ITU (International Telecommunication Union), we discuss possible
> frequency allocations for Satellite Operators in the 40 GHz range and
> around 94 GHz.  I think we need to consider how to monitor and mitigate
> interferences due to space-to-earth direction transmissions from satellites.
> It is necessary to make real-time monitoring to avoid unwanted emissions
> from satellites.
> 

An addition requirement (6.4.1.-R3) has been added in that sense:

\item {\red The Pipeline shall be able to detect interference,
e.g. due to communication satellites operating at ar near the observed
frequencies, flag the data accordingly, and trigger alarms if
necessary. \reqpri{3}}

> ---005--------------------------------------------------------------------------
> (RayPlante)
> Hi Robert,
> 
> Unfortunately, my first comment is a about requirement that is (I think)
> missing, so I feel compelled to break from the requested format
> in Part 1.  Part 2 refers to existing requirements.  I hope my discussion
> is not too lengthy to be helpful.  Your committee's response need not be
> as detailed.
> 
> 
> 1. User Access to Software
> 
> I feel an explicit requirement is necessary about the usefulness of
> data in the archive to end users (astronomers); something like,
> 
>   "Astronomers shall have [free?] access to software necessary for
>   processing all data products available from the archive."
> 
> This may seem obvious (hopefully, I didn't overlook such a statement). It
> is certainly implied by other requirements in sections 1, 3, and 4;
> however, I think it's important to spell this out as I think it affects
> how these other requirements are met (and the cost of doing so).  Some of
> my other comments below hinge on an implicit assumption of this nature.
> 
> You probably should go even further.  Here are some possible riders:
> 
>   *  "In particular, the astronomers should have access to all software
>      necessary for duplicating or redoing the [reversible] processing
>      carried out by the pipeline"
>   *  "...on their local computing platforms"
>   *  the data reduction scripts are locally runnable
> 
> To the extent that these are true, they should be expressed
> explicitly.  The extent to which they need not be true, that should be
> stated as well--for example,
> 
>   *  restriction: limited to data products in the Observational
>      Archive.
>   *  possible exceptions: Atmospheric Model, Quick Look operations
>   *  products made available by astronomers (e.g. final images)
>      resulting from local processing are exempt from this
>      requirement.
>   *  some projects may require high performance computing to perform
>      the processing; performance requirements (execution time, memory,
>      disk space, etc.) should not be considered when meeting this
>      requirement for such projects.
>


You should look into the requirements for off-line data processing. 
Look in particular OL-1.1-R5, OL-1.2-R8.

The pipeline will also be runnable at places where the archives are
kept; these are known as "Regional Support Centers", and are foreseen
by the ALMA project.


> 
> ---007--------------------------------------------------------------------------
> (TimCornwell)
> Apologies for being late. I hope that you can consider my suggestions. I
> appreciate being asked to review these requirements. I found it helpful
> for our own work here on end-to-end processing of data from NRAO
> telescopes. I have a few global concerns:
> 
> -        The distinction between the various pipelines is insufficiently
> motivated.
 

We have tried to clarify the distinctions. The Science Calibration and
Science Imaging pipeline were merged into a single "Science
Pipeline". The real-time calibration and the quick-look pipeline have
obvious inter-relations, and could therefore be merged into a single
"real-time pipeline". However, we kept the distinction, in order to
emphasize in the requirements document which operations are necessary
for the system running, and which are only performed for monitoring.

The actual distinctions between the pipelines is more an architecture
issue than a requirement issue. In fact the Analysis group has taken
the view of integrating the (real-time) Calibration Pipeline in the
Instrument Operation Subsystem, not the Data Reduction subsystem.

> -        The goal of processing all standard modes should be qualified
> by the attachment of quality measures.
>
 
Accepted. An addtional requirement (6.5-R5) has been added for that purpose.


> -        There is a tendency to be overly prescriptive in describing how
> the reduction is to be done. I think that our best practices will evolve
> as ALMA comes online and the requirements should reflect this.
> 

We have added a number of "provisions" to make it clear that only the
current best-practices are described in this document. On another
hand, the ALMA pipeline should be developed during the coming years,
and the SSR document should be as prescriptive as possible, in order
to be useful for the developers.

> 
> ---009--------------------------------------------------------------------------
> (WimBrouw)
>         Comments on Pipeline part of ALMA SSR and Use cases --  version 4
> 
> Introduction
> 
> Add in the introduction that the Science calibration and Imaging will be a
> sub-set of the off-line data reduction package on the one hand; and that
> the real-time calibration will be available in the off-line data procesing
> package.
>

This would imply that the Pipeline be built with the Off-Line package.
While this may be true in practice, it is felt that this is not a
requirement for ALMA software. The requirement is that all operations
done by the Pipeline can also be done off-line.
 
> Add that pipeline processing will have parallel streams, with a mixture of
> synchronous and asynchronous operation
> 

This is an implementation issue.

> Question (in principle for Use cases): will the measured WVR be archived? If
> not, calibration based on WVR should not be included in off-line package.
> 

Yes, the WVR data will be archived. See Req. 7.2-R2.

> ---012--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 26, 3.6.1 "NOte" --> "Note"
> 
> "Occurrence" is misspelled as "occurence" everywhere in this section
> 
> 3.6.1.1 "make results available to the dynamic scheduler" -- this assumes
> a certain division of the system into items that include a "dynamic scheduler."
> Since things like preliminary phase rms will be used to adjust target and
> calibrator dwell and cycle times, the requirement as expressed would not
> address this use. In general, it would be better if the authors indicated
> how they want quantities used, rather than to which (as yet undefined) "part"
> of the system they should be sent.
>

Accepted. We have replaced "Dynamic Scheduler" by "dynamic scheduling
system".  This part of the system is described in another part of the
SSR document; the list on required inputs to the dynamic scheduling
algorithm is given in Req. 4.0-R3. Some of those input parameters are
supplied by the Pipeline, which we have indicated in the Pipeline
requirements. This should be cross-checked with the dynamic scheduling
requirements.

> Are baseline and holography calibrations really "(quasi) real time"? Robert
> Lucas indicated to the software group that baseline calibrations might be
> deferrable to a time after scientific observations corresponding to the (new)
> antenna configuration had been completed--in some cases.

Accepted. Baseline and holographies are not quasi-real time operations
-- except possibly in some cases, as high-frequency work, where the
dynamic scheduling may have to take into account the baseline
quality. As suggested by another reviewer, we could consider such
operations as "plug-ins". The Requirements 6.3.4-R2 to R5 have been
rephrased accordingly.

> 
> Again, "make results available to the sequencer" doesn't say what these
> results are to be used for. "Sequencer" is not a meaningful concept at the
> requirements stage.
> 

Accepted. "Sequencer" has been replaced by the very general term
"real-time control system". Some parameters derived by the Pipeline
(e.g. pointing offsets) shall actually be used by the control system
in order to update the instrument parameters.

> In the same vein, requiring Quick Look Operations to be done during and after
> an "observing session" doesn't give developers the information they need to
> make these operations useful. Who is to look at the results? When? What kinds
> of corrective actions should be made possible by these results?
>

The QL Pipeline is there to allow the ALMA staff to monitor all
observational/astronomical informations that can provide a valuable
information to detect any misbehaviour of the instrument. The results
are also provided to the observer (PI) by means of a web page (see
4.0-R9 and UC 4.5.2).

 
> ---013--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p.26 3.6.1.1  Real-time Calibration Operations
>    In Telescope / Array Calibrations, although I could be wrong, do we really
> need to reduce the holography data ? See also subsection 3.6.3.
>

See comment 12.
 
> ---014--------------------------------------------------------------------------
> (TimCornwell)
> Page 26, 3.6.1.1 Pipeline operations
> 
> The distinction between Science Calibration and Science Imaging is too
> fine to be worth building in at this level. While calibration and
> imaging used to be considered as separate items in a waterfall model, a
> more modern view is that the two are intertwined via the concept of
> self-calibration. 

Agreed. The two sections are now merged.

> I am similarly dubious about the distinction between
> real-time calibration and quick look operations. I'd recommend the
> distinction be between Real-time and Post-observing Processing.
> 

Cf comment 007. The difference here is in the motivation: real-time
calibration is for automatic detection of problems, quick look is for
humans.

> ---015--------------------------------------------------------------------------
> (WimBrouw)
> p26 3.6.1.1: Most of the following details in this section are valid for all
> types of operations. Iso saying 'relates to interferometric', why not have
> an indicator to exclude the few items that are not valid for single dish
> or light-bucket operations.
> 

Agreed.

> ---016--------------------------------------------------------------------------
> (WimBrouw)
> p26 3.6.1.1 RT cal operations
> - Some of these are 'real' RT, i.e. they have to be done before observing can
> proceed, since errors will remove information (like focus; delay; offset
> pointing); others are not (holographics; baseline tests)

Agreed.

> - The Atmospheric model is RT; some of the astronomical ones could be
> (atmospheric calibration idf that is the one based on the atmospheric model);
> others are definitely not (bandpass calibration; prelimin phase and ampl cal)

Agreed for bandpass. But the preliminary phase and amplitude
calibrations are needed to provide feedback to the dynamic scheduling
system, as well as to the ALMA staff (QL monitoring) -- they are thus
to be obtained quasi-RT.

> - Is the 'preliminary phase rms' purely based on atmospheric information? If
> not, what is its purpose for the dynamic scheduler (since the phase noise
> will depend on baseline; ampl; atmosphere, ..

The phase rms are computed from the calibrator measurements. They are
used to estimate the seeing, and are an input for the dynamic
scheduler.

> - Can it be indicated which one are 'await result RT', and which are not?

The list given in 3.6.1.1 is mainly to give a overall view of
each group of operations; the precise definitions are given in the 
requirement list.

> - Indicate that this processing should include, but not be limited to the
> list given

Agreed. 

> - Should results be made available to sequencer (and/or scheduler) or should
> they be archived, and be available through archive?
>

This is an implementation issue. 
 
> ---017--------------------------------------------------------------------------
> (WimBrouw)
> p26
> 3.6.1.1 QL operations
> - Occurrence: after observing session, and during observing session at tbd
> intervals
> - Monitoring-- done through archive access of cal data?
> - Quick calib -- apply should be before resample/integration; also what is the
> calibration done here -- oh, I see apply only; not do. Maybe some terminology
> to indicate active and passive use of the verb 'to calibrate' should be
> used. Maybe use here 'correction' rather than 'calibration'
> - Quick imaging: rather than 'no deconvolution'. I would think 'subset of
> data' is more important (specified beforehand e.g. for spectral line to
> integrate channels with expected result
> - Display tools: display 'information about' current observations. Display of
> monitoring information and non-imaged data (e.g. baseline-time amplitude
> images) and statistical extractions are more important than 'images'
> (especially for non-SNR=100 single sources)
> 

The list given in 3.6.1.1 is only to give a overall view of each group
of operations; the actual definitions are given in the requirement
list. In order to avoid ambiguities, the list in 3.6.1.1 has been
simplified.

> ---018--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 27, Science Imaging Operations
> 
> Accessing the archive for previous observations, producing and archiving
> deconvolved images are expensive to do "after completion of observing session."
> My understanding from the SSR discussions was that these would be done,
> *at most* (and this is what is implied by requirement 3.1-R13) 
> after breakpoints and (of course) when
>  the project (or at least observations of the target) is
> complete. This requirement is also in conflict with Use Case 4.5.3 on p. 73.

Accepted.

After discussion in the Granada Meeting we have decided to separate
occasions when full science data processing is performed ("monitor
points"), from the break points and the end of sessions. The PI will
specify in the Observing Tool when Monitor Points are requested: after
each scheduling block, after each end of session (provided a minimum
observing time is reached), or at break points only. The necessary
computing resources will be evaluated by the Observing Tool and this
evaluation will be available to the reviewers.

> 
> 3.6.1.3 It seems as though "instrumental" calibrations were called "telescope"
> calibrations in 3.6.1.1. The change in terminology is confusing.
> 

Agreed.

> Although bandpass calibrations do not require a time interpolation, and
> can therefore be "immediately derived and stored, to be applied to all
> following observations," 3.6.1.1 says it's "derived or applied" after
> completion of observing sessions. This seems inconsistent to me, and is
> another example of how the requirements can get confused when they contain
> implementation considerations instead of real requirements.

Agreed.

> 
> ---019--------------------------------------------------------------------------
> (TadafumiTakata)
> p27 3.6.1.1 Pipeline Operation
>            Science Imaging Operation
> Comment
> Accurate positional calibration may be needed in producing
> deconvolved images, which are most popular data for general
> archive users. They have to include coordinate information
> like FITS WCS headers which can be interpreted by image
> browsers to be provided to ALMA users etc.
> 
> (Please ignore this comment if it is already included in the
> pointing and/or instrument calibration before producing science
> images.)

Agreed.

> 
> ---020--------------------------------------------------------------------------
> (TimCornwell)
> Page 27, 3.6.1.3 Calibration
> 
> I don't think that one can draw a line between items requiring
> interpolation and those not. All items must benefit from interpolation
> (or modeling in time and space). Another important and related factor is
> quality control: one needs access to a time-series to allow discovery of
> errant values. The key point here is whether one must wait for all
> relevant values before making a prediction of calibration values at some
> point in time and space.
> 

The Science Pipeline should use the best technique available, including
checks of any time-variation of all calibration parameters. However, this
would make no sense for the Quick-Look pipeline, which should use the
simplest method.

> ---021--------------------------------------------------------------------------
> (WimBrouw)
> p27 3.6.1.1 Science cal operations (science correction(?))
> - Why after completion of session? Why not also require option during
> session. I could think of a program; and hence an observing session,
> observing many individual fields as snapshots for getting to know some
> parameters of a series of 'sources'. Run it in between
> - derive and/or apply: bandpass (not a curve according to other places but
> fixed; a session could have many bands and places in bands; and hence many
> bandpasses
> - what is 'final' phase and ampl calibration: cannot eb precise enough ijn
> pipeline with 'limited or no deconvolution and selfcal). Or is 'final' as far
> as the pipeline is concerned?
> - flux scale is that derived here?
>
 
The list given in 3.6.1.1 is only to give a overall view of each group
of operations; the actual definitions are given in the requirement
list. In order to avoid ambiguities, the list in 3.6.1.1 has been
simplified.

 
> ---022--------------------------------------------------------------------------
> (WimBrouw)
> p 27
> 3.6.1.1 Science imaging
> - should 'produce temperature-calibrated visibilities' be in previous step?
> (btw why day 'uv tables',is implementation
> - is 'deconvolved' true here? If so; there should be an extensive
> (self-)calibration step in scince calib operations to be able to derive that.
>

See comment 21.
 
> ---023--------------------------------------------------------------------------
> (WimBrouw)
> p27
> 3.6.1.2
> - astronomical 'source': a project can have many sources (list) or a large
> (mosaiced) field. Mention that source here is not the normal single, limited
> source.
> 

Rephrased.

> ---024--------------------------------------------------------------------------
> (WimBrouw)
> p27
> 3.6.1.3
> - instrumental: indicate which ones (as mentioned above) are part of the
> RT-wait set; and which ones are not
> - no time interpolation: I am not sure it is a good idea to give these
> examples as 'time-invariant' calibrations. There are quite a few schemes
> where it is easily possible to have to interpolate e.g. bandpass (filter
> poles can be very T sensitive; if receivers are going to apply diurnal
> Doppler); bandpass could also have a phase error. I would leave it
> open. Also: bandpass and pol are non time-critical:
> Also, indicate listsinclude but are not limited to)

See comment 20.

> 
> ---025--------------------------------------------------------------------------
> (WimBrouw)
> p28. Finally ... average of observed...' I.e. leave out 'the', and indicate
> that the average will be done outside the pipeline operations (there could
> be contaminated channels e.g.)
> 

The pipeline shall produce a pseudo-continuum measurement, using
astronomers' inputs. The final average is to be done by the
astronomers, off-line.


> ---026--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 29, 3.6.2-R2 What does "data-driven" mean here?
> 

It means that the Pipeline should run automatically as soon as new
data is acquired and that operations to be performed should be
determined by the nature and purpose of the data, as indicated by
header information associated with it.

> 3.6.2-R3 The meaning of, and need for, "templates" isn't clear. Isn't this
> more of a design issue for the people doing the observing preparation tools?

Agreed. The second part of the requirement has been removed.

> 
> 6.2-R4 I understand a "pipeline" as something that, once started, runs
> automatically. In contrast, "tools" are usually instruments for
> doing some task that one wants to direct interactively.
> "Automatic flagging tools" sounds like a contradiction to me.
> I think you mean that certain data should be flagged automatically.

Rephrased.

> 
> 6.2-R6 This requirement is unreasonable, in my opinion. You can't repeat a
> series of previous operations and you can't resort to a copy of the dataset
> at an intermediate state. "Sufficient recording..." then means that results
> of every step have to be saved -- but not as a "copy of the dataset". Maybe
> I misunderstand, but I think that a straightforward interpretation of what's
> written here will produce a very heavy and expensive system. I think the
> kinds of "steps" that can be reversed and redone should be spelled out in more
> detail.
>

This requirement has been removed.
 
> 6.2-R9 I don't understand. Doesn't the quantity to be calibrated determine
> whether it's baseline- or antenna-based? (Maybe this is a stupid question,
> but the Aperture Synthesis Summer School isn't until after the deadline for
> comments.)
> 

In some cases, antenna-based calibration may fail, in which case
baseline-based calibration is to be used.

> ---027--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 29 3.6.2  Pipeline general requirements
> In 6.2-R4, the degree of interference should be added as a condition to
> flag data.
>

Agreed.
 
> ---028--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.29 6.2-R3
>      'through readable and comprehensible data reduction scripts'
>      Sounds good but I think there are additional points, namely: a)
>      if observing scripts exists it would be better to use the same
>      control language for both observing and reduction scripts, and b)
>      in order not to be too dependent on a specific pipeline engine
>      the script language must be independent of any specific
>      processing system (ref. Reduction Blocks for the VLT).  That is
>      the script should provide the flow control and specify the
>      reduction tasks to be executed while the engine just should
>      execute these tasks as best it can.  Further, there is a
>      verification issues (like for observing scripts) if users are
>      allowed to change them.
> 

The second part of the requirement has been removed, in order to be
less prescriptive.

> ---029--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.29 6.2-R6
>      'Sufficient recording ... shall be carried out so that any step
>      can be reversed' is a very general requirements.  Some processing
>      is difficult to reverse and often it is simpler to save some
>      intermediate results.  Recording all reduction steps is clearly
>      needed but already covered by 6.2-R5.
> 

The requirement has been removed.

> ---030--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.29 3.6.2 Pipeline General Requirements
>      I am missing some requirement on that the engine used for pipeline
>      processing should be well separated from the ALMA system i.e.  it
>      should be possible to replace the pipeline engine without any
>      significant change in the ALMA software.  Although this partly is
>      a design concern since one would not like to have the ALMA
>      software depend on an alien, uncontrolled package, it is also a
>      science concern.  One would like to enable anybody to contribute
>      useful pipeline modules to ALMA and not limiting them to software
>      written in a specific system.  Further, with a lifetime of
>      decades for ALMA it is likely that it will outlast one or several
>      pipeline engines.
> 

We believe this is a fair design concern, based on experience.

We propose to add the general requirement:

The choice of a specific data processing package to provide the main
pipeline functionality should not impose unnecessary constraints on
the rest of the ALMA software system, in particular on the manner in
which the data is stored and handled, as in the future new or better
functionalities may be provided by newly developed data processing
systems.


> ---031--------------------------------------------------------------------------
> (RayPlante)
> p. 29, 3.6.2, 6.2-R6:  (Reversability of processing)
> 
> I think the general aim of this requirement is necessary: we want the
> ability to remove or substitute any part of the processing.  In
> particular, we want to be able redo it with small modifications to the
> parameters.  However, the wording of this requirement concerns me when
> I consider its strict application to complex processing; some
> clarification may be helpful.
>

The requirement 6.2-R6 has been removed.
 
> This requirement places a corresponding requirement on the off-line
> software (assuming point 1 above), the data formats used, and on what
> products must be stored in the archive.  Should one be able to back up
> an arbitrary number of steps in processing ("with out resorting to a
> copy at an itermediate state")?  What constitutes a step?  Suppose
> target source has been self-calibrated with multiple loops; does
> reversability mean that we have to keep the generated gain table and
> image models for each loop?  Taken to its extreme, this requirement
> could be pretty costly:
> 
>   *  if all datasets required for step-wise reversability, then the
>      archive must organize and label these products in a way that they
>      make sense to the user.
>   *  the data formats must retain sufficient information for backing
>      up.
>   *  for every processing step implemented, its reversal must also be
>      implemented.  The cost is greater if this capability must be
>      available to the end astronomer (see 1 above).
> 
> If we only mean to apply this requirement to the extent already
> supported by current packages, we may be okay here.  If we only
> require needing to make one step backward at any given step in the
> chain, we might be okay.
> 
> The cost is of particular concern if it's an unnecessary one.  Thus,
> I'm curious as to the intent of the clause "without...resorting to a
> copy of the dataset at the intermediate step."  Redoing processing by
> going back to an intermediate product is often the easiest way to
> "back out" (as opposed to reversing multiple steps in turn).  If the
> processing script is available (which is required) and the assumption
> from 1. (above) is true, then this is straight forward.
> 
> This requirement could be clarified in the context of the definition
> of ranked data products in the archive.  For example:
> 
>    Level 0 -- raw data: has no/certain/all real-time calibratation
>       operations applied
>    Level 1 -- calibrated data: has science calibration applied
>    Level 2 -- deconvolve images:  has Science imaging operations
>       applied
>    Level 3 -- final, user-supplied (locally processed) data products
> 
> Then, say, if one starts with the data products at a given level and
> apply the scripts that produce the next level stepwise, it should be
> then possible step backwards at any point in the chain.
> 
> By the way, defining ranked data products will help the user
> understand what they need from the archive.  They can easily choose
> what level of processing they want to accept and know what products
> they, therefore, need to retrieve.
> 
> ---032--------------------------------------------------------------------------
> (RayPlante)
> p. 29, 3.6.2, 6.2-R8:
> "Sequencer" -- you probably need an entry in the Glossary for
> this.  I was a little unclear about what it does.  According
> to my pdf reader, this is the first use of this term in the document;
> although, its function may be made clear in the Use Cases (which I did
> not study as well).
> 

"Sequencer" has been replaced by the very general term "real-time
control system". 


> ---033--------------------------------------------------------------------------
> (TadafumiTakata)
> p29 6.2-R1.1
> Comment
>  Is there any need to have a feed-back (like result images and
> so on) from this operation ? If so, some tools for helping the
> process to reflect the result to ALMA archive should be necessary.
> 

This is not considered as a requirement for the Pipeline, but is worth
considering for the Regional Support Centers activities. However, the
concern is that if one allows PIs to contribute the results of those
special projects into the Archive, then ALMA has no control whatsoever
over the quality of the data. It is understood that, to be usable for
archival research (VO), the quality of the data has to be evaluated in
a well defined way.

> ---034--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, Pipeline General Requirements
> 
> I think that the goal of being able to process all data from the array
> in standard modes is too ambitious, certainly so for a telescope yet to
> be built. For the EVLA, VLBA, and GBT pipelines, we are planning to
> attach a quality measure to pipeline results. This adds an important
> level of qualification. A division might be:
> 
>             Meets all quantified scientific goals
> 
>             Meets some quantified scientific goals
> 
>             Meets none of the quantified scientific goals
> 
>   A priority 2 requirement could be added that the pipeline must process
>   all data from all standard modes to the highest level of quality. The
> _ priority 1 requirement is to process all standard modes and attach a
>   quality measure.
> 

Excellent suggestion. A requirement (6.5-R5) has been added in that sense.

> The requirement that the pipeline not constitute a bottleneck is a fine
> sentiment but I'm not sure who it is directed to? - TAC, operations?

Computing engineers, I guess.

> 
> ---035--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, 6.2-R2:
> 
> Some default parameters (cellsize, field of view, calibration methods,
> etc) should come from a standards database that is under change control.

We believe that those parameters should be provided at proposal
preparation phase, or defaulted at this occasion (so that the user may
check their values if needed).

> 
> ---036--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, 6.2-R3:
> 
> The second sentence is an unnecessary implementation detail. For NRAO
> pipelines, we are generating scripts from production rules encapsulated
> as make files. This is arguably superior to templates. In any event,
> it's not necessary to state how the scripts will be generated.
> 

Agreed. The second sentence has been removed.

> ---037--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, 6.2-R6:
> 
> Redoing is not the same as reversing. I'd recommend removing the
> reversing part. To redo, one simply needs checkpoints. To reverse, one
> needs much more. I think this requirement is close to requiring the
> ability to undo operations, which is very expensive.
> 

The requirement 6.2-R6 has been removed.

> ---038--------------------------------------------------------------------------
> (TimCornwell)
> Page 29, 6.2-R9:
> 
> I think this requirement is not necessary. It is true that antenna-based
> calibration is better than baseline calibration for effects that really
> are antenna-based. However, it almost certainly is true that physical
> modeling of antenna phases by e.g. time and space parameterized phase
> screen is superior to antenna-based calibration.  Perhaps the
> requirement should state that "Best practices" must be following in
> calibration?
> 

Requirement rephrased.


> ---039--------------------------------------------------------------------------
> (WimBrouw)
> p29 6.2-R1.1 This holds for science data; I would suggest that the pipeline
> always handles 'active calibrater' data
>

The calibration pipeline should handle these data in the standard way
(pointing, focusing, checking phase stability) in order to ensure that
the requested observing conditions are met.
 
> ---040--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R3: shall operate through 'automaticly generated ' readable ... Drop the
> second part (implementation)
>

Agreed.
 
> ---041--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R4: 'step' is undefined; say: the pipeline shall include automatic flagging
> of data. Do not use 'discard' (bypass? not use).
>

Requirement rephrased.
 
> ---042--------------------------------------------------------------------------
> (WimBrouw)
> p29
> add: R4.1: Flagging should be multi-level to enable selctive re-use of
> automatically flagged data; or the automatic flagging must be reversable in
> the off-line stage using identical algorithms
>

It is already mentioned in this requirement that the flagging must be
reversible at the off-line stage.
 
> ---043--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R5: is 'output' archiving (should be)?
> 

Yes.

> ---044--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R6: yes; but add : '.. reversed and redone, taking into account the order of
> operations and the fact that some of them are non-commutative, and have to be
> 'done and redone' ...'
> 

6.2-R6 has been removed, because it was felt it was not a requirement
for the pipeline operations.


> ---045--------------------------------------------------------------------------
> (WimBrouw)
> p29
> R7 and R2 are mutually exclusive. Maybe R7 could be rephrased slightly?
>

Rephrased. 
 
> ---046--------------------------------------------------------------------------
> (WimBrouw)
> p29
> 6.2-R9: talking about Ampl and phase only?
> R9: could you give one or more examples were baseline calibration (of what)
> is required?
>

In some cases, antenna-based calibration may fail, in which case
baseline-based calibration should be used, e.g. when atmospheric
decorrelation during the basic integration time is important.
Baseline-based calibration is also required for effects that are truly
baseline-based -- something happening in the correlator?

 
> ---047--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 30, 3.6.3.1 "Astronomical Calibration: Atmospheric Model" -- this
> terminology is inconsistent with that of 3.6.1.1, where "Atmospheric Model"
> and "Astronomical Calibration" are listed as separate categories.
>

Title of 3.6.3.1 changed.
 
> ---048--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 30 3.6.3 Real-time Calibration Operations
>   In 6.3-R1 it says, "The real-time Calibration Operations shall be activated
> AFTER each observations."  However in page 26, in subsection 3.6.1.1,
> it says, "Real-time Calibration Operations occur (quasi) real time."
> I feel these are inconsistent.
>

They are not inconsistent, because the term "obervation" is used
according to its definition given in the SSR document, p. 10: it
actually represents a short integration period.
 
> ---049--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.30 6.3-R1
>      '... after each observation' It sounds too strong for me.  I
>      would have expected that only after calibration observations
>      the calibration Operations are done. Normally, calibrations
>      would be done after each science observation which then would
>      trigger the calibration operation but the current requirement
>      is stronger.
> 

If the last observation is a science one, then the calibration
pipeline has nothing to do. If it is a calibration observation, then
several operations have to be performed. The requirement has been
modified for clarity.


> ---050--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.30 6.3.2-R2
>      'convert the raw data into temperatures, or, alternatively, store
>      the conversion factor' There seems no reason to give an
>      alternative -  either one or the other.  I would prefer the
>      latter since this would store the raw, acquired data with the
>      factor and not apply it.
>

Requirement removed.
 
> ---051--------------------------------------------------------------------------
> (RayPlante)
> p. 30, 3.6.3, 6.3-R2:
> "Whenever the results ... allows to identify" ==>
>      "Whenever the results ... allow one to identify"
> 
> ---052--------------------------------------------------------------------------
> (TadafumiTakata)
> p30 6.3.1-R3
> Comment
>  The resultant parameters of model calculation should be referred
> by users (including astronomers) via "The Data Extractor Tool" or so
> for evaluating how the data are affected by this parameters.
> (I think it is already included in enviromental information extraction.)
>

The results of the atmospheric model are to be archived, and should
thus be accessible later on.
 
> ---053--------------------------------------------------------------------------
> (TimCornwell)
> Page 30, 6.3-R2:
> 
>   I think this should be priority 2. It's going to be very hard to do from
>   the real-time calibration. An equivalent goal for the post-observing
> _ processing is possible and should be priority 1.
>

Agreed.


> ---054--------------------------------------------------------------------------
> (TimCornwell)
> Page 30, 6.3.2-R3:
> 
> This seems too prescriptive. What is described is the current best
> practice for mm arrays. This may not be the best practice for ALMA. Does
> one want to bind ALMA to work this way?
>

Requirement rephrased.
 
> ---055--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3-R2 '...after?? ..' RT calib is necessary before some data can be
> taken (atmospheric model). Certainly an observing session should also end
> with one.

The term "observation" is used here with the meaning defined in the
SSR document, p. 10: it does represent a short amount of integration
time, not the whole observing session. For clarity, this requirement
has been rephrased: the RT calibration pipeline is to be activated
after each calibration scan.

> If you do flag; multi-level flagging is even more important.
>

See requirement 2.3-R2
 
> ---056--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3.1-R1 ...The prediction will be based on measured atmospheric data,
>     including, but not limited to:
>

Rephrased.

 
> ---057--------------------------------------------------------------------------
> (WimBrouw)
> p30
> R1: what is 'line-of-sight'?: on a per antenna basis (what I would assume,
> certainly for higher frequencies and longer available baselines); and does it
> assume some isotropy and/or weighting over the HPBW FOV?

This is done for each antenna independently (though some consistency
between neighbouring antenna could be checked). There is no planned
method to measure variations of atmospheric properties inside the
antenna primary beam (may be infrared sensors could be used?). However
most of the absorption occurs in the low atmospheric layers where all
line of sights inside the primary beam see the atmosphere averaged
over the same near-field antenna pattern.
 
> ---058--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3.1-R3 already covered in R1? If not, what is different?
>

R1 is a general requirement that a powerful atmospheric modelling
be available. R2 and R3 are indicating two crucial uses of the
atmopheric model.
 
> ---059--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3.2-R1: 'store the results'? or 'archive the results'
> 

The verb 'archived' may suggest that the Pipeline writes the results
listed in this requirement directly in the Archive.  This is actually
an implementation issue. What matters here is that those quantities
are available for further Pipeline operations -- they could thus be
written in the Archive at a latter stage, together with several other
values. We therefore prefer the term 'stored', which is more general.

> ---060--------------------------------------------------------------------------
> (WimBrouw)
> p30
> R1.*: a time-constant indication is necessary here. It is a large variety of
> items (and again the list ois probably not complete over life-time of
> instrument). I think 1 and 3 are per observation normally (or even
> longer). R2 could be fast varying?? (probably depend on length of observation
> as well); R4: is that in bore-direction (is really long-term to determine
> correctly ). The polarised antena pattern is something measured annually
> maybe.
> 

The software should be able to reduce all those observations. The
frequency at which such calibrations are performed is an operation
issue, which will largely depend on the observing modes, and on the
experience acquired using the ALMA. Hence, we prefer not to give such
timescales in the software requirements.

> ---061--------------------------------------------------------------------------
> (WimBrouw)
> p30
> 6.3.2-R2: alternative is incorrect. Tsys should always be stored
>     (=archived). The alternative is the deferment of the conversion, not the
>     storage.

Agreed. This requirement has been removed. 

> 
> ---062--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 31, 6.3.2-R3.4 "...passed to the Dynamic Scheduler..." Again, it is
> difficult to understand how this data is to be used. Scheduling decisions are
> typically made on the timescale of an SB (nominally 15 minutes-1/2 hour, in
> order to take account also of the time needed to bring a new receiver band
> to a ready state). So what's needed here? Averages? Instantaneous values?
> Predictions of how these values will evolve over the next 1/2 hour?
> 

Cf. comment 12.

> ---063--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 31, 6.3.4-R2, How are the baseline calibrations to be used by the
> "Sequencer"?
>

The "sequencer" ( = the real-time control system, cf comment 12) shall
use the antenna locations to compute the delays to be applied on each
baseline by the correlator.
 
> ---064--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 31 3.6.3.4 Telescope / Array Calibration
>   In 6.3.4-R4, do we really need to reduce the holography data real-time ?
> 

No, not real time. Cf comment 12.

> ---065--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.31 6.3.3-R1
>      '... and pass the results to' For me 'pass' suggests something
>      active i.e. the pipeline explicitly sends the information.
>      I would prefer 'made available' only.
> 

Agreed.


> ---066--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.31 6.3.3-R2
>      Same comment as for 6.3.2-R2
> 

Same answer.

> ---067--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.31 6.3.4-R1
>      'must be passed or ...' Same comment as for 6.3.3-R1
>

Agreed.
 
> ---068--------------------------------------------------------------------------
> (TimCornwell)
> Page 31, 6.3.4-R2:
> 
> "baselines" is jargon. I'd change this to "antenna locations".
>

Agreed.

 
> ---069--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.2-R3.3 : How can you have different parameters for the correction
> that is done on a fast basis before integration over samples?
>

The idea is to choose the best conversion coefficient when the
observation has been repeated using a few values. This may not be
necessary.
 
> ---070--------------------------------------------------------------------------
> (WimBrouw)
> p31
> R3.4: what do you gain by doing this operation per baseline??
>

See comments 46.
 
> ---071--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.3: Drop; or indicate again the earlier comment
>

6.3.3-R2 is removed.
 
> ---072--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.4-R1: why explicitly made available to sequencer (earlier comment). Let
>     sequencer determine what it needs: just archive/log it.
>

Cf comment 12 on dynamic scheduling. 
 
> ---073--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.4-R1.1: indicate (or in a more general comment) if it handles on offset
>     pointing here (as I would assume).
>

Rephrased.
 
> ---074--------------------------------------------------------------------------
> (WimBrouw)
> p31
> 6.3.4-R4: should piepline handle this? Why?
> Mybe make it more generic:
> ' The pipeline should be able to accomodate plug-ins for the handling of
> special observations like holography; absolute pointing model generations;
> baseline determination etc'
>

Excellent suggestion. Requirement rephrased. 
 
> ---075--------------------------------------------------------------------------
> (JoeSchwarz)
> p.32, 6.3.4-R5 Is the derivation of the primary beam properties from planets,
> etc., really a "real-time" operation?
>

No, just like holography or absolute pointing models, these are
telescope calibration operations to be performed regularly, but not
to be reduced in "real-time". The requirements have been rephrased
for clarity.
 
> ---076--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 32, 6.3.4-R6 "...passed or made available to the Sequencer..." Once again,
> we need to know how these results are to be used, and on what timescale. Even
> if we accept that there will be a "Sequencer" in the system, we have no
> "Sequencer Requirements" chapter (and I hope we don't write one!), so we need
> to know *what* is to be done with pointing, focus, skydip, etc. Are they to
> be used to correct the control system's pointing model so that the antenna
> points where it's supposed to, or rather to correct the data already taken,
> or both...?
> 

Cf comment 12.

> ---077--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.32 6.3.4-R6
>      'must be passed or ...' Same comment as for 6.3.3-R1
> 

Agreed.

> ---078--------------------------------------------------------------------------
> (WimBrouw)
> p32
> 6.3.4-R5: drop the 'and aperture efficiency'
>     Add 'squint' to the list; or jusdt make it 'generic beam parameters like
>     ...'
>     Should this also be a plug-in? I assume this is not done on a daily
>     basis.
>

Requirement rephrased.

 
> ---079--------------------------------------------------------------------------
> (WimBrouw)
> p32
> R6: drop
>

Requirement rephrased.

 
> ---080--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 33, 6.4-R2 This seems too vague to me. How much data should be made
> available to the PI's over the Internet? When? In near real-time? Suppose the
> PI's aren't available or are asleep? It's entirely unclear how this data is
> to be used, e.g., whether we're talking about letting the PI play operator
> from his/her institute in Europe or America or just check on what an image
> looks like from time to time...
>

This was introduced last year after the previous review.  The way it
should work is explained in 4.0-R9 and Use Case 4.5.2.
 
> ---081--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.33 6.4-R1
>      'shall be activated automatically after each ...' That is after
>      each observation which may be too much.  I would weaken the
>      statement and not make it mandatory but configurable.
>

Requirement rephrased.
 
> ---082--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.33 6.4-R2
>      '..., via the Internet' It sounds nice but it may give a
>      bandwidth problem if lots of people tries to get Quick Look
>      data like images.
>

Cf comment 80.
 
> ---083--------------------------------------------------------------------------
> (WimBrouw)
> p33:
>  6.4.1-R3.4: integrated? Since the seesion will contain 'simple scans' in
> frequency and/or position (mosaicing..) difficult to define exactly
> here. Just say 'indication of above over seesion' or so
> 

Rephrased.

> r3.5: noise per pointing?
> 

Added R3.6: for mosaics, it should be possible to monitor all quantities 
(R3.1 to R3.5) per pointing center

> ---084--------------------------------------------------------------------------
> (WimBrouw)
> p33
> 3.6.4.2/3: See it as an indicator of what should be done at the
> minimum. A good list. Maybe add to R3: 'TAKING INTO ACCOUNT FLAGGING'
>

It is implicit in the definition of 'flagging' that all data affected
are not taken into account in subsequent operations, unless
specifically mentioned.

 
> ---085--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 34, 6.4.2-R4 "Mosaic and self-calibration projects shall be supported."
> We need much more information than this to understand this requirement, and
> hence to know whether we have fulfilled it! What kind of Quick Look Operations
> are appropriate here?
> 

Requirement rephrased.

> ---086--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 34 3.6.4.3 Data Processing: Single Dish Data
>   In 6.4.3-R1.1, "on/off " could be replaced by "position switch".
>

Agreed.

 
> ---087--------------------------------------------------------------------------
> (TimCornwell)
> Page 34, 6.4.2-R3:
> 
> The mention of the Fourier transform is too prescriptive, and is
> unnecessary. It is too prescriptive because it is entirely possible that
> we might end up using some efficient linear algebra technique to make
> images quickly. It would be better to require something like along the
> lines that any image be available in a time comparable to a typical
> observing block.
>

Agreed. This requirement is rewritten accordingly.

 
> ---088--------------------------------------------------------------------------
> (TimCornwell)
> Page 34, 6.4.2-R4:
> 
> More explanation is needed: does the Quick Look pipeline just process an
> observing block or all relevant observing blocks? E.g. in a mosaicing
> experiment, does the QL pipeline just image the last patch of the sky
> observed? This is also a problem for self-calibration.

The QL pipeline deals with all data acquired during the current
session. For a mosaic, it should probably use a quick-and-dirty method
to produce an image, the best processing being deferred to the Science
Pipeline and/or to the off-line stage.

> 
> ---089--------------------------------------------------------------------------
> (TimCornwell)
> Page 34, 6.4.2-R5.
> 
> This is too prescriptive and probably wrong: one is better off doing
> this in the visibility plane. I'd make this a best practice requirement.
> 

Agreed. See comment 91.

> ---090--------------------------------------------------------------------------
> (WimBrouw)
> P34
> 6.4.2-R4: mosaic yes; self-calibration not as stated (see R3; and many
> earlier remarks about 'no or limited'. Should be limited to make it a
> non-bottleneck
> 

Agreed.

> ---091--------------------------------------------------------------------------
> (WimBrouw)
> P34
> R5: Compare to 'clean-beam'?? I do not get this at all. Maybe state: For
> ...the pipeline shall use the data to produce an estimate of the seeing.'
> (and pointing??)
> 

The idea was to compare the size of the source with that of the clean
beam, to check the seeing effects. This requirement has been rephrased
to be less prescriptive: the pipeline shall be able to make any
valuable test on the data taken on a point-like source.

> ---092--------------------------------------------------------------------------
> (WimBrouw)
> p34
>  6.3.4-R1: a question: no statement about 'supported modes' is made for
> synthesis. Switching, OTF etc could be there as well. Should be added
> somewhere?
> 

There are actually less observing modes for aperture synthesis than
with a single-dish antenna: single-field imaging, mosaicing, OTF
mosaicing, snapshots. All modes can actually be calibrated in a
similar way, the imaging processing being different. Requirements
6.4.2-R3 and R4 have been modified to make it more explicit.


> ---093--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 36, 6.5-R1, R2 The term "session" is used too imprecisely here. Isn't it
> possible that some data preceding and following a session will be relevant?
> If, for example, a baseline calibration was done by the project that was
> executing 5 minutes before the current one, do we really want to repeat it?
> I imagine that there is other calibration data coming from outside the
> "session" that could be useful, too.
>

Several calibrations are to be done by the ALMA staff, and results
used by all subsequent projects. This includes baseline, pointing
model, primary beam measurements, etc. The results of such
observations are to be used by the real-time control system, to update
antenna parameters. For 6.5-R1.1/6.5-R1.2, 6.5-R2, only very recent
data is probably usable (same receiver tuning needed).
 
> I would also change "shall find in the Archive all data..." to "shall use all
> data...". Where this data comes from (whether it's in the Archive, or cached
> in memory, or whatever) is irrelevant from a requirements point of view.
> 

Agreed.


> ---094--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 36 3.6.5.2 Single Dish Data
>   In 6.5.2-R1.1, "on/off " could be replaced by "position switch".
>

Agreed.
 
> ---095--------------------------------------------------------------------------
> (TimCornwell)
> Page 36, 6.5-R2:
> 
> Can and should the Science Calibration Pipeline make use of historical
> observations which are not part of the project?
>

Yes: I think all observations of known sources identified as
calibration observations should be usable by all projects. In case a
particular project is observing a famous calibrator e.g. NRAO150 to
search for specific lines at high frequency resolution, then these
observations should be identified as target observations and will not
be considered by the system as calibrations.

I think the main interest is for flux monitoring of calibrators,
anyway, and that should be handled by an observatory project using all
calibrations, not by each project directly.
 
> ---096--------------------------------------------------------------------------
> (WimBrouw)
> p36
> 6.5.1-R1: The way the bandpass and time-variations are given here does
> not indicate that the requirement could be for multiple bandpasses (depending
> on type of session) and time-ferquency variations.
>

Agreed, there may be several bandpass calibrations in a session.
 
> ---097--------------------------------------------------------------------------
> (WimBrouw)
> p36
> R2: should the pipeline 'observe' such a source if none available??
>

No. It is the responsibility of ALMA to make sure that the observing
modes are consistent: the observing procedure should be intelligent
enough to ensure that a project is not done without checking that e.g.
flux and a bandpass calibration are performed. The Pipeline will not
go back and observe.
 
> ---098--------------------------------------------------------------------------
> (WimBrouw)
> p36
> R3: why not drop R3.1 and make it just amplitude and phase corrected for the
> appropriate frequency ('scaling' could be to simple depending on how the
> delay is done as a mixture of time-delay and phase rotation??
> 

Rephrased.

> ---099--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6-R1 "subproject" is not defined anywhere in this document.
> 

Changed to 'project'.

> ---100--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6-R4 Whether the Internet is an appropriate delivery medium for
> Science Imaging results regardless of their size seems to me debatable. If
> a PI has been waiting for six months for a project to complete (because,
> for example, it requires different antenna configurations), why can't he/she
> wait a few more days for the delivery of a DVD? I think that this decision
> should be made in view of a) how urgent the project is (a basis for coordinated
> or follow-up observations?); b) how much data is involved (Alma will produce
> 180 Tb/year, so it's not impossible that we might occasionally have datasets
> that are significant fractions of a Terabyte); and c) what the real capacity,
> cost and reliability of the Internet is in the Alma epoch.
>

Agreed. We have rephrased the requirement, the data should be in the
archive and accessible to the PI (in fact I do not think we have
required that the data are `delivered' to the PI).
 
> ---101--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6.1-R1, R2 "...find in the Archive..." Again, where it comes from
> isn't relevant.
>

Agreed.

 
> ---102--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6.1-R3, R4 If the flux scales are different, what is to be done?
> What are the possible consequences of "direct comparison of the redundant
> data"?
>

In both cases, if the data are not compatible, the quality flag should
be down.
 
> ---103--------------------------------------------------------------------------
> (JoeSchwarz)
> p. 38, 6.6.1-R7 I suggest that having "several [deconvolution] algorithms
> running in parallel" is better done as an offline, rather than an online
> task. If it *must* be done as part of the production pipeline processing,
> I don't see how it can be met as a "priority 1" requirement, which, as Page
> 12 of this document defines it, means:
> 
> "Must be there for Interim Science period, when the system is commissioned
> to produce meaningful science results."
> 

Requirement rephrased: having a deconvolution algorithm available is
in Priority 1. Having several algorithms running in parallel is in
priority 3.

> ---104--------------------------------------------------------------------------
> (MasatoshiOhishi)
> p. 38 3.6.6.1 Interferometric Data
>   In 6.6.1-R7, it would be worthy to add a new CLEAN algorithm,
> the Wavelet-CLEAN, developed by Japanese group. And it would be
> very important how we judge the image quality.  Which is the best ???
>

Agreed. Requirement rephrased.
 
> ---105--------------------------------------------------------------------------
> (PrebenGrosbol)
> p.38 6.6-R4
>      '..., via the Internet' Same comment as for 6.4-R2

See comment 100.

> 
> ---106--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 3.6.6 Science Imaging Operations
> 
> There is a potential problem in all pipeline processed images: the
> provenance of each scientific result must be knowable and
> straightforward. If a virtual observatory is to make use of ALMA
> results, there must be an ALMA standard product that was produced in
> some standard way without strange choices for e.g. cellsize, field of
> view made by the observer. By this logic, one is forced to produce at
> least two results: the "standard product", and that required by the
> observer. In many cases, the observer may just ask for the standard
> product. This whole aspect of the scientific imaging pipeline must be
> clarified. It affects a large number of the requirements following. My
> specific recommendation would be that two products are produced: the
> standard product defined by ALMA, and the observer's product, defined by
> the observer.
> 

The idea of having two products is reasonable, provided the
non-standard products are not required too often for large images (in
order not to increase the archive size needlessly) Note that I have no
idea how to define the standard product in terms of frequency
sampling: using the whole available correlator output at all times
would boots up the data rate! Here I think we have to trust the
observer, as well as with the mapped area. So what remains are the
cell size and the mapping algorithm. We have control over those as we
provide them in the pipeline.

My guess is that we should take your option whenever the user-provided
parameters (e.g. cell size, algorithm) are not regarded as `safe'
(e.g. may degrade the fidelity more that we like) by the pipeline. 

> Another question that must be resolved is whether results are processed
> only before insertion into the archive or also on exit from the archive
> (e.g. triggered if best practices have changed). This is a hard question
> to answer. This is discussed in requirements below but I think it needs
> to be thought through a bit more.

I think we should have this possibility. This is mentioned in the
Introduction and in the Archive section.

> 
> Finally, is the observer free to use the pipeline repeatedly or is the
> processing limited to that specified in the observing setup? If the
> former, how is time allocated? I think the latter is therefore
> preferred.

These are operational issues. The idea was that at regional centers
the pipeline may be run to reprocess data to the current standards of
data reduction. Experience seems to show that reprocessing data one
year after observations can be done at a fraction of the time (and
cost) of the first processing. But I agree this may be a problem for
large projects.  


> 
> ---107--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 6.6-R2
> 
> In many cases, the final product could be a linear mosaic instead of a
> deconvolved image. I'd therefore remove the second sentence.
> 

In all cases, a deconvolution will be applied to the data, so we prefer
keeping the current phrasing.

> ---108--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 6.6-R3
> 
> Can relevant non-proprietary data from other projects be included?
> 

This is a policy decision obviously. My position would be yes,
provided that the availability of such data was checked before proposing;
then the reviewers and/or project management may decide... 

> ---109--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 6.6.1-R3
> 
> Phase centers and polarization frames must also be checked.
>

This is included in the general term "instrument setup and properties".
 
> ---110--------------------------------------------------------------------------
> (TimCornwell)
> Page 38, 6.6.1-R6, 6.6.1-R7.
> 
> If ALMA works as well as expected, the extra images requested here will
> be unnecessary: for example, the various weightings should be less
> divergent than for the VLA. Similarly for the deconvolved images. Also
> there is a combinatorial explosion possible (weighting x deconvolution x
> other parameters). I'd recommend that the Standard Product be just one
> image.
>

Yes, but the pipeline should have the possibility, if explicitely 
required, to use different weighting schemes.
 
> ---111--------------------------------------------------------------------------
> (WimBrouw)
> p38
> 6.6.1-R5: There is no 'continuum measurement'
> 

Requirement rephrased: the Pipeline shall extract visibilities of a
PSEUDO-continuum, corresponding to the sum of the spectral channels.

> ---112--------------------------------------------------------------------------
> (WimBrouw)
> p38
> R2: 'compatible'?
>

Yes
 
> ---113--------------------------------------------------------------------------
> (WimBrouw)
> p38
> R5: 'appropriate'? I would think that the data is used to produce the required
> image cube; not the other way around.

Yes 

> 
> ---114--------------------------------------------------------------------------
> (TimCornwell)
> Page 39, 6.6.1-R9
> 
> This is really a call for more development of automated methods for
> identifying and removing continuum. In the absence of any known method,
> one cannot require that it be used!
> 

Requirement rephrased.

> ---115--------------------------------------------------------------------------
> (WimBrouw)
> p39
> 6.6.1-R7.3: Add: 'model fitting and data subtraction'

Yes

> 
> ---116--------------------------------------------------------------------------
> (WimBrouw)
> p39
> R8: do not specify both the domains. Why? I would think that image-plane
> subtraction is a non-option in general.
> 

Yes




--------------0A08B4311B6BF3FC3D30D931--