ALMA Science Software Requirements - an alternate spin

[Barry Clark <bclark@aoc.nrao.edu>]

Robert's document is a list of scientific desiderata.  For me, things
become much clearer if we take the next step, on converting the 
desiderata into an operational plan, in which we trace the flow
of information through the system.  I have done that below with
Robert's document, with additional remarks of my own about how 
I think things should be done.

1.  Pre-proposal software.
In the early days of the VLA, Alan Bridle's spreadsheet for recommending
configurations, bands, integration times, time required, etc. was a
great help for novice users.  At least the equivalent is needed for
Alma.  (Converting the recent Butler & Wootten memo into a Java program
would be a reasonable first step.)  There are various simulation tools
running under AIPS or AIPS++.  These are, I think, needed only by 
people who know how to take care of themselves - we don't need to 
take a hand in this.

2.  Proposal preparation software.
Do we want proposals to be machine parsable?  If so, we need to provide
tools to help in the effort and to complain at the user if he violates
the syntax.  In my mind, this is about a break-even.  The programmer
time to construct the tools would only be repaid by reduced effort in
operations over years if at all.

3.  Proposal evaluation tools.
I would expect at least as many proposals for ALMA as for the VLA,
which runs about 400 per year.  This is enough that software help is
mandatory for routing proposals to referees, collating their ratings, 
looking for conflicts, sorting out previous observations, and deciding
what proposals will fit in the available time.

4.  Scheduling tools - chunking
If we are to have dynamic scheduling, all long proposals must be divided
into chunks.  It is unclear to me who divides the proposal into chunks -
the proposer or array operations.  (This is a policy question, not a
software question.)  Do we want to have standard sized chunks of say a 
couple of hours?  Doing so simplifies life downstream by a lot.  If we 
do have standard sized chunks, it simplifies things even more if they 
occur on standard boundaries - eg 0000-0200 LST, 0200-0400 LST, etc.
(Another policy question which has largish software implications.)

5.  Scheduling tools - detailed scheduling
The detailed scheduling tools provide a reasonable human interface between
the astronomer and the real-time system.  I would suggest that the
detailed scheduling tools speak to the real-time system by way of a 
text interface that is human readable, at least by specialists.  We have
found that being able to read this stream is invaluable for the VLA and
VLBA - it lets real-time programmers talk directly to scientists without
having to go through the programmers of the scheduling tools.
Note that many of Robert's desiderata simply come down to being specs
on this script.

I can see valid arguments for having three different scheduling tools.
(They are enough effort to write that it's not clear that all three will
have sufficient priority to get done when push comes to shove.)  A
windows based scheduler, like the VLA's observe and Jobserve programs
lets the user see what he is getting in a very efficient manner.  A
language based scheduler, like the VLBA's sched program, is perhaps 
most efficient at taking advantage of contingencies; it could even
have conditionals built in, and it can conveniently optimize to keep
antenna move times under control.  The third mechanism would be a 
virtual control panel, like any other telescope control panel, with
a push button to send its setup to the real-time system.

There are a lot of decisions to make on which side of the scheduling
tool/real-time system various features are to be put.  Most of these
I favor putting on the non-real-time side, because it is easier to 
change things there.  For instance
-  Doppler tracking.  (VLA does doppler tracking relatively infrequently
to keep bandpasses stationary.  With digital filter bandpass formation,
this might be unnecessary, so put in the scheduling tools so you can
change your mind easily.)
-  Mosaicing.  (Requires that the tool/real-time interface have separate
specifications for antenna pointing and phase tracking.)  Real-time
system need know nothing about mosaicing, but instead just have the 
capability of changing tracking or phase tracking centers very quickly.
Of course, data reduction systems must be able to figure out what the
scheduling tool had in mind in the way of mosaicing.  (Possibly a mechanism
for passing messages from the scheduling tools to be archived with the
data.)  (I lump with mosaicing scanning in any sort of reasonable or
unreasonable coordinate system.)
-  Ephemerides for solar system bodies.
-  Subarraying.  (This would be a second level of subarraying - the 
array operator must be able to accommodate more than one user on the 
array at once, but an observer may want to be able to divide his antennas
into subarrays as well, for example to track ephemeral phenomena at two
frequencies.)

6.  The real-time system.  The real work of the real-time system is the
control of the hardware according to the script provided.  However, most
of the software of the real-time system will be concerned with human
interfaces.  There are three sets of interfaces needed, for different 
users.  They should be similar enough in style so that a technician 
is not totally at sea if he looks at the operator's display, or if the
array operator looks at a scientist's array.  These interfaces should 
be networked, so that any interface may be invoked from any of a large
number of possibly distant computers.  The three interface systems are
-  The technician's interface.  A separate window for each device in the
system, that enables the technician to examine any monitor point it contains,
and to perform diagnostics on it.
-  The operator's interface.  Lots of information screens showing what 
hardware devices are doing and if they are in good health.  Control screens
for putting devices offline for repair or maintenance, or for swapping
antennas from one observer to another.  Communications facilities for 
talking to technicians or observers, including but not limited to writing
an observing log.
-  Scientist's interface.  Information screens about what is happening on
the array, and what the weather is like.  (Scientist interfaces into data
processing systems are a separate question.)

7.  The data archiving system.  This is also a real-time system.  It's work
is to get the data flowing from the correlator written on a permanent medium.
It must also provide a port that can be tapped to provide data to data
reduction systems in a quasi-real-time mode for the quick-look systems.
As well as archiving the correlator data, it must also archive the 
ancillary and instrumental data coming from sensors on the instrument or
such sources.  An important part of the activity of this system is 
providing locators, so that the data can be found again by the distribution
system.

8.  Quasi-real time systems with feedback.  I primarily have in mind pointing
offset determination, focus measurement, and antenna phase determination 
for phased array operation.  However, it might also be called upon to 
evaluate and adjust parameters for the real-time system task which adjusts 
phase based on radiometry.  Might well also include accumulating calibrator
information for determination of calibrator fluxes and evaluation of 
phase stability.

9.  Data distribution system.  Takes archive data, in whatever format or
medium, and produces a FITS output.  The system needs the capability of
extracting subsets of the data, if the requester doesn't want to handle
the full data set at the moment.  It also probably is the place to put
security provisions, to make sure the requester is authorized to have the
data, and to notify array operations and the proposal PI.  It is unclear 
to me whether the appropriate place for calculating u,v,w is in the 
real-time system, the distribution system, or the data processing system.

10.  Near real-time data reduction system.  Something, probably AIPS++,
to generate a first cut map, for archiving and for sending to the user,
run by array operations.  I picture this being only moderately real-time,
with turnaround of hours to days.  If an observer needs something faster,
he may start his own data reduction system, with its data filler connected
to a distribution system port piped directly out from the archive system.

11.  Observer's data reduction system.  We have little or no input.