Discarding bad time dumps
Keywords: mrtcal-prog-manual

Next: Dump cycle identification Up: Reading the DATA column Previous: Reading the DATA column Contents

#
Discarding bad time dumps

The BackendDATA table provides the data dumps collected during the subscan integration. In particular, it shows the dump `MJD`, its integration time `INTEGTIM`, its phase number through cycles (`ISWITCH`) and the data itself (channels) in the `DATA` column.

It may happen that a dump is flagged out by the control system. In this case, its `ISWITCH` value is 0. Such a dump should be discarded during the calibration process. This is achieved with the command `MSET CALIBRATION BAD YES|NO` (default `NO`, i.e. discard bad time dumps).

In practice, this is not just a matter of skipping undesired rows when dealing with the `DATA` column. The first issue is that the values in the associated columns are unreliable. This is particularly true with the `MJD` value which is not set to the actual MJD of the dump. This introduces inconsistent values in this column, resulting in an unsorted column, breaking the dichotomic search engine. In order to solve this issue, MRTCAL compresses at read time the `MJD`, `INTEGTIM`, and `ISWITCH` columns, discarding the rows where `ISWITCH` is null. The size of those columns is decreased from to (with ).

MRTCAL keeps also track of the discarded dumps thanks to two new columns added. One column named `FOREPOIN` (size ) provides a forward pointer, i.e. `FOREPOIN(i)` is the position of the dump (as found in the original columns) in the compressed columns. If the dump is bad, the corresponding position is set to 0 (i.e. dump not available in the compressed columns). It also adds a column named `BACKPOIN` (size ) which provides a backward pointer: `BACKPOIN(j)` is the position of the dump (as found in the compressed columns) in the original columns.

At this stage, MRTCAL has patched the BackendDATA columns so that the bad dumps are removed, but this is not fully transparent for later use.

Next: Dump cycle identification Up: Reading the DATA column Previous: Reading the DATA column Contents