Cookbook for Zpectrometer data reduction

revision 2008.03.03 - Andrew Baker (minor edits by Andrew Harris)


Finding a data reduction platform on the Green Bank system

Use one of the machines for data reduction to avoid bogging down the general server.  A list of appropriate machine is on http://www.gb.nrao.edu/pubcomputing/data-reduction.shtml

Starting a GBTIDL session

These instructions assume that you will be working in the aharris account and have a directory
/users/aharris/oct07
that contains copies of the current versions of zpreduce.pro:
cd oct07
source ../wasp2/wasp_start
gbtidl
GBTIDL> .compile zpreduce

Getting data into GBTIDL

Loading Zpectrometer data into GBTIDL is a two-step process. First, the data produced by the Zpectrometer per se must be combined with data from other devices involved in the data acquisition process (e.g., the antenna and the receiver) and "filled" into a single, merged SDFITS (single-dish FITS) file. In parallel, an index file tabulating the key parameters for every line in the SDFITS file is created in the same directory. Second, the contents of the SDFITS file must be loaded into GBTIDL. A detailed description of both steps is available at this web page, but here are the key command sequences:
Online mode
If you are observing at the telescope and new data are arriving in real time: Offline mode
If you are reprocessing old data: There are a few operations that can be performed directly on the filled data (i.e., after the SDFITS file has been created, but before its contents have been read into GBTIDL):

Manipulating scan lists

Once you have your data loaded into GBTIDL (by assumption, in a structure named "rawdata"), you can list its contents by M&C (monitor & control) scan number by typing
GBTIDL> zpmclist, rawdata, outfile='master.scans'
Fields listed include M&C scan number, source, tracking beam, type of scan (TPWR or DOBS), and the first index number (of 4 or 8 total) corresponding to this M&C scan number. If outfile is specified, the listing will be directed to "master.scans" in your local directory. If outfile is not specified, the listing will by default be directed to the screen. Note: as of 2007.10.05, identification of scan type may not work correctly for subreflector-nodded data.

Suppose now that you want to select a set of M&C scans corresponding to observations of a particular source named "SMG" (exact name match) that were done in mode "T" (for TPWR; there is an analogous mode "D" for DOBS). You can create an array of scan numbers on disk by typing

GBTIDL> zpmclist, rawdata, outfile='smg-tp.scans', source='SMG', type='T'
Note that for the current version of the software, the source name should not contain any spaces. If you want to add scan numbers to this list (e.g., because new data have just come in), you can type
GBTIDL> zpaddsca, 'smg-tp.scans', [86,88,90,96]
where the final argument is an array of (integer) scan numbers. This file must exist first (perhaps by copying an existing list, adding one current number and then dropping the rest)!  To drop scan numbers from this list (e.g., because some of the scans in the full set are known to be bad), you can type
GBTIDL> zpsubsca, 'smg-tp.scans', [3,15,120]
where the last argument can again be adapted as necessary. The latter two commands will operate on any preexisting scan list.

Reducing data without subreflector nodding

The first step in reducing any Zpectrometer dataset is to invert the cwsigs dataset used to convert raw lags to observed frequencies. When the SDFITS file is being created, the filler will copy any cwsigs files it encounters (along with ancillary WASPDEFS files) from the data directory into the current working directory. In general, inversion is executed separately for each of the four Zpectrometer "backends", so the sequence of GBTIDL commands for an input cwsigs file "2006_11_17_23:14:31.fits" would be
GBTIDL> zpcwsinv, 0, '2006_11_17_23:14:31.fits'
GBTIDL> zpcwsinv, 1, '2006_11_17_23:14:31.fits'
GBTIDL> zpcwsinv, 2, '2006_11_17_23:14:31.fits'
GBTIDL> zpcwsinv, 3, '2006_11_17_23:14:31.fits'
A choice of svdparam=0.02 is hard-coded into the zpcwsinv routine as of this writing. If you have correct scan lists for your source (say, "smg-tp.scans") and a calibrator (say, "3c84-tp.scans"), then you can derive a spectrum of your source normalized to the spectrum of the calibrator with these three statements:
GBTIDL> tpnods, rawdata, sigsfile='2006_11_17_23:14:31.fits', plotbe='0123', mcfile='smg-tp.scans', smg-on, all, tot
GBTIDL> tpnods, rawdata, sigsfile='2006_11_17_23:14:31.fits', plotbe='0123', mcfile='3c84-tp.scans', 3c84-on, all, tot
GBTIDL> zpdivide, smg-on, 3c84-on, smg-norm, 0.0012
If you wish to specify a list of input scans using an internal IDL array rather than the contents of an external file, then replace, e.g., "mcfile='smg-tp.scans'" with "mcscan=[10,12,14]". The "plotbe" parameter is an input string specifying which of the four Zpectrometer backends are to have their data reduced. The output variables "all" and "tot" contain an array of unaveraged difference spectra (one per number of scans in the input list) and the sum of the weights for the mean difference spectrum (to allow combination with other spectra), respectively. The final parameter taken by zpdivide is a threshold value: if the second spectrum falls below this in some channel, then the ratio spectrum will be set to 0 rather than something very large.

The equivalent commands for reducing "DOBS" observations will be

GBTIDL> pro dinods, rawdata, sigsfile='2006_11_17_23:14:31.fits', plotbe='0123', plotsc='sc', mcfile='smg-di.scans', cal-on-di, smg-on-di, allspec, totwgt, totwgtc
GBTIDL> zpdivide, smg-on-di, 3c84-on, smg-norm, 0.0012
The differences between this command and "tpnods" are that To rebin the output spectra (by the recommended factor of 3) and store them to data containers, from which they can be plotted or written out, type
GBTIDL> zprebin4, smg-norm, temp, 3
GBTIDL> zpmakedc, rawdata, 22, sigsfile='2006_11_17_23:14:31.fits', 0, temp[*,0], smg-dc0
GBTIDL> zpmakedc, rawdata, 22, sigsfile='2006_11_17_23:14:31.fits', 1, temp[*,1], smg-dc1
GBTIDL> zpmakedc, rawdata, 22, sigsfile='2006_11_17_23:14:31.fits', 2, temp[*,2], smg-dc2
GBTIDL> zpmakedc, rawdata, 22, sigsfile='2006_11_17_23:14:31.fits', 3, temp[*,3], smg-dc3
In the zprebin4 command, "3" is the factor by which to rebin, and the spectra for all four backends are rebinned in parallel; a single spectrum can be rebinned with the zprebin1 command. In the zpmakedc commands, "22" is the number of a M&C scan from which the header information (source name, velocity axis, etc.) can be copied in creating the new data container; the digit following the "sigsfile" argument indicates the backend whose velocity axis information should be copied over.

To store the spectra of all four backends to an ascii output file (some columns will be 0 unless "plotbe='0123'"), type

GBTIDL> zpprint4, smg-dc0, smg-dc1, smg-dc2, smg-dc3, outfile='smg-norm.data'
while to direct output spectra to the GBTIDL plotter, type (for example)
set_data_container, smg-dc0
copy, 0, 11
show, 11, color=!white
set_data_container, smg-dc1
copy, 0, 12
oshow, 12, color=!red
set_data_container, smg-dc2
copy, 0, 13
oshow, 13, color=!green
set_data_container, smg-dc3
copy, 0, 14
oshow, 14, color=!blue

Reducing data with subreflector nodding

The key parameter that characterizes the observing configuration for fast subreflector nodding is the number of dumps per (full) nod cycle; this is stored in the variable zpDumpPerNod in the standard Astrid script. Using data taken 2007 October 8 as an example, here is the sequence of steps for calculating and displaying a ratio of binned spectra of W51 and Jupiter:
zpsdload,'T_08OCT2007.raw.zpec.fits',rawdata
tpsubr,14,rawdata,sigsfile='gbt_071005.fits.fits',plotbe='0123',mcscan=[19,20],jupavg,all,totwgt
tpsubr,14,rawdata,sigsfile='gbt_071005.fits.fits',plotbe='0123',mcscan=[32,33,34,35,36,37,38],w51avg,all2,totwgt2
zpdivide,w51avg,jupavg,w51norm,0.0001
zpmakedc,rawdata,32,sigsfile='gbt_071005.fits.fits',0,w51norm[*,0],dc0
zpmakedc,rawdata,32,sigsfile='gbt_071005.fits.fits',1,w51norm[*,1],dc1
zpmakedc,rawdata,32,sigsfile='gbt_071005.fits.fits',2,w51norm[*,2],dc2
zpmakedc,rawdata,32,sigsfile='gbt_071005.fits.fits',3,w51norm[*,3],dc3
clear
show,dc3,color=!blue
oshow,dc2,color=!green
oshow,dc1,color=!red
oshow,dc0,color=!white
freq
setx,25.3,38.5
sety,-0.1,0.5
The syntax of the tpsubr command is similar to that for tpnods, but the first argument is precisely the number of dumps per subreflector nod cycle. The last statements set the x axis units in the plotter to frequency and specify photogenic ranges for plotting in both coordinates; autoscaling of the two axes can be recovered with freex and freey, respectively.