Go Terps!
The Maryland-Magellan Tunable Filter
printable version
Data Reduction

The following sections describe the MMTF pipeline for reducing IMACS data. This collection of perl-driven IRAF scripts is a work in progress and still have not been tested exhaustively - they should be used with care. Please contact Michael McDonald with comments, questions, or bug reports.

Download the script package v1.4 (post April '08 data), or package v1.1 (pre April '08 data)

Download the appropriate optical axis file

Pipeline:

Additional tasks:

The data from IMACS emerge as 8 individual files for each exposure, one file per chip. The bias subtraction, flatfielding, cosmic ray masking, sky subtraction, image registration and combiningg steps described below operate on these individual files.

Bias Subtraction and Flatfielding

The bias subtraction and flat fielding steps are straightforward. To use the script, make a list of bias files, one per line, disregarding the chip and ".fits" labels. (Alternatively, copy the bias files from a previous reduction into the current directory and select the "--bias" switch.) In a second file, make a list of each source file, one per line. On each line, the source file is followed by a comma-separated list of flats to be combined and applied to that exposure. Alternatively, one may leave a blank, in which the flats default to those from the previous exposure.

For example:

Contents of file "bias.lst":

ccd0001
ccd0002
ccd0003
...
ccd0010

Contents of file "objfl.lst":

ccd0011 ccd0090,ccd0091,ccd0092
ccd0012
ccd0013 ccd0093,ccd0094,ccd0095
ccd0014

In this case, frame 0011 and 0012 share the same flats, as do ccd0013 and ccd0014. The script is then run:

% imacsbf bias.lst objfl.lst

This produces an IRAF script, which can then be run from the IRAF prompt. The following steps are performed by the IRAF script:

  1. The row overscan is subtracted from all files.
  2. The biases are combined. The combined bias has filenames biasc1.fits, biasc2.fits, etc.
  3. The combined bias is subtracted from each source and flatfield file.
  4. The column overscan is subtracted.
  5. The flats are combined. The combined flats have filenames flat1c1.fits, flat1c2.fits, . . . , flat2c1.fits, etc.
  6. Each flat is normalized so that the central region of the FOV has an average value ~1. The same normalization factor is applied to each chip.
  7. If specified, scattered light signatures are removed from the flat field.
  8. The source files are flatfielded.
Cosmic Ray and Bad Pixel Masking

Cosmic ray removal is achieved in two stages. First, we attempt to remove cosmic rays for each image individually. This is done using Pieter van Dokkum's LA Cosmic task for IRAF (for details, see: LA-Cosmic). This task is called in the imacscr script, which has switches to allow control of such things as the sky level and the threshold sigma for rejection. Generally, four iterations is enough to remove the majority of the cosmic rays in each field. A typical run of imacscr is as follows:

Contents of file "mask.lst":

fccd0011
fccd0012
fccd0013
fccd0014
...

Then run the following command:

 % imacscr mask.lst 

This will produce an IRAF script (crremove.cl) which should be run from the IRAF prompt. The following steps will be performed by crremove.cl:

  1. The LA Cosmic task will be loaded into IRAF using the stsdas.task command and the default values such as readnoise and gain for the IMACS camera will be set.
  2. LA Cosmic is run on each of the 8 chips for each image prefix.
  3. Known bad pixel regions (e.g. bad columns) are appended to the cosmic ray masks, which are saved as msk*.fits

It is important to note a few things about this routine. First, LA Cosmic runs best on images that have not had the sky subtracted. We therefore recommend running imacscr immediately after imacsbf. However, if you have already subtracted the sky for some reason, you can specify roughly what the sky was using the --sky=[value] switch. Secondly, imacscr will produce output images with cosmic rays removed, saved as cr*.fits. These images are useful to check how well LA Cosmic is working, but will not be used by our pipeline (more on that later!). Finally, although a very effective task, LA Cosmic tends to leave residual bright pixels unmasked (although very rarely does it miss a cosmic ray completely!). This is remedied by using the IRAF "crgrow" task later on to expand the masks by a single pixel in each direction.

Sky Subtraction

Sky subtraction is performed using the MMTF calibration software. Given knowledge of the mapping of the CCD pixels to the focal plane and the location of the optical axis, the sky spectrum is azimuthally averaged. Sources and cosmic rays are filtered out using a biweight statistic. If necessary, median smoothing is employed to smooth out the sky spectrum. This results in a robust removal of the sky using a single command.

To subtract sky from one or more exposures, create a file containing, on each line, the exposure name, the (approximate) filter wavelength, and the optical axis coordinates (chip #, column #, row #). For instance:

Contents of file "skysub.lst":

fccd0011  6600  5 -12 4056
fccd0012  6600  5 -12 4056
fccd0013  8150  5 -12 4056
fccd0014  8150  5 -12 4056
...

Then run the following command:

% ringbatch --list=skysub.lst --sub --ms=1

For single exposures, one may simply run the RING command:

% ring ccd0011 6600 --sub --ms 1

This procedure runs non-interactively. On a multi-threaded 3 GHz PC processor running Linux, with 1.5 Gb of memory, the sky subtraction takes approximately 30-45 minutes per exposure (for unbinned data). The result is a series of files with the "ss" suffix prepended. The default FOV has a radius of 4000 pixels in the focal plane; pixels outside this radius do not have the sky subtracted from them. These pixels can be masked using the procedure described below.

If the optical axis is not accurate to within ~20 pixels or so, dipole residuals may appear after sky subtraction, either in individual exposures or in a stack of several exposures with small dithers. If the etalon is properly parallelized, the optical axis estimate provided by the instrument scientist should be correct at this level. In the event that it is not, we recommend that the observer re-measure the optical axis using the actual science data and bright stars in the field of view, following this procedure (except that the mask image/primary ghost pairs are in this case stellar image/ghost pairs). Note that the stellar ghosts will be out-of-focus.

For exposures with large, extended sources in the field of view, these sources may have to be masked before accurate subtraction can be performed. Using the --skip flag, the user can ignore specific chips in the azimuthal calculation. If an extended source is at the center of the field of view, the observer must extrapolate the sky spectrum (as output by the RING/RINGBATCH procedures) into the central regions. The user may modify the spectrum by changing the biweight values in the *.rings output file, using the good part of the spectrum as a guide. This modified file (which must have the same format as the original) can then be put back into RING using the "--spec" command-line switch. (This feature is not currently available under RINGBATCH.)

Astrometry

IMACS Distortion Errors

Median astrometry errors for the new IMACS camera. The upper panel shows absolute errors (in arcseconds) as a function of pixel position, while the lower panel shows the magnitude (multiplied by 500) and direction of the errors as a function of pixel position. These fields were constructed using 8000+ stars over several different pointings.

Perhaps the most challenging aspect of IMACS data reduction is determining the astrometric solution that maps the physical CCD coordinates to locations on the sky. Because of the small f-ratio of the IMACS camera (f/2) when the MMTF is in use, as well as the wide field of view, significant deviations from a linear mapping occur. If your science data occupies only a small, central region of the FOV, standard astrometric techniques may be applied on a chip-by-chip basis. However, to rectify the whole field, we suggest the following procedure.

The higher-order polynomial terms of the FOV-to-sky mapping are known from observations of dense star fields. We provide a script that applies this mapping to any IMACS data, using estimates of the tangent point and sky position angle from the FITS headers. This script, imacswcs, is executed in the following way:

Contents of file "wcs.lst":

ssfccd0011
ssfccd0012
ssfccd0013
ssfccd0014
...

Then run the following command:

 % imacswcs --list=wcs.lst 

For single exposures, one may simply use the following syntax:

 % imacswcs ssfccd0011 

This will produce an IRAF script (imacswcs_wcs.lst.cl) which should be run from the IRAF prompt. When executed, this script will apply the WCS information to all 8 chips for each exposure. This information contains the high-order distortions in the chips as well as the accurate chip spacings and the rotation and scale of the field of view. This information, which is only stored in the header for the time being, will be applied in the next step by the imacsreg script.

Registration and Image Mosaicing

In order to properly register each field of view, imacsreg must be run. This script uses the high-order solution provided by imacswcs and applies various IRAF tasks to fine-tune this solution on a case-by-case basis. The script is run as follows:

Contents of file "reg.lst":

ssfccd0011
ssfccd0012
ssfccd0013
ssfccd0014
...

Then run the following command:

 % imacsreg reg.lst 

This will produce an IRAF script (register.cl) which should be run from the IRAF prompt. The following steps will be performed by register.cl:

  1. Cosmic ray masks are extended by one pixel in each direction to prevent ringing caused by residual cosmic rays in the interpolation steps.
  2. Masked pixels in the input files (e.g. ssfccd0011) are replaced by 0.0 (the sky value) to prevent blurring of cosmic rays. These values will not play a role in the final image, since we will be ignoring contribution from masked images. The only reason for this coarse interpolation is to prevent ringing in the next step.
  3. The IRAF task "mscred.mscimage" is run on each individual chip in order to apply the high-order TNX corrections. The bad pixel masks undergoes the same transformation as the image. This task is run on each chip individually because "mscimage" will not apply TNX corrections on mosaiced images (this is a known IRAF issue).
  4. The chips are combined into a composite MEF file and "mscimage" is run a second time to create a single file with a single bad pixel mask including cosmic ray, bad column and chip gap locations.
  5. A catalogue of stellar positions is created using the IRAF task "mscred.mscgetcat". Stars falling anywhere within roughly 3500 pixels from the center and with magnitudes ranging from 19-10 are used.
  6. The IRAF task "mscred.msccmatch" is used to match the stellar positions to pixel positions in order to determine the lower-order corrections for the mosaicced image. These corrections will be different from exposure to exposure due to various atmospheric effects and pointing errors. These small corrections are also applied to the bad pixel masks.
  7. Using the catalogue of stellar positions, a catalogue of PSFs are made for each field of view. This will be used in the combining stage when PSF matching is required.
PSF-Matching and Stacking

Before combining dithered exposures into a single, deep image imacscombine will PSF-match and align the image groups. The script operates on several groups of images, each group occupying a single line and separated by commas. imacscombine, which is executed once imacsreg has produced mosaiced images with accurate astrometry, is run as follows:

Contents of file "combine.lst":

aassfccd0011,aassfccd0012,aassfccd0013,aassfccd0014
aassfccd0021,aassfccd0022,aassfccd0023,aassfccd0024,aassfccd0025
aassfccd0029,aassfccd0030,aassfccd0031
...

Then run the following command:

 % imacscombine combine.lst 

This will produce an IRAF script (combine.cl) which should be run from the IRAF prompt. The following steps will be performed by combine.cl:

  1. Using the PSF catalogues produced by imacsreg, the mode seeing value is computed for each exposure and inserted into the header.
  2. The images are aligned based on the WCS information. These offsets account for any dithering or pointing errors. Bad pixel masks are also shifted accordingly.
  3. All similar frames are degraded to the worst seeing using the iraf task "psfmatch". **NOTE** If one image has anomolously bad seeing, it should be removed from the list to avoid degrading the images too far.
  4. Similar frames are stacked using the "imcombine" task in IRAF. The process of combining images will ignore masked pixels and will also reject all values above a certain threshold (default=2.0 sigma), thus removing any residual cosmic rays.
Converting to Multi-Extension FITS (MEF) Files

The routine provided creates an IRAF script that will convert the eight individual files in an exposure, one per chip, into a single multi-extension FITS file. Such files are useful for input to, e.g., the IRAF mosaic reduction (MSCRED) package. MSCRED can, among other things, display the entire mosaic using the task MSCDISPLAY, or refine the image WCS for astrometric calibration.

Radial Masks

Radial masks can be created using the task RADMASK, part of the MMTF calibration software. This procedure assumes knowledge of the optical axis. The masks created can have any inner or outer radii, and could be used for cosmetic masking of the outer, unilluminated parts of the field of view or to mask an extended object centered in the FOV.

Optical Axis File (optaxis.dat)
The position of the optical axis can vary slightly with each mounting of the MMTF in the IMACS disperser wheel. This table contains measurements of the optical axis position which are required to properly subtract the sky. For a description of how to measure the optical axis position, see this page.
Date Created Optical Axis File
Mar 2010 optaxis_0310.dat
Nov 2009 optaxis_1109.dat
Jul 2009 optaxis_0709.dat
May 2008 optaxis_0508.dat
Dec 2008 optaxis_1208.dat
Nov 2007 optaxis_1107.dat