Spitzer/c2d archive usage cookbook


  1. python (it will have to be enabled with the pyfits and numarray modules, and perhaps a few more in the future). Details how to do it, including installing python if you don't have it, are explained in the INSTALL file in the genv/ directory. See also next item. A rough cut-and-pastable session
    #   The examples below follow the csh syntax, and  for convenience, we'll also 
    #   assume the following commands take place in a $c2d directory.
    #   Modify for your machine appropriately.
    #   This script lives as genv/scripts/install_python
    set c2d=`pwd`
    #   you may need to install your own python if you don't have root access to /usr/lib/python2.2
    wget http://www.python.org/ftp/python/2.2.3/Python-2.2.3.tgz
    tar zxf Python-2.2.3.tgz
    cd Python-2.2.3
    ./configure --prefix=$c2d
    make                                            # takes about a minute
    make install
    set path=($c2d/bin $path)                  # add new python to path
    which python                                 # observe the new python location !!!
    #    numarray (the new one)  1.0 is now also available
    #    numeric is the old one, Numeric-23.3.tar.gz being the last one - don't use it
    cd $c2d
    wget http://aleron.dl.sourceforge.net/sourceforge/numpy/numarray-1.0.tar.gz
    tar zxf numarray-1.0.tar.gz
    cd numarray-1.0
    which python                                    # just making sure where it goes
    python setup.py install
    #    pyfits (needs numarray) - notice the hardcoded python version path !!!
    cd $c2d
    wget http://www.stsci.edu/resources/software_hardware/pyfits/current/pyfits.py
    cp pyfits.py $c2d/lib/python2.2/site-packages/
    #    cfitsio (needed by pCFITSIO)
    cd $c2d
    wget ftp://heasarc.gsfc.nasa.gov/software/fitsio/c/cfitsio2500.tar.gz
    tar zxf cfitsio2500.tar.gz
    cd cfitsio
    ./configure --prefix=$c2d
    make install
    #    pCFITSIO (needs cfitsio sources, and numarray)
    cd $c2d
    wget http://panoramix.stsci.edu/~npirzkal/python/pcfitsio/downloads/pCFITSIO-0.99.2.tar.gz 
    tar zxf pCFITSIO-0.99.2.tar.gz
    cd pCFITSIO-0.99.2
    ### alas, you can either edit setup.cfg with the value of $c2d/cfitsio
    ### vi setup.cfg
    python setup.py build_ext --include-dirs=$c2d/include --library-dirs=$c2d/lib
    python setup.py install
    cp pcfitsio.so ../lib/python2.2/site-packages/
  2. genv (see also our SIRTF CVS intro page for more info and links) is our collection of script for data archive manipulation (ingestion, digestion and some new basic viewing)
    	setenv CVSROOT :pserver:$USER@cvs.astro.umd.edu:/home/cvsroot
    		(if you don't have a $USER account with us, use anonymous instead)
    	cvs login
    		(no password, only needed if you have not done this before)
    		(anonymous uses no password, so just hit RETURN)
    	cvs co -d genv SIRTF/genv
    	cd genv
    		(this should also report if you have a proper version of python)
    	source genv.rc
    		(now "genv" has been loaded in your current shell)
    		(read the INSTALL file if you need to update your python installation)
  3. map2 software. Not needed now, but they do have ds9 viewing tools for mosaiced images. We may integrate some of the genv viewing and map2 viewing tools, since both are written in python in simular fashion.

c2d archive access

Once 'genv' has been installed, and you have a correct/working .genv.rc file, and you have setup automated ssh authentication using any of the team (s-umd, s-ut, s-sao,...) archive access can be done via the gget/gput routines:
  more description to follow here:

  gls        list c2d data on the TX archive
  gget	     get an AOR via the c2d index number
  gput	     put your possibly modified AOR back to the TX archive

c2d_findsource: locating and displaying IRAC/MIPS frames in which given RA/DEC is located

c2d_findsource is a python script that makes use of the archiving utilities, and can be used for some elementary IRAC/MIPS data selection and viewing. It is not intended to duplicate things that qlvt (which the IRAC-CC team uses) does.

CAVEAT: using just the corners, and not a full WCS description, can cause some RA/DEC selections to be inside, when they are really outside, and vice versa. A new scheme is being worked out storing a more complete WCS description (sans higher order SSC pixel correction terms that seem to be up to 1/2 pixels correction terms).


  c2d_findsource $C2D_ARCHIVE/.corners.log 203.93 -75.94 1 0 2
  python -i $C2D_ROOT/scripts/c2d_findsource $C2D_ARCHIVE/.corners.log 203.93 -75.94 1 0 0
The arguments are:
  - a corners.log file (the ingestion team makes it, and will not be needed in a next version)
  - RA (decimal for now, H:M:S shortly)
  - DEC (decimal for now, D:M:S shortly)
  - bcd/icd/ecd identification (1=bcd, icd=2, ecd=3)
  - channel (0=all, 1,2,3,4 is the requested channel)
  - view (0=just compute and list, >1 means views this selected frame from the list)

In the first example the command is only run once, which is terribly expensive since reading the big $C2D_ARCHIVE/.corners.log file takes 10-30", depending on your cpu. If you run it in interactive mode, the data will be in memory and much quicker to interact with. An example of the two will illustrate this:

ds9 &                                     <--- you probably want to start ds9 before 

time c2d_findsource $C2D_ARCHIVE/.corners.log 203.93 -75.94 1 0 0        <--- a small benchmark
Found 197652 fields in /sirtf1/ARCHIVE/TX_archive//.corners.log
# Matched 36 fields bcd=1 channel=0
There are 36 frames
11.220u 0.300s 0:11.82 97.4%    0+0k 0+0io 646pf+0w                  <--- takes about 12 seconds on "preakness"

python -i c2d_findsource $C2D_ARCHIVE/.corners.log 203.93 -75.94 1 0 0         <--- the same thing, for interactive work
Found 197652 fields in all-corners.log
# Matched 36 fields bcd=1 channel=0              <--- two python objects, a and b, were  created in your session
There are 36 frames
>>>                                              <--- this is the python prompt
>>> a.show()					 <--- a is a python object, show() the function
ra=13:35:43.200 dec=-75:56:24.000

>>> a.list()

# Found 36 fields for ra=203.93 dec=-75.94
1 /home/teuben/sirtf/data/cham_archive//IRAC/BCD/0005741312/1/SPITZER_I1_5741312_0_0_1_bcd.fits
36 /home/teuben/sirtf/data/cham_archive//IRAC/BCD/0005741312/4/SPITZER_I4_5741312_9_0_1_bcd.fits

>>> a.ds9(4)					<---  show field 4 from this list

>>> a.ds9(4,zoom=5,arcsec=4)					<---  and with different defaults
>>> help(a.ds9)					<---  use python builtin help

>>> a.find(204,-76,channel=3,bcd=2)		<---  select only channel 2 and the ICD
# Matched 6 fields bcd=2 channel=3
[.....]						<---  a python list of 6 filenames
>>> b=a.find(204,-76,channel=3,bcd=2)		<---  do it again
>>> b[2]					<---  list the 3rd one (python starts arrays at 0)

The current implementation of c2d_findsource uses the corners of a field (hence corners.log), which is ok for small fields like IRAC, but can still cause problems for high dec sources where the curvature was ignored. I'm looking into a better WCS based algorithm, but it's bound to be slower.
Last updated: 9-aug-2004 PJT