LMTOY>> seq_pipeline: 2-feb-2022
LMTOY>> READING lmtoy_functions 24-dec-2021 via /lma1/teuben/lmtoy/examples/seq_pipeline.sh
Working directory ./2018-S1-MU-8/91112
LMTOY>> Hang on, creating a bootstrap lmtoy_91112.rc from path=/n/lma1/lmt/data_lmt/
#
# version=2-feb-2022
# ifproc="/n/lma1/lmt/data_lmt//ifproc/ifproc_2020-02-20_091112_00_0001.nc"
date_obs="2020-02-20T01:03:56"
# skytime=6939.72 sec
# inttime=4984.32 sec
obsnum="91112"
calobsnum="91111"
obspgm="Map"
obsgoal="Science"
ProjectId="2018-S1-MU-8"
# SkyOff=720 0
# HPBW=12 arcsec
# XYLength=420 600 arcsec
# XYRamp=48 48 arcsec
# XYoff=0 0 arcsec
# XYstep=1 0.5
numbands=1
vlsr=463 # km/s
skyfreq=115.10110355576857 # GHz
restfreq=115.271204 # Ghz
src="NGC5194"
resolution=11 # arcsec
cell=5.5 # arcsec
x_extent=420 # arcsec
y_extent=600 # arcsec
instrument="SEQ"
#
LMTOY>> this is your startup lmtoy_91112.rc file:
# seq_pipeline: 2-feb-2022
# DATE: 2022-02-03T22:08:35.607882878
# obsnum=91112
# Using ifproc=/n/lma1/lmt/data_lmt//ifproc/ifproc_2020-02-20_091112_00_0001.nc
# path=/n/lma1/lmt/data_lmt/
#
# version=2-feb-2022
# ifproc="/n/lma1/lmt/data_lmt//ifproc/ifproc_2020-02-20_091112_00_0001.nc"
date_obs="2020-02-20T01:03:56"
# skytime=6939.72 sec
# inttime=4984.32 sec
obsnum="91112"
calobsnum="91111"
obspgm="Map"
obsgoal="Science"
ProjectId="2018-S1-MU-8"
# SkyOff=720 0
# HPBW=12 arcsec
# XYLength=420 600 arcsec
# XYRamp=48 48 arcsec
# XYoff=0 0 arcsec
# XYstep=1 0.5
numbands=1
vlsr=463 # km/s
skyfreq=115.10110355576857 # GHz
restfreq=115.271204 # Ghz
src="NGC5194"
resolution=11 # arcsec
cell=5.5 # arcsec
x_extent=420 # arcsec
y_extent=600 # arcsec
instrument="SEQ"
#
# based on vlsr=463, dv=100, dw=250
b_order=0
b_regions=[[113,363],[563,813]]
l_regions=[[363,563]]
slice=[113,813]
v_range=363,563
pix_list=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
rmax=3
otf_a=1.1
otf_b=4.75
otf_c=2
sample=-1
otf_cal=0
edge=0
LMTOY>> Sleeping for 5 seconds, you can abort, edit lmtoy_91112.rc, then continuing
LMTOY>> process_otf_map2 in 2 seconds
{'--b_order': '0',
'--b_regions': '[[200,380],[580,670]]',
'--bank': '0',
'--eliminate_list': '0',
'--help': False,
'--l_regions': '[400,600]',
'--obsnum': '91112',
'--output': 'NGC5194_91112.nc',
'--path': '/n/lma1/lmt/data_lmt/',
'--pix_list': '1,2,3,4,6,7,8,9,10,11,12,13,14,15',
'--restfreq': None,
'--sample': None,
'--save_tsys': False,
'--slice': '[200,800]',
'--stype': '2',
'--tsys': '250.0',
'--use_cal': True,
'--use_otf_cal': False,
'--x_axis': 'VLSR'}
program process_otf_map options
pix_list = [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
eliminate_list = [0]
use_cal = True
b_order = 0
b_regions = [[200, 380], [580, 670]]
l_regions = [400, 600]
slice = [200, 800]
path = /n/lma1/lmt/data_lmt/
output_file_name = NGC5194_91112.nc
use_otf_cal = False
x_axis = VLSR
bank = 0
obsnum = 91112
output = NGC5194_91112.nc
stype = 2
tsys = 250.0
data_path = /n/lma1/lmt/data_lmt/
2020-02-20T01:03:56 begin /n/lma1/lmt/data_lmt/ifproc/ifproc_2020-02-20_091112_00_0001.nc
2020-02-20T02:59:36 end /n/lma1/lmt/data_lmt/ifproc/ifproc_2020-02-20_091112_00_0001.nc
BufPos 1 17.5 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 27.0 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 26.8 sec 91112
BufPos 1 32.3 sec 91112
BufPos 0 26.5 sec 91112
BufPos 1 32.9 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 32.6 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 33.1 sec 91112
BufPos 0 26.6 sec 91112
BufPos 1 32.7 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 31.6 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.9 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 32.7 sec 91112
BufPos 0 26.6 sec 91112
BufPos 1 33.1 sec 91112
BufPos 0 26.0 sec 91112
BufPos 1 31.8 sec 91112
BufPos 0 28.3 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 26.6 sec 91112
BufPos 1 33.2 sec 91112
BufPos 0 28.8 sec 91112
BufPos 1 31.2 sec 91112
BufPos 0 26.1 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 26.1 sec 91112
BufPos 1 31.8 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.3 sec 91112
BufPos 0 26.6 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.6 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.0 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 32.6 sec 91112
BufPos 0 26.0 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 27.0 sec 91112
BufPos 1 32.6 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 32.1 sec 91112
BufPos 0 26.1 sec 91112
BufPos 1 32.1 sec 91112
BufPos 0 26.5 sec 91112
BufPos 1 33.1 sec 91112
BufPos 0 28.3 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 26.5 sec 91112
BufPos 1 31.8 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.5 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.4 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 32.3 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.1 sec 91112
BufPos 0 28.6 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 28.6 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.4 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 34.0 sec 91112
BufPos 0 26.0 sec 91112
BufPos 1 32.7 sec 91112
BufPos 0 26.6 sec 91112
BufPos 1 32.3 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 32.3 sec 91112
BufPos 0 26.0 sec 91112
BufPos 1 32.9 sec 91112
BufPos 0 28.6 sec 91112
BufPos 1 31.2 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 35.8 sec 91112
BufPos 0 26.6 sec 91112
BufPos 1 32.1 sec 91112
BufPos 0 26.1 sec 91112
BufPos 1 33.1 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 33.0 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.2 sec 91112
BufPos 0 26.6 sec 91112
BufPos 1 34.1 sec 91112
BufPos 0 28.0 sec 91112
BufPos 1 32.1 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.1 sec 91112
BufPos 0 28.3 sec 91112
BufPos 1 31.4 sec 91112
BufPos 0 28.4 sec 91112
BufPos 1 32.6 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.5 sec 91112
BufPos 0 28.0 sec 91112
BufPos 1 32.3 sec 91112
BufPos 0 26.6 sec 91112
BufPos 1 32.0 sec 91112
BufPos 0 27.8 sec 91112
BufPos 1 31.1 sec 91112
BufPos 0 26.5 sec 91112
BufPos 1 31.9 sec 91112
BufPos 0 28.0 sec 91112
BufPos 1 32.7 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.7 sec 91112
BufPos 0 26.0 sec 91112
BufPos 1 33.5 sec 91112
BufPos 0 26.5 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 26.5 sec 91112
BufPos 1 33.7 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 31.8 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 33.8 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 28.0 sec 91112
BufPos 1 32.7 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 27.9 sec 91112
BufPos 1 33.4 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.1 sec 91112
BufPos 0 28.0 sec 91112
BufPos 1 32.3 sec 91112
BufPos 0 28.0 sec 91112
BufPos 1 31.3 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.1 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 32.9 sec 91112
BufPos 0 26.1 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 31.8 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 33.0 sec 91112
BufPos 0 26.5 sec 91112
BufPos 1 33.4 sec 91112
BufPos 0 27.8 sec 91112
BufPos 1 31.1 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 33.4 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.3 sec 91112
BufPos 0 27.4 sec 91112
BufPos 1 32.8 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 34.4 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.8 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 32.8 sec 91112
BufPos 0 26.0 sec 91112
BufPos 1 32.9 sec 91112
BufPos 0 27.4 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 28.8 sec 91112
BufPos 1 32.0 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.6 sec 91112
BufPos 0 26.5 sec 91112
BufPos 1 31.7 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 32.8 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.3 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.3 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.7 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 31.9 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.8 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.5 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 33.4 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.3 sec 91112
BufPos 0 27.6 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 27.4 sec 91112
BufPos 1 32.4 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.2 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.0 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.4 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 33.5 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 31.9 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.8 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 31.8 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 33.8 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.9 sec 91112
BufPos 0 26.3 sec 91112
BufPos 1 32.8 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 31.5 sec 91112
BufPos 0 26.2 sec 91112
BufPos 1 32.9 sec 91112
BufPos 0 26.4 sec 91112
BufPos 1 11.8 sec 91112
before read npix
from pixels npix = 16
from xlen npix = 16
Map Parameters: Ra Continuous
HPBW= 12.0 XLength= 420.0 YLength= 600.0 XStep= 1.00 YStep= 0.50Traceback (most recent call last):
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/ifproc/ifproc.py", line 536, in __init__
self.chop = self.nc.variables['Data.Msip1mm.BeamChopperActPos'][:]
KeyError: 'Data.Msip1mm.BeamChopperActPos'
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/ifproc/ifproc.py", line 667, in __init__
self.chop = self.nc.variables['Data.Msip1mm.BeamChopperActPos'][:]
KeyError: 'Data.Msip1mm.BeamChopperActPos'
/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/numpy/core/fromnumeric.py:748: UserWarning: Warning: 'partition' will ignore the 'mask' of the MaskedArray.
a.partition(kth, axis=axis, kind=kind, order=order)
/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/spec/spec.py:547: RuntimeWarning: invalid value encountered in true_divide
self.tsys_spectrum = 280 * sky_spectrum / (hot_spectrum -
/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/spec/spec.py:547: RuntimeWarning: divide by zero encountered in true_divide
self.tsys_spectrum = 280 * sky_spectrum / (hot_spectrum -
/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/spec/spec.py:392: RuntimeWarning: invalid value encountered in true_divide
spectra.append((self.raw_spec[i,:] - ref) / ref)
/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/spec/spec.py:392: RuntimeWarning: divide by zero encountered in true_divide
spectra.append((self.raw_spec[i,:] - ref) / ref)
/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/spec/spec.py:434: RuntimeWarning: invalid value encountered in multiply
self.reduced_spectra = np.array(spectra) * tsys_spectrum
/n/lma1/lmt/data_lmt/ifproc/ifproc_2020-02-20_091112_00_0001.nc does not have bs parameters
91112 is a Map observation
get chop
'Data.Msip1mm.BeamChopperActPos'
no chop
2020-02-20T01:03:30 begin /n/lma1/lmt/data_lmt/ifproc/ifproc_2020-02-20_091111_00_0001.nc
2020-02-20T01:03:37 end /n/lma1/lmt/data_lmt/ifproc/ifproc_2020-02-20_091111_00_0001.nc
BufPos 3 4.4 sec 91111
BufPos 2 2.6 sec 91111
before read npix
from pixels npix = 16
from xlen npix = 16
/n/lma1/lmt/data_lmt/ifproc/ifproc_2020-02-20_091111_00_0001.nc does not have map parameters
/n/lma1/lmt/data_lmt/ifproc/ifproc_2020-02-20_091111_00_0001.nc does not have bs parameters
91111 is a Cal observation
get chop cal
'Data.Msip1mm.BeamChopperActPos'
no chop cal
read_roach /n/lma1/lmt/data_lmt/spectrometer/roach0/roach0_91112_0_1_NGC5194_2020-02-20_060348.nc nspec,nchan=207580,2048
r:0 inp:1 pix:1 time_offset:-0.030000
r:0 inp:2 pix:2 time_offset:-0.030000
r:0 inp:3 pix:3 time_offset:-0.030000
read_roach /n/lma1/lmt/data_lmt/spectrometer/roach1/roach1_91112_0_1_NGC5194_2020-02-20_060348.nc nspec,nchan=208112,2048
r:1 inp:0 pix:4 time_offset:-0.030000
r:1 inp:2 pix:6 time_offset:-0.030000
r:1 inp:3 pix:7 time_offset:-0.030000
read_roach /n/lma1/lmt/data_lmt/spectrometer/roach2/roach2_91112_0_1_NGC5194_2020-02-20_060348.nc nspec,nchan=208768,2048
r:2 inp:0 pix:8 time_offset:-0.030000
r:2 inp:1 pix:9 time_offset:-0.030000
r:2 inp:2 pix:10 time_offset:-0.030000
r:2 inp:3 pix:11 time_offset:-0.030000
read_roach /n/lma1/lmt/data_lmt/spectrometer/roach3/roach3_91112_0_1_NGC5194_2020-02-20_060348.nc nspec,nchan=209244,2048
r:3 inp:0 pix:12 time_offset:-0.030000
r:3 inp:1 pix:13 time_offset:-0.030000
r:3 inp:2 pix:14 time_offset:-0.030000
r:3 inp:3 pix:15 time_offset:-0.030000
RESTFREQ 115.271 GHz
read_roach /n/lma1/lmt/data_lmt/spectrometer/roach0/roach0_91111_0_1_NGC5194_2020-02-20_060329.nc nspec,nchan=228,2048
r:0 inp:1 pix:1 time_offset:-0.030000
r:0 inp:2 pix:2 time_offset:-0.030000
r:0 inp:3 pix:3 time_offset:-0.030000
read_roach /n/lma1/lmt/data_lmt/spectrometer/roach1/roach1_91111_0_1_NGC5194_2020-02-20_060329.nc nspec,nchan=232,2048
r:1 inp:0 pix:4 time_offset:-0.030000
r:1 inp:2 pix:6 time_offset:-0.030000
r:1 inp:3 pix:7 time_offset:-0.030000
read_roach /n/lma1/lmt/data_lmt/spectrometer/roach2/roach2_91111_0_1_NGC5194_2020-02-20_060329.nc nspec,nchan=236,2048
r:2 inp:0 pix:8 time_offset:-0.030000
r:2 inp:1 pix:9 time_offset:-0.030000
r:2 inp:2 pix:10 time_offset:-0.030000
r:2 inp:3 pix:11 time_offset:-0.030000
read_roach /n/lma1/lmt/data_lmt/spectrometer/roach3/roach3_91111_0_1_NGC5194_2020-02-20_060330.nc nspec,nchan=240,2048
r:3 inp:0 pix:12 time_offset:-0.030000
r:3 inp:1 pix:13 time_offset:-0.030000
r:3 inp:2 pix:14 time_offset:-0.030000
r:3 inp:3 pix:15 time_offset:-0.030000
TSYS[1] = 273.263 +/- 48.7576 (2048 channels)
TSYS[2] = 221.602 +/- 13.5308 (2048 channels)
TSYS[3] = 212.105 +/- 12.6954 (2048 channels)
TSYS[4] = 226.898 +/- 15.9192 (2048 channels)
TSYS[6] = 255.03 +/- 20.6341 (2048 channels)
TSYS[7] = 219.999 +/- 13.5357 (2048 channels)
TSYS[8] = 212.025 +/- 13.5689 (2048 channels)
TSYS[9] = 223.186 +/- 12.5194 (2048 channels)
TSYS[10] = 218.49 +/- 13.2839 (2048 channels)
TSYS[11] = 221.354 +/- 14.0549 (2048 channels)
TSYS[12] = 205.61 +/- 11.7105 (2048 channels)
TSYS[13] = 246.224 +/- 12.712 (2048 channels)
TSYS[14] = 232.042 +/- 13.0656 (2048 channels)
TSYS[15] = 231.193 +/- 14.9446 (2048 channels)
Spectral Band velocity range: -576.795 1502.8 km/s
Total Number of OTF Spectra = 340024
NSPEC: 340024
NCHAN: 590
FAST_NC: True
Looping over 14 pixel list [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]:
Processing 1 CAL's for Tsys
Pix Nspec Mean Std MAD_std Min Max RMS_max Warnings
TSYS[1] slice: 278.48 (4.33315) minmax: 270.081 286.518
1 24274 0.005 1.441 1.441 -7.642 7.569 1.441 1.703
TSYS[2] slice: 223.382 (3.22973) minmax: 217.289 229.964
2 24274 0.004 1.155 1.155 -6.379 6.283 1.157 1.395
TSYS[3] slice: 211.083 (3.42892) minmax: 205.146 218.502
3 24274 0.004 1.093 1.093 -6.054 5.850 1.098 1.285
TSYS[4] slice: 227.869 (4.51838) minmax: 219.954 238.054
4 24290 0.004 1.178 1.177 -6.398 6.333 1.184 1.429
TSYS[6] slice: 255.172 (5.47346) minmax: 245.115 265.315
6 24290 0.005 1.319 1.319 -7.170 7.193 1.324 1.545
TSYS[7] slice: 220.02 (4.04182) minmax: 212.161 228.265
7 24290 0.004 1.138 1.138 -6.060 6.179 1.141 1.352
TSYS[8] slice: 211.776 (3.77303) minmax: 204.899 219.586
8 24291 0.005 1.096 1.096 -5.409 6.590 1.102 1.330
TSYS[9] slice: 222.5 (3.34003) minmax: 216.443 229.769
9 24291 0.005 1.152 1.152 -6.128 6.339 1.156 1.378
TSYS[10] slice: 218.124 (4.05311) minmax: 211.181 226.103
10 24291 0.005 1.129 1.129 -6.297 5.965 1.133 1.376
TSYS[11] slice: 221.359 (4.20833) minmax: 213.749 229.827
11 24291 0.005 1.144 1.144 -6.021 6.189 1.147 1.374
TSYS[12] slice: 205.176 (3.1839) minmax: 199.328 212.181
12 24292 0.004 1.062 1.062 -5.883 5.554 1.066 1.276
TSYS[13] slice: 245.509 (4.03596) minmax: 238.037 254.015
13 24292 0.004 1.270 1.269 -6.895 6.847 1.275 1.589
TSYS[14] slice: 231.324 (3.39435) minmax: 225.103 238.356
14 24292 0.005 1.198 1.197 -6.834 6.355 1.202 1.396
TSYS[15] slice: 230.601 (4.06938) minmax: 223.33 239.176
15 24292 0.005 1.194 1.194 -6.524 6.847 1.201 1.420
CPU TIME: 166.202 sec
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/anaconda3/bin/process_otf_map2.py", line 7, in
exec(compile(f.read(), __file__, 'exec'))
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/process_otf_map2.py", line 110, in
main(sys.argv[1:])
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/process_otf_map2.py", line 105, in main
specfile.write_ncdata()
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/spec/specfile.py", line 325, in write_ncdata
self._create_nc_data()
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/spec/specfile.py", line 286, in _create_nc_data
nc_tsys[:,:,:] = tmp_tsys[:,:,:]
File "src/netCDF4/_netCDF4.pyx", line 4918, in netCDF4._netCDF4.Variable.__setitem__
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/netCDF4/utils.py", line 365, in _StartCountStride
datashape = broadcasted_shape(shape, datashape)
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/netCDF4/utils.py", line 973, in broadcasted_shape
return np.broadcast(a, b).shape
ValueError: shape mismatch: objects cannot be broadcast to a single shape
LMTOY>> view_spec_file
{'--help': False,
'--input': 'NGC5194_91112.nc',
'--pix_list': '1,2,3,4,6,7,8,9,10,11,12,13,14,15',
'--plot_range': '-1,3',
'--plots': 'NGC5194_91112_specviews',
'--rms_cut': '-4',
'--show_pixel': None,
'--skip_tsys': False}
SLR Plots initialized with 1 method=NGC5194_91112_specviews
Max sequence=24292
Plots saving in NGC5194_91112_specviews.1.png
Plots saving in NGC5194_91112_specviews.2.png
Plots saving in NGC5194_91112_specviews.3.png
Plots saving in NGC5194_91112_specviews.4.png
Plots saving in NGC5194_91112_specviews.5.png
NEW TSYS PIC
SHAPE (1, 14, 590)
PIXEL: 1 [ 0 1 2 ... 24271 24272 24273]
PIXEL: 2 [24274 24275 24276 ... 48545 48546 48547]
PIXEL: 3 [48548 48549 48550 ... 72819 72820 72821]
PIXEL: 4 [72822 72823 72824 ... 97109 97110 97111]
PIXEL: 6 [ 97112 97113 97114 ... 121399 121400 121401]
PIXEL: 7 [121402 121403 121404 ... 145689 145690 145691]
PIXEL: 8 [145692 145693 145694 ... 169980 169981 169982]
PIXEL: 9 [169983 169984 169985 ... 194271 194272 194273]
PIXEL: 10 [194274 194275 194276 ... 218562 218563 218564]
PIXEL: 11 [218565 218566 218567 ... 242853 242854 242855]
PIXEL: 12 [242856 242857 242858 ... 267145 267146 267147]
PIXEL: 13 [267148 267149 267150 ... 291437 291438 291439]
PIXEL: 14 [291440 291441 291442 ... 315729 315730 315731]
PIXEL: 15 [315732 315733 315734 ... 340021 340022 340023]
Plots saving in NGC5194_91112_specviews.6.png
/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/numpy/core/fromnumeric.py:748: UserWarning: Warning: 'partition' will ignore the 'mask' of the MaskedArray.
a.partition(kth, axis=axis, kind=kind, order=order)
SLR Plots initialized with 3 method=NGC5194_91112_specpoint,png,1
MAD: 1 1.4403477 0.06280940412366087 1.6915852880034326
MAD: 2 1.1571515 0.05024266295010695 1.3581221124480107
MAD: 3 1.0977861 0.0476322137829355 1.2883149240480627
MAD: 4 1.183182 0.0514825821195241 1.389112329591988
MAD: 6 1.3229907 0.05763525350798533 1.5535316699309891
MAD: 7 1.140969 0.05009252235650423 1.3413391274356605
MAD: 8 1.101436 0.0486367153294229 1.2959828802614783
MAD: 9 1.156172 0.050448476630213325 1.3579659436454872
MAD: 10 1.132325 0.049162870181871744 1.328976533942514
MAD: 11 1.1465492 0.05010153609431911 1.346955369230792
MAD: 12 1.0658963 0.046797382595316385 1.2530858030405674
MAD: 13 1.2746544 0.055587102514500816 1.4970027984857377
MAD: 14 1.2018313 0.05294881680407754 1.413626608006105
MAD: 15 1.2001882 0.05255389136981589 1.4104037254218906
Plots saving in NGC5194_91112_specpoint.1.png
/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/numpy/core/fromnumeric.py:748: UserWarning: Warning: 'partition' will ignore the 'mask' of the MaskedArray.
a.partition(kth, axis=axis, kind=kind, order=order)
SLR Plots initialized with 3 method=NGC5194_91112_specpoint,png,2
MAD: 1 1.4403477 0.06280940412366087 1.6915852880034326
MAD: 2 1.1571515 0.05024266295010695 1.3581221124480107
MAD: 3 1.0977861 0.0476322137829355 1.2883149240480627
MAD: 4 1.183182 0.0514825821195241 1.389112329591988
MAD: 6 1.3229907 0.05763525350798533 1.5535316699309891
MAD: 7 1.140969 0.05009252235650423 1.3413391274356605
MAD: 8 1.101436 0.0486367153294229 1.2959828802614783
MAD: 9 1.156172 0.050448476630213325 1.3579659436454872
MAD: 10 1.132325 0.049162870181871744 1.328976533942514
MAD: 11 1.1465492 0.05010153609431911 1.346955369230792
MAD: 12 1.0658963 0.046797382595316385 1.2530858030405674
MAD: 13 1.2746544 0.055587102514500816 1.4970027984857377
MAD: 14 1.2018313 0.05294881680407754 1.413626608006105
MAD: 15 1.2001882 0.05255389136981589 1.4104037254218906
Plots saving in NGC5194_91112_specpoint.2.png
LMTOY>> make_spec_fits (waterfall)
pix [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
Min nsamp 24274
Binning [1, 1]
FITS file will be 24274 x 590 x 14
Written waterfall cube to NGC5194_91112.wf.fits
pix [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
Min nsamp 24274
Binning [10, 1]
FITS file will be 2427 x 590 x 14
New shape: (14, 590, 2427)
Written waterfall cube to NGC5194_91112.wf10.fits
LMTOY>> grid_data
{'--cell': '6.25',
'--edge': '0',
'--help': False,
'--input': 'NGC5194_91112.nc',
'--n_samples': '256',
'--noise_sigma': '1',
'--otf_a': '1.1',
'--otf_b': '4.75',
'--otf_c': '2',
'--otf_select': '1',
'--output': 'NGC5194_91112.fits',
'--pix_list': '1,2,3,4,6,7,8,9,10,11,12,13,14,15',
'--program_path': 'spec_driver_fits',
'--resolution': '12.5',
'--rmax': '4',
'--rms_cut': '-4',
'--sample': '-1',
'--weight': 'NGC5194_91112.wt.fits',
'--x_extent': '550',
'--y_extent': '550'}
STDOUT ****************************
spec_driver_fits LMTSLR 26-jan-2022
CMD: spec_driver_fits -i NGC5194_91112.nc -o NGC5194_91112.fits -w NGC5194_91112.wt.fits -l 12.5 -c 6.25 -u [1,2,3,4,6,7,8,9,10,11,12,13,14,15] -z -4 -s 1 -x 550 -y 550 -f 1 -r 4 -n 256 -0 1.1 -1 4.75 -2 2 -b -1 -e 0
file list=NGC5194_91112.nc
DEBUG:6.25
sample_list=-1
Warning: sample list is not a multiple of 3
nsamples=1
Processing 1 SpecFiles:
Opening SpecFile file NGC5194_91112.nc
file: NGC5194_91112.nc nspec= 340024 nchan= 590
SpecFile version 6-mar-2021
X-range: -308.02 298.975 Y-range: -397.885 397.905 arcsec
X-ramp: -224.62 215.575 Y-ramp: -314.485 314.505 arcsec
MapSize: 606.995 x 795.79 arcsec
91112
NGC5194
DATE-OBS 2020-02-20T01:03:56
CF.n_cells= 9 delta=0.196078 cell=6.25 resol=12.5 oft_select=1
C-AXIS 2 starts at 799.778
C-AXIS 1 starts at -550
C-AXIS 0 starts at -550
P-AXIS 1 starts at -550
P-AXIS 0 starts at -550
P-AXIS 1 starts at -550
P-AXIS 0 starts at -550
WARNING fuzzy_edge=0
pixel: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
used?: 0 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1
rms_stats: n=340024 rms_cutoff=-4
Cube Completed, 339976/340024 Spectra accepted = 1.000
Weighting Completed, fuzzy_edge=0
Weight at 0.000000 0.000000 is 44.794239
write to NGC5194_91112.fits
write weights to NGC5194_91112.wt.fits
STDERR ****************************
Exit Code: 0
LMTOY>> Some NEMO post-processing
### nemo Debug Info: FITS file: Image size 177 177 590
### Warning [fitsccd]: Setting map minimum from 0 to -6.34127
### Warning [fitsccd]: Setting map maximum from 0 to 7.25824
### nemo Debug Info: There were 11129760 blank values in the image (60.212583 %)
### nemo Debug Info: FITS file: Image size 177 177
### Warning [fitsccd]: Setting map minimum from 0 to -1.29433
### Warning [fitsccd]: Setting map maximum from 0 to 51.3343
### nemo Debug Info: There were no blank values set in the image
LMTOY>> STATS NGC5194_91112.ccd centerbox robust
### Warning [ccdsub]: Sampling will be done
### Warning [ccdsub]: Attempting to fix the WCS
### nemo Debug Info: WCS Corner: 202.395 47.2698 799778
Number of points : 4779000
Min and Max : -1.274614 1.572847
Mean and dispersion : 0.005308 0.160451
Skewness and kurtosis : 0.107227 1.027307
Sum and Sum*Dx*Dy*Dz* : 25368.444142 77.679918
Points per beam (map) : 1
N Robust : 4707181
Mean Robust : 0.004220
Sigma Robust : 0.148519
Median Robust : 0.003816
0/4779000 out-of-range points discarded
LMTOY>> STATS NGC5194_91112.wt.ccd centerbox robust
### Warning [ccdsub]: Sampling will be done
### Warning [ccdsub]: Attempting to fix the WCS
### nemo Debug Info: WCS Corner: 202.395 47.2698 0
Number of points : 8100
Min and Max : 9.249841 51.334267
Mean and dispersion : 38.399908 8.449006
Skewness and kurtosis : -1.889993 2.538776
Sum and Sum*Dx*Dy* : 311039.253934 -0.937498
Points per beam (map) : 1
N Robust : 6382
Mean Robust : 42.326941
Sigma Robust : 1.535881
Median Robust : 42.445847
0/8100 out-of-range points discarded
### nemo Debug Info: Keeping 177*177*590 cube
### Warning [ccdfits]: Replacing header from NGC5194_91112.wt.fits
SLABS: [[200,380],[580,670]] == 200,380,580,670
### Warning [ccdslice]: axis=1 not fully supported yet
### nemo Debug Info: Axis: 590 0 799778 -1015.92
### nemo Debug Info: Axis min/max: 799778 201400
### nemo Debug Info: slab 200 380 -> 590 413
### nemo Debug Info: slab 580 670 -> 216 127
### nemo Debug Info: Total planes: 268
### Warning [ccdslice]: Code not converted to fix reference pixel value
### Warning [ccdfits]: Replacing header from NGC5194_91112.wt.fits
### Warning [ccdfits]: Replacing header from NGC5194_91112.wt.fits
### Warning [ccdfits]: Replacing header from NGC5194_91112.fits
[0;31mQAC_STATS: NGC5194_91112-full 0.00333695 0.237399 -6.34127 7.25824 24541.1 0.0210151[0m
### Warning [ccdsub]: Sampling will be done
### Warning [ccdsub]: Attempting to fix the WCS
### nemo Debug Info: WCS Corner: 202.395 47.2698 799778
[0;31mQAC_STATS: NGC5194_91112-cent 0.00530832 0.160451 -1.27461 1.57285 25368.4 0.0424567[0m
### nemo Debug Info: FITS file: Image size 177 177 590
### Warning [fitsccd]: Setting map minimum from 0 to -0.0739696
### Warning [fitsccd]: Setting map maximum from 0 to 0.650974
### nemo Debug Info: There were no blank values set in the image
spectab : # 88 88 590 min/mean/sig/max -0.421156 0.0599745 0.17259 0.729372 0.00022734 0.187898 0.769826
specstab: # 88 88 590 min/mean/sig/max -0.0468879 0.0552772 0.0971416 0.352116 -1.03647e-05 0.00630875 0.0459222
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/examples/fitsplot.py", line 21, in
f = aplpy.FITSFigure(fitsfile)
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/decorators.py", line 28, in wrapper
return f(*args, **kwargs)
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/core.py", line 232, in __init__
self._get_hdu(data, hdu, north, convention=convention,
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/core.py", line 309, in _get_hdu
raise IOError("File not found: " + filename)
OSError: File not found: NGC5194_91112.mom0.fits
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/examples/fitsplot.py", line 21, in
f = aplpy.FITSFigure(fitsfile)
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/decorators.py", line 28, in wrapper
return f(*args, **kwargs)
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/core.py", line 232, in __init__
self._get_hdu(data, hdu, north, convention=convention,
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/core.py", line 309, in _get_hdu
raise IOError("File not found: " + filename)
OSError: File not found: NGC5194_91112.mom1.fits
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/examples/fitsplot.py", line 21, in
f = aplpy.FITSFigure(fitsfile)
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/decorators.py", line 28, in wrapper
return f(*args, **kwargs)
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/core.py", line 232, in __init__
self._get_hdu(data, hdu, north, convention=convention,
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/core.py", line 309, in _get_hdu
raise IOError("File not found: " + filename)
OSError: File not found: NGC5194_91112.mom2.fits
INFO: Auto-setting vmin to -4.993e+00 [aplpy.core]
INFO: Auto-setting vmax to 5.118e+01 [aplpy.core]
INFO: Auto-setting vmin to -5.060e+00 [aplpy.core]
INFO: Auto-setting vmax to 5.150e+01 [aplpy.core]
Writing NGC5194_91112.wt.png
INFO: Auto-setting vmin to -6.210e+00 [aplpy.core]
INFO: Auto-setting vmax to 6.893e+01 [aplpy.core]
INFO: Auto-setting vmin to -6.183e+00 [aplpy.core]
INFO: Auto-setting vmax to 6.863e+01 [aplpy.core]
Writing NGC5194_91112.wt2.png
INFO: Auto-setting vmin to -3.001e+01 [aplpy.core]
INFO: Auto-setting vmax to 3.332e+02 [aplpy.core]
INFO: Auto-setting vmin to -1.751e+01 [aplpy.core]
INFO: Auto-setting vmax to 1.944e+02 [aplpy.core]
Writing NGC5194_91112.wt3.png
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/examples/fitsplot.py", line 21, in
f = aplpy.FITSFigure(fitsfile)
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/decorators.py", line 28, in wrapper
return f(*args, **kwargs)
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/core.py", line 232, in __init__
self._get_hdu(data, hdu, north, convention=convention,
File "/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/aplpy/core.py", line 309, in _get_hdu
raise IOError("File not found: " + filename)
OSError: File not found: NGC5194_91112.wtn.fits
INFO: Auto-setting vmin to -1.194e-01 [aplpy.core]
INFO: Auto-setting vmax to 1.325e+00 [aplpy.core]
INFO: Auto-setting vmin to -1.181e-01 [aplpy.core]
INFO: Auto-setting vmax to 1.311e+00 [aplpy.core]
Writing NGC5194_91112.wtr.png
WARNING: FITSFixedWarning: RADECSYS= 'FK5 ' /
the RADECSYS keyword is deprecated, use RADESYSa. [astropy.wcs.wcs]
INFO: Setting slices=[0] [aplpy.core]
INFO: Auto-setting vmin to -1.231e+04 [aplpy.core]
INFO: Auto-setting vmax to 3.386e+04 [aplpy.core]
INFO: Auto-setting vmin to -1.192e+04 [aplpy.core]
INFO: Auto-setting vmax to 3.488e+04 [aplpy.core]
Writing NGC5194_91112.mom0.png
LMTOY>> Created NGC5194_91112.nf.fits and NGC5194_91112.nfs.fits
LMTOY>> ADMIT post-processing
LMTOY>> lmtoy_admit: 31-oct-2021
/lma1/teuben/lmtoy/examples/lmtoy_admit.sh: line 33: export: `NGC5194_91112.nfs.fits': not a valid identifier
No PDIR directory used, all work in the current directory
ADMIT = /lma1/teuben/lmtoy/admit
version = 1.0.8.5
CASA_PATH = /n/astromake/opt/casa/58 linux admit lma.astro.umd.edu
CASA_ROOT = /n/astromake/opt/casa/58
prefix = /an/terra_astromake/opt/casa/casa-release-5.8.0-109.el7
version = 5.8.0-rel-109
revision = 1
Trying CASA6, no worries if this fails
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named casatools
LMTOY>> ADMIT post-processing
Found 33 Xvfb
1343197 Sl Xvfb
1342272 Sl Xvfb
1341515 Sl Xvfb
1261803 Sl Xvfb
### (1/1) : processing NGC5194_91112.nfs.fits
NGC5194_91112.nfs.fits [177, 177, 590] NGC5194 202.4696 47.19517 0.0007997778 0.069
/lma1/teuben/lmtoy/admit/admit/test/admit1.py --basename x NGC5194_91112.nfs.fits
82.108u 2.812s 1:37.08 87.4% 0+0k 864+461352io 0pf+0w
Logfile: NGC5194_91112.nfs.fits.log
LMTOY>> lmtoy_admit: 31-oct-2021
/lma1/teuben/lmtoy/examples/lmtoy_admit.sh: line 33: export: `NGC5194_91112.nf.fits': not a valid identifier
No PDIR directory used, all work in the current directory
ADMIT = /lma1/teuben/lmtoy/admit
version = 1.0.8.5
CASA_PATH = /n/astromake/opt/casa/58 linux admit lma.astro.umd.edu
CASA_ROOT = /n/astromake/opt/casa/58
prefix = /an/terra_astromake/opt/casa/casa-release-5.8.0-109.el7
version = 5.8.0-rel-109
revision = 1
Trying CASA6, no worries if this fails
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named casatools
LMTOY>> ADMIT post-processing
Found 33 Xvfb
1344686 Sl Xvfb
1343197 Sl Xvfb
1342272 Sl Xvfb
1341515 Sl Xvfb
### (1/1) : processing NGC5194_91112.nf.fits
NGC5194_91112.nf.fits [177, 177, 590] NGC5194 202.4696 47.19517 0.0007997778 0.069
/lma1/teuben/lmtoy/admit/admit/test/admit1.py --basename x NGC5194_91112.nf.fits
65.344u 2.481s 1:19.36 85.4% 0+0k 312+397344io 0pf+0w
Logfile: NGC5194_91112.nf.fits.log
LMTOY>> Created NGC5194_91112.fits and NGC5194_91112.wt.fits
LMTOY>> Parameter file used: lmtoy_91112.rc
LMTOY>> Making summary index.html:
Making index.html for obsnum=91112
NGC5194_91112_?
NGC5194_91112_?_specviews
NGC5194_91112_?_specpoint
Writing index.html
Writing index_pipeline.html
cp: cannot stat 'NGC5194_91112_?_specviews.1.png': No such file or directory