LMTOY>> seq_pipeline: 17-dec-2021
LMTOY>> READING lmtoy_functions 17-dec-2021 via /lma1/teuben/lmtoy/examples/seq_pipeline.sh
Working directory ./2018S1SEQUOIACommissioning/92996
LMTOY>> Hang on, creating a bootstrap lmtoy_92996.rc from path=/n/lma1/lmt/data_lmt/
#
# version=17-dec-2021
# ifproc="/n/lma1/lmt/data_lmt//ifproc/ifproc_2021-12-17_092996_00_0001.nc"
date_obs="2021-12-17T02:01:17"
# skytime=146.216 sec
# inttime=80.04 sec
obsnum="92996"
calobsnum="92995"
obspgm="Map"
ProjectId="2018S1SEQUOIACommissioning"
# SkyOff=199.98 0
# HPBW=14 arcsec
# XYLength=120 120 arcsec
# XYRamp=0 0 arcsec
# XYoff=0 0 arcsec
# XYstep=2 0.45
vlsr=-5 # km/s
skyfreq=86.2393 # GHz
restfreq=86.2434 # Ghz
src="Ori-KL"
resolution=15 # arcsec
cell=7.5 # arcsec
x_extent=120 # arcsec
y_extent=120 # arcsec
instrument="SEQ"
#
LMTOY>> this is your startup lmtoy_92996.rc file:
# seq_pipeline: 17-dec-2021
# DATE: 2021-12-20T07:00:36.150785299
# obsnum=92996
# Using ifproc=/n/lma1/lmt/data_lmt//ifproc/ifproc_2021-12-17_092996_00_0001.nc
# path=/n/lma1/lmt/data_lmt/
#
# version=17-dec-2021
# ifproc="/n/lma1/lmt/data_lmt//ifproc/ifproc_2021-12-17_092996_00_0001.nc"
date_obs="2021-12-17T02:01:17"
# skytime=146.216 sec
# inttime=80.04 sec
obsnum="92996"
calobsnum="92995"
obspgm="Map"
ProjectId="2018S1SEQUOIACommissioning"
# SkyOff=199.98 0
# HPBW=14 arcsec
# XYLength=120 120 arcsec
# XYRamp=0 0 arcsec
# XYoff=0 0 arcsec
# XYstep=2 0.45
vlsr=-5 # km/s
skyfreq=86.2393 # GHz
restfreq=86.2434 # Ghz
src="Ori-KL"
resolution=15 # arcsec
cell=7.5 # arcsec
x_extent=120 # arcsec
y_extent=120 # arcsec
instrument="SEQ"
#
# based on vlsr=-5, dv=100, dw=250
b_order=0
b_regions=[[-355,-105],[95,345]]
l_regions=[[-105,95]]
slice=[-355,345]
v_range=-105,95
x_extent=128
y_extent=128
pix_list=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
rmax=3
otf_a=1.1
otf_b=4.75
otf_c=2
sample=-1
otf_cal=0
edge=0
LMTOY>> Sleeping for 5 seconds, you can abort, edit lmtoy_92996.rc, then continuing
LMTOY>> process_otf_map2 in 2 seconds
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/ifproc/ifproc.py", line 582, in __init__
self.chop = self.nc.variables['Data.Msip1mm.BeamChopperActPos'][:]
KeyError: 'Data.Msip1mm.BeamChopperActPos'
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/ifproc/ifproc.py", line 713, in __init__
self.chop = self.nc.variables['Data.Msip1mm.BeamChopperActPos'][:]
KeyError: 'Data.Msip1mm.BeamChopperActPos'
/lma1/teuben/lmtoy/anaconda3/lib/python3.8/site-packages/numpy/core/fromnumeric.py:748: UserWarning: Warning: 'partition' will ignore the 'mask' of the MaskedArray.
a.partition(kth, axis=axis, kind=kind, order=order)
{'--b_order': '0',
'--b_regions': '[[-355,-105],[95,345]]',
'--bank': '0',
'--eliminate_list': '0',
'--help': False,
'--l_regions': '[[-105,95]]',
'--obsnum': '92996',
'--output': 'Ori-KL_92996.nc',
'--path': '/n/lma1/lmt/data_lmt/',
'--pix_list': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15',
'--restfreq': None,
'--sample': None,
'--save_tsys': False,
'--slice': '[-355,345]',
'--stype': '2',
'--tsys': '250.0',
'--use_cal': True,
'--use_otf_cal': False,
'--x_axis': 'VLSR'}
program process_otf_map options
x_axis = VLSR
b_order = 0
use_cal = True
output = Ori-KL_92996.nc
eliminate_list = [0]
use_otf_cal = False
b_regions = [[-355, -105], [95, 345]]
slice = [-355, 345]
bank = 0
obsnum = 92996
data_path = /n/lma1/lmt/data_lmt/
stype = 2
tsys = 250.0
path = /n/lma1/lmt/data_lmt/
l_regions = [[-105, 95]]
pix_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
output_file_name = Ori-KL_92996.nc
2021-12-17T02:01:17 begin /n/lma1/lmt/data_lmt/ifproc/ifproc_2021-12-17_092996_00_0001.nc
2021-12-17T02:03:43 end /n/lma1/lmt/data_lmt/ifproc/ifproc_2021-12-17_092996_00_0001.nc
BufPos 1 11.7 sec 92996
BufPos 0 128.4 sec 92996
BufPos 1 6.1 sec 92996
before read npix
from pixels npix = 16
from xlen npix = 16
TRACKING Sequoia PIXEL 10
Map Parameters: Az Continuous
HPBW= 14.0 XLength= 120.0 YLength= 120.0 XStep= 2.00 YStep= 0.45
/n/lma1/lmt/data_lmt/ifproc/ifproc_2021-12-17_092996_00_0001.nc does not have bs parameters
92996 is a Map observation
get chop
'Data.Msip1mm.BeamChopperActPos'
no chop
2021-12-17T02:00:59 begin /n/lma1/lmt/data_lmt/ifproc/ifproc_2021-12-17_092995_00_0001.nc
2021-12-17T02:01:05 end /n/lma1/lmt/data_lmt/ifproc/ifproc_2021-12-17_092995_00_0001.nc
BufPos 3 4.2 sec 92995
BufPos 2 2.6 sec 92995
before read npix
from pixels npix = 16
from xlen npix = 16
TRACKING Sequoia PIXEL 10
/n/lma1/lmt/data_lmt/ifproc/ifproc_2021-12-17_092995_00_0001.nc does not have map parameters
/n/lma1/lmt/data_lmt/ifproc/ifproc_2021-12-17_092995_00_0001.nc does not have bs parameters
92995 is a Cal observation
get chop cal
'Data.Msip1mm.BeamChopperActPos'
no chop cal
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/anaconda3/bin/process_otf_map2.py", line 7, in
exec(compile(f.read(), __file__, 'exec'))
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/process_otf_map2.py", line 110, in
main(sys.argv[1:])
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/process_otf_map2.py", line 87, in main
I, S = read_obsnum_otf(Opts.obsnum,
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/utils/reader.py", line 214, in read_obsnum_otf
specbank = SpecBankData(files, ifproc,
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/spec/spec.py", line 1009, in __init__
SpecBank.__init__(self, roach_files, ifproc_data,
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/spec/spec.py", line 717, in __init__
self.nchan = self.roach[0].nchan
IndexError: list index out of range
LMTOY>> view_spec_file
{'--help': False,
'--input': 'Ori-KL_92996.nc',
'--pix_list': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15',
'--plot_range': '-1,3',
'--plots': 'Ori-KL_92996_specviews',
'--rms_cut': '-4',
'--show_pixel': None,
'--skip_tsys': False}
SLR Plots initialized with 1 method=Ori-KL_92996_specviews
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/anaconda3/bin/view_spec_file.py", line 7, in
exec(compile(f.read(), __file__, 'exec'))
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/view_spec_file.py", line 91, in
main(sys.argv[1:])
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/view_spec_file.py", line 67, in main
SV = SpecFileViewer(input_file_name)
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/viewer/spec_file_viewer.py", line 20, in __init__
nc = netCDF4.Dataset(netcdf_filename, 'r', format='NETCDF4')
File "src/netCDF4/_netCDF4.pyx", line 2330, in netCDF4._netCDF4.Dataset.__init__
File "src/netCDF4/_netCDF4.pyx", line 1948, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: b'Ori-KL_92996.nc'
SLR Plots initialized with 3 method=Ori-KL_92996_specpoint,png,1
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/anaconda3/bin/view_spec_point.py", line 7, in
exec(compile(f.read(), __file__, 'exec'))
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/view_spec_point.py", line 74, in
main(sys.argv[1:])
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/view_spec_point.py", line 61, in main
SV = SpecFileViewer(nc_file)
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/viewer/spec_file_viewer.py", line 20, in __init__
nc = netCDF4.Dataset(netcdf_filename, 'r', format='NETCDF4')
File "src/netCDF4/_netCDF4.pyx", line 2330, in netCDF4._netCDF4.Dataset.__init__
File "src/netCDF4/_netCDF4.pyx", line 1948, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: b'Ori-KL_92996.nc'
SLR Plots initialized with 3 method=Ori-KL_92996_specpoint,png,2
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/anaconda3/bin/view_spec_point.py", line 7, in
exec(compile(f.read(), __file__, 'exec'))
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/view_spec_point.py", line 74, in
main(sys.argv[1:])
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/view_spec_point.py", line 61, in main
SV = SpecFileViewer(nc_file)
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/viewer/spec_file_viewer.py", line 20, in __init__
nc = netCDF4.Dataset(netcdf_filename, 'r', format='NETCDF4')
File "src/netCDF4/_netCDF4.pyx", line 2330, in netCDF4._netCDF4.Dataset.__init__
File "src/netCDF4/_netCDF4.pyx", line 1948, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: b'Ori-KL_92996.nc'
LMTOY>> make_spec_fits (waterfall)
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/anaconda3/bin/make_spec_fits.py", line 7, in
exec(compile(f.read(), __file__, 'exec'))
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/make_spec_fits.py", line 38, in
main(sys.argv[1:])
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/make_spec_fits.py", line 33, in main
SV = SpecFileViewer(nc_file)
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/viewer/spec_file_viewer.py", line 20, in __init__
nc = netCDF4.Dataset(netcdf_filename, 'r', format='NETCDF4')
File "src/netCDF4/_netCDF4.pyx", line 2330, in netCDF4._netCDF4.Dataset.__init__
File "src/netCDF4/_netCDF4.pyx", line 1948, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: b'Ori-KL_92996.nc'
Traceback (most recent call last):
File "/lma1/teuben/lmtoy/anaconda3/bin/make_spec_fits.py", line 7, in
exec(compile(f.read(), __file__, 'exec'))
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/make_spec_fits.py", line 38, in
main(sys.argv[1:])
File "/lma1/teuben/lmtoy/SpectralLineReduction/bin/make_spec_fits.py", line 33, in main
SV = SpecFileViewer(nc_file)
File "/lma1/teuben/lmtoy/SpectralLineReduction/lmtslr/viewer/spec_file_viewer.py", line 20, in __init__
nc = netCDF4.Dataset(netcdf_filename, 'r', format='NETCDF4')
File "src/netCDF4/_netCDF4.pyx", line 2330, in netCDF4._netCDF4.Dataset.__init__
File "src/netCDF4/_netCDF4.pyx", line 1948, in netCDF4._netCDF4._ensure_nc_success
FileNotFoundError: [Errno 2] No such file or directory: b'Ori-KL_92996.nc'
LMTOY>> grid_data
{'--cell': '7.5',
'--edge': '0',
'--help': False,
'--input': 'Ori-KL_92996.nc',
'--n_samples': '256',
'--noise_sigma': '1',
'--otf_a': '1.1',
'--otf_b': '4.75',
'--otf_c': '2',
'--otf_select': '1',
'--output': 'Ori-KL_92996.fits',
'--pix_list': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15',
'--program_path': 'spec_driver_fits',
'--resolution': '15',
'--rmax': '3',
'--rms_cut': '-4',
'--sample': '-1',
'--weight': 'Ori-KL_92996.wt.fits',
'--x_extent': '128',
'--y_extent': '128'}
STDOUT ****************************
spec_driver_fits LMTSLR 26-sep-2021
CMD: spec_driver_fits -i Ori-KL_92996.nc -o Ori-KL_92996.fits -w Ori-KL_92996.wt.fits -l 15 -c 7.5 -u [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] -z -4 -s 1 -x 128 -y 128 -f 1 -r 3 -n 256 -0 1.1 -1 4.75 -2 2 -b -1 -e 0
file list=Ori-KL_92996.nc
DEBUG:7.5
sample_list=-1
Warning: sample list is not a multiple of 3
nsamples=1
PJT: OTF->fuzzy_edge = 0
Processing 1 SpecFiles:
Opening SpecFile file Ori-KL_92996.nc
Error: No such file or directory
STDERR ****************************
Exit Code: 2
LMTOY>> Some NEMO post-processing
LMTOY>> Problems finding Ori-KL_92996.fits. Skipping NEMO work.
LMTOY>> ADMIT post-processing
LMTOY>> lmtoy_admit: 31-oct-2021
/lma1/teuben/lmtoy/examples/lmtoy_admit.sh: line 33: export: `Ori-KL_92996.fits': not a valid identifier
Warning: file Ori-KL_92996.fits does not exist, no ADMIT processing
LMTOY>> Created Ori-KL_92996.fits and Ori-KL_92996.wt.fits
LMTOY>> Parameter file used: lmtoy_92996.rc
LMTOY>> Making summary index.html:
Making index.html for obsnum=92996
Writing index.html
Writing index_pipeline.html
Writing index_admit.html
Writing index_pars.html
Writing index_log.html