CompareWarp Evaluation

Current status and latest problems with CompareWarp as of DM-12445.

Instructions to use CompareWarpAssembleTask:

To run assembleCoadd.py use the --warpCompareCoadd argument like:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!UPDATED 12/11/2017!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

export wideVisitsR=11442^11446^11450^11470^11476^11478^11506^11508^11532^11534

echo 'config.assembleStaticSkyModel.subregionSize = (10000, 200)

config.assembleStaticSkyModel.doApplyUberCal = True' > config_assemble.py

assembleCoadd.py --warpCompareCoadd /datasets/hsc/repo --rerun private/yusra/psfMatching/DM-12445:private/yusra/psfMatching/DM-12445-clipped --id tract=8766 filter=HSC-R  --selectId ccd=0..8^10..103 visit=$wideVisitsR --configfile config_assemble.py

To run coaddDriver.py create a config file and apply with the --configfile flag.

export wideVisitsY=6478^6482^6486^6496^6498^6522^6524^6528^6532^6544^6546^6568^13152^13154

echo 'from lsst.pipe.tasks.assembleCoadd import CompareWarpAssembleCoaddTask

config.assembleCoadd.retarget(CompareWarpAssembleCoaddTask)

config.assembleCoadd.assembleStaticSkyModel.subregionSize = (10000, 200)

config.assembleCoadd.assembleStaticSkyModel.doApplyUberCal = True' > config_coaddDriver.py

coaddDriver.py  /datasets/hsc/repo --rerun RC/w_2017_44/DM-12388:private/yusra/psfMatching/DM-12445  --batch-type=slurm --mpiexec='-bind-to socket' --job Y_psfMatch-DM-12445 --time 600 --cores 23  --id tract=8766 filter=HSC-Y  --selectId ccd=0..8^10..103 visit=$wideVisitsY --configfile config_coaddDriver.py

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^UPDATED 12/11/2017^^^^^^^^^^^^^^^^^^^^^^^^^

(1) Photometry via pipe analysis:

Run on two tracts: WIDE: 8766 and DEEP: 9813

for the following 2 repos:

/datasets/hsc/repo/rerun/private/yusra/psfMatching/DM-12445

/datasets/hsc/repo/rerun/RC/w_2017_44/DM-12388

coaddAnalysis.py:

DM-12445 (compareWarp as of DM-12445):

https://lsst-web.ncsa.illinois.edu/~yusra/artifact_rejection/DM-12445_analysis/

w_2017_44 (safeClip as of weekly 44):

https://lsst-web.ncsa.illinois.edu/~yusra/artifact_rejection/w_2017_44_analysis/

compareCoaddAnalysis.py

Comparison of DM-12445 and w_2017_44:

https://lsst-web.ncsa.illinois.edu/~yusra/artifact_rejection/compare-DM-12445/

With Lauren looking over my shoulder, we flipped through some of her favorite plot type (the CModel difference between runs: e.g. compare-t9813-HSC-R-diff_modelfit_CModel-psfMagHist.png). We saw:

The number of “splits” Star/Galaxy designation from one run to another is scary. Filed ticket: DM-12723

​​(1) Photometry via pipe analysis:

(1) Photometry via pipe analysis: