This ticket serves to explain ticket in DM-13637 to help users decide if they want to enable it by setting doPrefilterDefects=True once it get merged.
CompareWarp decides whether an artifact candidate is ephemeral(and should be clipped) or persistent and should not depending on how many times it appears in the PSF-matched image differences(Warp - SigmaClipped Template Coadd). The number of times a pixel is an outlier in one of the warps is called the epochCountImage. One failure mode is when a survey is minimally dithered and camera defects line up building up the epochCountImage with these defects.
HSC-Z 9813 3,3 epoch count image as of DM-12692
Where is this a problem:
On the DUD fields there are patches that overlap. Plotting the clipped regions will show patches with minimal clipping. You might find patches where regions are completely unclipped. If this is the case, you probably want to rerun with this.
As of DM-14075 using public data on lsst-dev
Solutions:
The best solution would be to add functionality to SourceDetection to ignore specified mask planes.
Another solution would be to detect without growing, convert the bad mask plane to spanSets, cut them out of the footprints, then grow the footprints.
Third option is to pre-filter out the artifact candidates that have < some percentage good pixels.
Overall result is that with the ticket more artifacts are clipped in regions with overlapping defects.
CLIPPED planes
CLIPPED regions 9813 DEEP:
BEFORE
AFTER
BEFORE
AFTER
BEFORE
AFTER
CLIPPED regions 9697 WIDE:
Note the while the defects are no longer marked clipped this does not strongly affect the wide fields, because the defect overlaps are less frequent.
BEFORE
AFTER
BEFORE
AFTER
Case study:
Case study of HSC-Z, tract=9813, patch=6,7, blinking the following two file shows improvement most everywhere,
Results
CLIPPED planes
DM-14075_v2/deepCoadd/HSC-Z/9813/6,7.fits