ZTF Data Quality Issues

Info about ZTF DQ.

Topics
ZTF data quality
Nov 2022, Difference image rejects.
Nov 2022, Bad flats.
Nov 2022, Streaked images in Archon 4 streaks.

ZTF Data Quality Issues

Rejected Difference Images, Nov 2022

In November 2022 the ZTF pipeline reduction alerted that images in quadrants 8,9,12,13,24,25,28,29 were being rejected with exist code 44. This is a somewhat general exit code without a clear origin. The error was traced to a difference image quality flag. However, inspection of the images revealed relatively normal looking difference images that should not have been rejected.


Difference image and pixel histogram of 2022-11-10 images rejected by ZTF pipeline.

However, in comparison with a normal difference image not rejected by the pipeline we notice that the saturated stars are not being masked properly. So, the pixel distribution shows a long tail that cause the pipeline to reject it.


Difference image and pixel histogram of 2022-11-10 images not rejected by ZTF pipeline.

When we compare the scale of the un-subtracted science images we see the that saturation level in the poorly masked and rejected quadrants is much lower (40K vs 60K). So, it is the new lower saturation level that is causing poorly masked images to be rejected.


Comparion of the science image pixel histogram associated with a rejected difference image (left), vs an OK science image histogram (right).

Bad flatfields

Close inspection of the pipeline-rejected difference images suggested that there were features from the flatfields (dust rings) that were not being correctly removed. As expected the flatfields associated with bad quadrants were also found to have significant issues. In particular all the g-band flats taken since 2022-11-02 were found to be bad in those quadrant (except for those from 2022-11-07). However, in some cases the problems with the flats are only seen in g-band. On other nights both g and r-band are affected.

The problem is very likely due to the change in the saturation level (which itself is probably due to recent changes in the electronics).

<
A bad g-band flatfield image taken on 2022-11-06 along with the associated pixel histogram.

The issue with the bad flatfields does not cause them to be rejected as combined and normalized flats are still produced containing the errors.


Combined normalized bad g-band flatfield image from 2022-11-08 and the associated pixel histogram.

Image Streaks from Archon 4, Nov 2022

A Controller sync issues is causing the images in archon for to be streaked intermitently. These streaks were noted as problems with the parallel clocking.