ZTF Data Quality Issues

Info about ZTF DQ.

Topics
ZTF data quality
Images with bad PCOUNT, Sept. 2022.
Bad image examples.
Discussion.

ZTF Data Quality Issues

Images rejected due to bad PCOUNT, September 2022.

In September 2022 a few images from CCDs 10 and 14 were found to have been rejected by the IPAC pipeline due to bad values of PCOUNT. This effect so far only effects a small fraction of the images taken with these CCDs.

From the "Definition of FITS version 3.0 (Pence et al A\ampA 2010, 524, 42)" the PCOUNT is:


PCOUNT defined by Pence et al. 2010.

Initial the problem appeared only within bias and flatfield images. The problem was not reproduced in daytime tests. Towards the end of September the problem was also observed in science images.

Examples of rejected images: Flats


A ZTF flatfields taken on 2022-09_20 reject by the IPAC pipeline due to a bad PCOUNT value (20220920062025, CCD14, q1, zg).

In the plot above we can seen that the flatfield appears to have only been partial read out.



A zoom in of the black section of pixels in the ZTF flatfield given above and the histogram of all pixel values for the frame.

In the plot above we see that the black section of pixels actually contains values near zero. However the values are not zero. This image is not modified by the pipeline suggesting that the image is not actually truncated.

Examples of rejected images: Science frames



A full ZTF science image from 2022-09-20 (20220920428762, F352, zr, ccd14, q1) and a zoom-in.

The science image above was also rejected by the IPAC pipeline due to PCOUNT. This image also appears to show a truncated readout where the pixels in the black section are not actually zeros.


A full ZTF science frame from 2022-09-23 (20220923267558, F341, zr, ccd14, q1).

In contrast to the flatfields and science frames that appear truncated we also see science images where the image does not contains a blank image with bright features. These appear to be noise images where all the flux has been read within a few rows.



A zoom of the bright feature seen at the bottom of the bright feature in the ZTF science image above (20220923267558, F341, zr, ccd14, q1) and a histogram of all the pixel values within the frame.

A zoom on the bright band at the bottom of the image shows significant flux. Unlike normal science images there are no high pixel values due to stars or saturation. Almost all pixel values are near zero. The bright feature along the bottom of the image resembles earlier ZTF images where very bright stars caused long overflow features.

The bad images are only seen in quadrants one and four of CCDs 10 and 14 which are eletronically linked.


A full quadrant ZTF science image (20220922120023, F478, zr, c14, q1) from 2022-09-22.

In the science image above we see multiple features. The left of the image appears to show a regular image with stars present with bottom showing a significant bright band.



Zoom-in images of the ZTF science image above (20220922120023, F478, zr, c14, q1).

In the plots above we scale the left corner of the image to show the noise pattern in the black region and the structure at the transition between a regular image and the bright structure.

Discussion

We have investigated the images from CCDs 10 and 14 that were rejected by the IPAC pipeline. Some flagged images appear to be a mixture of images where the values were set to almost zero (read noise?) part way through reading an apparently normal images. Other frames appear to be purely noise with bright lines or a mixture of normal images with other electronic artifacts. In all the cases checked the frames do not actually appear to be truncated.

Overall the problems with the flagged images appear to be electronic in nature. Given that the effect is transitory, it seems possible that the cause is a poor connection that is flexing with the telescope motion. However, it could simply be some failing electronic component.