OR Why visual review of Flats is Important!
As part of the calibration of your light frames I hope everyone appreciates that Flats are becoming one if not THE most important aspect of the calibration frames that we take as astrophotographers. I have recently been working on a multi night project. During the three nights I have diligently followed a process of capturing Flats and DarkFlats. This is a change to my previous workflow which used Bias frames with Flats. I will not go into this here but suffice to say this change has been introduced along with my recent purchase of the ZWO ASI533MC Pro camera.
My flats are produced using Nina's Flat Wizard with an iPad displaying a pure white screen at full brightness while I cover the telescope lens with a T-Shirt stretched using a cross stitch frame. With my new camera I have taken on board the collective wisdom and set an exposure to get a 50% histogram with a 5% tolerance. This has tended to give me an exposure time of around 6s when taken through the Optolong L-Enhance filter at a gain of 100 (offset 40). I take 30 exposures each of Flats and Dark Flats using the Wizard.
I have noticed once or twice while following my own Pre-Processing steps that when I get to stacking the pre-processed flats (Master Dark Flat acting as the 'Bias') that the Winsorised Rejection Algorithm is throwing at a very high number of pixels. When I say high the exception is to be able to have no more than 0.1%-0.5% outside of a High or Low threshold. I have been getting values in the 15%-30% range and no matter what Sigma values are input (10 being the maximum) the rejection rate is still way too high. Question is, what is going on, this behaviour is not consistent, some sessions process fine?
Spolier Alert
I should have stepped through the raw Dark Flats and Flats using Siril's Image List feature (something I do not usually do unless blinking through my lights) but we will come back to that.
I had a number of initial suspicions:-
Was it possible new camera was more susceptible to picking up on scan lines of the iPad display?
Was the exposure length for the flats and dark flats too short or too long causing an excessive amount of noise?
In order to methodically put to bed the above thoughts I decided to take 4 sequences of Flats with corresponding DarkFlats (obviously taken with the same camera settings), not that it is necessary but to remove any thermal issue I also cooled the camera to -10ºC (my normal image shooting temp). Each sequence involved taking 20 Flats and Darkflats.
Sequence 1 - No diffuse material between the iPad screen and telescope aiming for a 50% histogram, the intention here was to remove any possible material texture but also to reduce the exposure time.
Exposure time of 1.96s
Sequence 2 - T-Shirt used to diffuse the iPad screen. Aimed for a 30% histogram (this is what I used to aim for with the DSLR).
Exposure time of 3.83s
Sequence 3 - T-Shirt used to diffuse the iPad screen. Aimed for a 50% histogram (this is what I have been aiming for based on research across the web).
Exposure time of 6.97s
Sequence 4 - T-Shirt + additional diffuse material in the form of some Kitchen Greaseproof paper. Aimed for 50% histogram, this was to try and extend the exposure time
Exposure time of 9.71s
Once I had taken all the Flats and Darks I ran through the Siril Manual processing steps for each sequence:-
Convert the Fits files into Siril - in effect this registers a new sequence
Stacked each DarkFlats using the following parameters to produce the Master Dark Flat for this sequence
The Console should be checked at this point to ensure that rejection %'s are within a range of 0.1% - 0.5% with some minor variance allowed. Sigma Low and High can be used to manipulate the rejection %'s (increasing Sigma will have the effect of decreasing the rejection rate)
Selecting the Flats sequence performed pre-processing using generated Master Darkflat as per following Siril parameters:-
Ensuring the pp_flats for this sequence are selected generate the Master Flat file, using Siril parameters as follows:-
Taking a look at the Console it is important that pixel rejections are within a range of 0.1% to 0.5% (some variance around this is acceptable). The problem statement is that in some cases I am seeing these percentages of > 10% which is not acceptable. The Sigma Low and high values can be used to control this to some extent but it should not be necessary to set these much beyond 4 or 5.
During the sequence runs the large rejection % was experienced as part of sequence 2 at the point of generating the Master Flat.
Statistics
The Statistics for each file that makes up each sequence where output, this was made easier using the Siril command:-
seqstat <Sequence name> <CSV Filename> "main"
This outputs the statistics for each file in the sequence. I created a number of graphs for each sequence in order to highlight any anomalies in any of the files. One particular graph for sequence 2 stood out, this is the sequence that exhibited the high rejection rate. The grey line in the chart opposite is the Median of each of the 20 Flats that make up this sequence. As can be seen there is a peak in Frame 6. All other sequences show very consistent values across each image.
Removing Frame 6 and reprocessing fixed the high percentage of rejections.
Lessons Learnt
Looking individually at frame 6 flat in sequence in isolation shows no evidence that there is anything wrong. However, blinking through the set of flats in the sequence showed that this Flat was observably brighter that the rest. The reason is unknown, the Flats Wizard in Nina automates the process and I do no remember anything memorable that might have caused that one image to be brighter than the others. As mentioned at the start of this section I should have blinked through the data before processing and removed any images that looked significantly different.
One other note of mention, while I was looking at the Histograms I started off using the application that came with my ZWO ASI camera (it is actually feely available from the ZWO website ASI Studio. The Fits Viewer it comes with worked fine to render and show the histogram of raw fit files, however after being processed by Siril in the native 32 bit mode the files although shown did not display correct statistical or histogram information. Therefore, it must be remembered not to trust that tool if looking at files that have been processed in Siril.
Comentarios