Re: Inquiry Regarding the Stability of Calibration Results i…

Top Page
Delete this message
Reply to this message
Author: MASSAHUD CARVALHO RIBEIRO, Emily
Date:  
To: pyfai, Time
Old-Topics: Inquiry Regarding the Stability of Calibration Results in pyFAI
Subject: Re: Inquiry Regarding the Stability of Calibration Results in pyFAI [SEC=OFFICIAL]
Hi Linjun,

I will share my 2 cents with you regarding the calibration process from my personal experience.
We have a detector in our facility which has distortions (physical distortions, as in, it is bent a little) near one of the corners.
That meant that using the outer most rings of a calibration image usually gave us bad results, so we started performing calibration using only the inner most rings (like 5 for example, but it really depends on your calibrant/wavelength/distance etc).
Outer rings are usually bad to use for refinement because if you dont have a full ring in the image, you provide worse statistics for the refinement process (full rings are definitely more trustworthy). so if you combine a detector distortion (not sure if that is your case) with not enough control points, you might get significant differences in the final geometry.
(oh, but also be careful with the other extreme: not providing enough rings will also give you bad statistics, and not be a good calibration - one ring is probably not good enough)

I believe pyfai has some good tutorial notebooks to measure detector distortions to confirm the hypothesis btw.

Not sure if you're using the pyFAI calib tool, or just doing it from a jupyter notebook, but in case it's the second, I have developed a ring_extraction module for pyfai a while back to perform automated geometry calibration routines: https://github.com/silx-kit/pyFAI/blob/main/src/pyFAI/ring_extraction.py

My tip is: always use the 2D integration image qualitatively to check whether the calibration was good (you wanna see the lines all vertical and matching the expected ring positions of the calibrant). also use the refinement statistics to check your goodness of fit

The control points used in the calibration are usually extracted using an algorithm, the massif, and it chooses random seeds to begin the control point extraction, which can lead to different results, yes (but if your peak picking is good enough, these differences should not be that big. that hints at other problems..)
Also, be conscious of the refinement routine, as detector distance and wavelength/energy are not independent variables, and their correlation makes things tricky during the refinement (I only calibrate for wavelength at a last pass, while fixing distance first, if needed, to avoid these issues).

In the end, a calibration is an iterative process, so no 2 calibrations will yield exactly the same results, it really all depends on the steps you took during the process. That is why it is important to integrate all your data against the same calibration, to make your results and analysis more reliable, and to visually inspect the calibration results against your calibrant data to ensure it is satisfactory.

I don't know whether I helped you made you more confused, lol, but I hope you can take something out of this.

Cheers,



Dr Emily Massahud

Scientific Computing Engineer

Australian Synchrotron

Email
Web

massahue@???<mailto:massahue@ansto.gov.au>
www.ansto.gov.au<http://www.ansto.gov.au/>

[fb]<http://www.facebook.com/DiscoverANSTO> [tw] <https://twitter.com/ANSTO> [ln] <http://www.linkedin.com/company/ansto> [yt] <http://www.youtube.com/user/ANSTOVideos>

[ansto]





________________________________
From: pyfai-request@??? <pyfai-request@???> on behalf of "Time" <pyfai@???>
Sent: Tuesday, March 17, 2026 3:14 PM
To: pyfai <pyfai@???>
Subject: Inquiry Regarding the Stability of Calibration Results in pyFAI

CAUTION, EXTERNAL EMAIL: This message has come from outside of ANSTO. Do not take action, click links or open attachments unless you trust the source of this message and know the content is safe. Report spam and phishing using the Report Message button or if unsure, forward this message to servicedesk@??? as an attachment.


Dear pyFAI Developers,

I hope this email finds you well.

My name is Linjun He, and I am a graduate student working on SAXS-related scattering data calibration and detector geometry analysis. During our use of pyFAI for calibrating a large number of experimental datasets, we observed a certain degree of instability in the fitted geometric parameters. I am therefore writing to seek your advice and would sincerely appreciate any guidance you may be able to provide.

Our experimental conditions are as follows, and here I use a single SAXS image as an example for illustration:

Data type: SAXS image

X-ray energy: 10 keV

Standard sample: AgBH

Detector: Pilatus 2M

Exposure time: 5 s

During the calibration process in pyFAI, we consistently used the same raw image (Attachment1) and applied the same mask file(Attachment2) throughout the entire workflow. In the peak-picking interface, we manually selected the peaks and then performed calibration under different ring number settings (1, 2, 3, 4, and 5).

We observed that, although the raw image and the mask file were exactly the same, the final fitted geometric parameters still showed noticeable differences when different numbers of rings were used for calibration (the settings interface is shown in Attachment 3, and the corresponding results are summarized in Attachment 4). The parameters that changed include:

poni1

poni2

rotation1

rotation2

rotation3

distance

In addition, we noticed that even for the same diffraction ring, the number of valid points identified in the peak-picking interface could sometimes differ (see the comparison in Attachment 5; the corresponding data are highlighted in red in Attachment 4).

Therefore, we would like to ask the following questions:

1.Under the condition of using the same image, the same mask, and the same manually selected peaks, why can the number of valid points on the same ring still change?

2.When the ring number is set to 1, 2, 3, 4, and 5, the fitted values of poni1, poni2, rotation1, rotation2, rotation3, and distance show certain differences. Is this behavior considered normal in pyFAI?

3.Are these variations mainly caused by differences in the number of points selected during the peak-picking stage, or are they primarily due to the different constraints introduced into the geometric fitting when different ring number settings are used?

4.For SAXS image calibration, would you recommend a more stable peak-picking or ring-selection strategy to reduce this type of parameter fluctuation?

We have organized the fitting results obtained from the same image under different ring number settings into a table, and it can be clearly seen that these parameters do vary to a certain extent. I would like to understand whether this behavior is expected in the pyFAI fitting process, or whether there are additional settings or precautions that we should pay particular attention to.

Thank you very much for taking the time to read this email. I would greatly appreciate your guidance and reply.

Sincerely,
Linjun He

________________________________
[http://res.mail.qq.com/zh_CN/htmledition/images/icon_att.gif]从QQ邮箱发来的超大附件
[http://res.mail.qq.com/zh_CN/htmledition/images/fj/fu_qita.gif]<https://mail.qq.com/cgi-bin/ftnExs_download?k=6f3361632ef15b9eeac9c4254439014e414700025701500414570507031450590a514c57500f074c0f02005a535a05560d035301621833204d4700000a54560f4d13504e237e714c0c403e5352090350191b524a4c4d5a07390e&t=exs_ftn_download&code=93acb93a>
Attachment 1-AGB-5s_00001.tif<https://mail.qq.com/cgi-bin/ftnExs_download?k=6f3361632ef15b9eeac9c4254439014e414700025701500414570507031450590a514c57500f074c0f02005a535a05560d035301621833204d4700000a54560f4d13504e237e714c0c403e5352090350191b524a4c4d5a07390e&t=exs_ftn_download&code=93acb93a> (9.45M, 无限期)
进入下载页面<https://mail.qq.com/cgi-bin/ftnExs_download?k=6f3361632ef15b9eeac9c4254439014e414700025701500414570507031450590a514c57500f074c0f02005a535a05560d035301621833204d4700000a54560f4d13504e237e714c0c403e5352090350191b524a4c4d5a07390e&t=exs_ftn_download&code=93acb93a>:https://mail.qq.com/cgi-bin/ftnExs_download?k=6f3361632ef15b9eeac9c4254439014e414700025701500414570507031450590a514c57500f074c0f02005a535a05560d035301621833204d4700000a54560f4d13504e237e714c0c403e5352090350191b524a4c4d5a07390e&t=exs_ftn_download&code=93acb93a
[http://res.mail.qq.com/zh_CN/htmledition/images/fj/fu_exl.gif]<https://mail.qq.com/cgi-bin/ftnExs_download?k=7a66646612fecb9ee39cc1201131064e48120600010302511d000757511c5659510049545405054c05545d5356520651055e5c5437243420441205055f5c510f444650461f051d4f480a171e370c&t=exs_ftn_download&code=0fdf714a>
Attachment 4.xlsx<https://mail.qq.com/cgi-bin/ftnExs_download?k=7a66646612fecb9ee39cc1201131064e48120600010302511d000757511c5659510049545405054c05545d5356520651055e5c5437243420441205055f5c510f444650461f051d4f480a171e370c&t=exs_ftn_download&code=0fdf714a> (12.2K, 无限期)
进入下载页面<https://mail.qq.com/cgi-bin/ftnExs_download?k=7a66646612fecb9ee39cc1201131064e48120600010302511d000757511c5659510049545405054c05545d5356520651055e5c5437243420441205055f5c510f444650461f051d4f480a171e370c&t=exs_ftn_download&code=0fdf714a>:https://mail.qq.com/cgi-bin/ftnExs_download?k=7a66646612fecb9ee39cc1201131064e48120600010302511d000757511c5659510049545405054c05545d5356520651055e5c5437243420441205055f5c510f444650461f051d4f480a171e370c&t=exs_ftn_download&code=0fdf714a
[http://res.mail.qq.com/zh_CN/htmledition/images/fj/fu_qita.gif]<https://mail.qq.com/cgi-bin/ftnExs_download?k=2a3965662501b99bb7c3c0201035514b1c4d50560104025d4909005e50185a5d570a48520e005049015853560f045701535a5704362c6325104d04055e58060a1019574b5b54100f4411514f185007026404&t=exs_ftn_download&code=d9ef65cd>
Attachment 2-mask.edf<https://mail.qq.com/cgi-bin/ftnExs_download?k=2a3965662501b99bb7c3c0201035514b1c4d50560104025d4909005e50185a5d570a48520e005049015853560f045701535a5704362c6325104d04055e58060a1019574b5b54100f4411514f185007026404&t=exs_ftn_download&code=d9ef65cd> (2.36M, 无限期)
进入下载页面<https://mail.qq.com/cgi-bin/ftnExs_download?k=2a3965662501b99bb7c3c0201035514b1c4d50560104025d4909005e50185a5d570a48520e005049015853560f045701535a5704362c6325104d04055e58060a1019574b5b54100f4411514f185007026404&t=exs_ftn_download&code=d9ef65cd>:https://mail.qq.com/cgi-bin/ftnExs_download?k=2a3965662501b99bb7c3c0201035514b1c4d50560104025d4909005e50185a5d570a48520e005049015853560f045701535a5704362c6325104d04055e58060a1019574b5b54100f4411514f185007026404&t=exs_ftn_download&code=d9ef65cd
[http://res.mail.qq.com/zh_CN/htmledition/images/fj/fu_doc.gif]<https://mail.qq.com/cgi-bin/ftnExs_download?k=733763625ed6d0c6eacdc62417610216414306565057080014560652544c050a0d514e5009555214090054070502050a58055451317430784d430201590c55574d175042195519175d58001a315c&t=exs_ftn_download&code=97cb1a09>
Attachment 3.docx<https://mail.qq.com/cgi-bin/ftnExs_download?k=733763625ed6d0c6eacdc62417610216414306565057080014560652544c050a0d514e5009555214090054070502050a58055451317430784d430201590c55574d175042195519175d58001a315c&t=exs_ftn_download&code=97cb1a09> (1.96M, 无限期)
进入下载页面<https://mail.qq.com/cgi-bin/ftnExs_download?k=733763625ed6d0c6eacdc62417610216414306565057080014560652544c050a0d514e5009555214090054070502050a58055451317430784d430201590c55574d175042195519175d58001a315c&t=exs_ftn_download&code=97cb1a09>:https://mail.qq.com/cgi-bin/ftnExs_download?k=733763625ed6d0c6eacdc62417610216414306565057080014560652544c050a0d514e5009555214090054070502050a58055451317430784d430201590c55574d175042195519175d58001a315c&t=exs_ftn_download&code=97cb1a09
[http://res.mail.qq.com/zh_CN/htmledition/images/fj/fu_doc.gif]<https://mail.qq.com/cgi-bin/ftnExs_download?k=7362363971878dcfea98937f16660a1f4116570d07040c0014010e5a074b0c020b541b5c09055a1d5f00520c05050b095c01570e307338714d16575a580b5d5e4d4203191852111e5d0d5541305b&t=exs_ftn_download&code=9b690f80>
Attachment 5.docx<https://mail.qq.com/cgi-bin/ftnExs_download?k=7362363971878dcfea98937f16660a1f4116570d07040c0014010e5a074b0c020b541b5c09055a1d5f00520c05050b095c01570e307338714d16575a580b5d5e4d4203191852111e5d0d5541305b&t=exs_ftn_download&code=9b690f80> (4.63M, 无限期)
进入下载页面<https://mail.qq.com/cgi-bin/ftnExs_download?k=7362363971878dcfea98937f16660a1f4116570d07040c0014010e5a074b0c020b541b5c09055a1d5f00520c05050b095c01570e307338714d16575a580b5d5e4d4203191852111e5d0d5541305b&t=exs_ftn_download&code=9b690f80>:https://mail.qq.com/cgi-bin/ftnExs_download?k=7362363971878dcfea98937f16660a1f4116570d07040c0014010e5a074b0c020b541b5c09055a1d5f00520c05050b095c01570e307338714d16575a580b5d5e4d4203191852111e5d0d5541305b&t=exs_ftn_download&code=9b690f80