Skip to main content

The Craft of Scientific Research - Chapter 5: How to produce reliable scientific information

Originally, this topic was not part of the doctoral course for which this book serves as the syllabus. But I felt the need to add it because I believe there is, in various scientific domains, a true crisis about how to assess the credibility of information.

I start by referring to the classic theory of information, where quantitative information is treated as an estimand whose estimates are produced by an estimator, which can be of three types: measured, inferred, or predicted.  I propose that the distinction between inferred and predicted is whether the estimator uses prior knowledge; in that case, we call it a predictive estimator. And prior knowledge is, in this narrow definition, considered only as explicit knowledge, such as the second law of dynamics (rigorously defined as an undefeated justified true belief).

Then to assess the credibility of the estimator (and thus of the information it estimates), we have three separate frameworks: for measured estimators, it is provided by metrology; for inferred estimators is provided by statistics; and for predicted estimators (in such narrow sense, so first-principle models) is provided by computational science & engineering (CS&E) with the so called VVUQ framework.

At this point, I highlight that the AI revolution is blurring the boundary between these three types: predictors built on implicit synthetic data, synthetic data, machine learning estimators trained on synthetic data, and physics-informed machine learning. Thus, I propose the S7 Framework, a generalisation of the three specialised frameworks from metrology, statistics and CS&E. These are special cases of the S7 framework for assessing the credibility of information, regardless of the type of estimator used to generate it. 

I must stress a caveat: the original paper proposing the S7 is currently available only as a preprint and has not been peer-reviewed (https://doi.org/10.5281/zenodo.18130930). I already sent it to two journals, and both rejected it because the topic is not of interest to them. I appreciate that the argument is quite theoretical, but I believe it has tremendous importance right now.  If you know a journal that might consider such a paper, please let me know. 


Enjoy the reading!


Summary of Chapter 5 of “The Craft of Scientific Research”, by Marco Viceconti, self-published and Green Open Access book on the Zenodo repository: https://doi.org/10.5281/zenodo.18069190. 


Comments

Post a Comment

Popular posts from this blog

The first complete version of the book is out

It's been a busy Christmas break, but it is finally out.  On December 27th, I published the first incomplete draft; today, you can find the complete first version of my book “The Craft of Scientific Research”, self-published as green open access through the Zenodo repository.  This DOI will give you access to the latest version: https://doi.org/10.5281/zenodo.18069190 . In the past five days, 182 people have already downloaded the first draft. I invite you all to download this new version, which, in addition to being properly formatted, has been extensively revised and enhanced with the inclusion of a full bibliography and a brief 'Afterthoughts' section at the end. This book is not published by a Traditional Publishing House, so its primary form of promotion is word of mouth.  Please repost on LinkedIn and pass the link to all your PhD students, asking them to share it as well. As a cover, after some discussions with my wife, Paola, I opted for a picture of an old shoema...
Post a Comment
Read more

A new blog for my book "The Craft of Scientific Research"

In 2024, I decided to gradually discontinue my research activities.  For more information, you can read my blog on How to Gently Retire.  For the few years left before real retirement, I decided to focus on teaching, and in particular, where possible, to share with the next generation of researchers the lessons I learned in nearly 40 years of honoured career.  For many years, I offered seminars to PhD students on specific transferable skills (how to write a paper, how to give a scientific presentation).  From that came the idea to expand that content and develop a 30-hour course on the job of academic research.  I discussed the idea with my university, and in January 2024, I taught for the first time a doctoral course entitled "The Craft of Scientific Research".  It went very well, being the first time.  More than 100 PhD students from the University of Bologna attended, representing 20 different doctoral programs in the fields of engineering, physical...
Post a Comment
Read more

The Craft of Scientific Research - Chapter 1: Why do we need to know?

 The whole book is about how to produce new knowledge. Therefore, it makes sense that we first ask ourselves: Why do we (as individuals and as a species) need to know? The purposes of scientific research are to increase the knowledge of humanity and to solve humanity’s problems. Why do we need to know how the universe works to solve humanity’s problems? And more in general, why do we need to know, even when this does not immediately solve a problem? As usual in this book, in front of such a big question, we resort to philosophers.  We discuss Mill’s “It is better to be a human being dissatisfied than a pig satisfied”; Nietzsche’s “life could be an experiment for the seeker for knowledge”, and we even explore how Zen Buddhism see it.  We return to the classic theory of knowledge, which originates from Plato’s Meno Socratic dialogue, and introduce the concepts of Justified True Belief and of Undefeated Justified True Belief. As is often the case with philosophers, while we ...
Post a Comment
Read more