Cargo cult science

Feynman coined the term "cargo cult science" during a commencement address. This term describing research that is aimed at confirming an assumed hypothesis became more widely known after the address was incorporated into the final chapter of his book Surely You're Joking, Mr. Feynman! Methods which superficially seem scientific will ultimately fail to deliver if researchers lack "utter honesty" - not just avoiding falsehoods, but bending over backwards to state all the possible flaws in your research. The latter is scientific integrity, the former is advertising.

Feynman argued adamantly against fooling the layman when talking about your research. He gives an example of an astronomer friend who asked what applications of his work he should mention in a radio interview. Feynman retorted "there aren't any" and the friend was dismayed because saying that would not attract continued funding support for his research.

This message remains relevant today, especially with increasing competition for grant funding and faculty positions, high impact journals with strict length limits, and big conferences with short talks. Even when we agree with being honest and discussing flaws in our research in principle, excuses inevitably come up:

"I don't have time to discuss limitations - I only have 10 minutes including questions."

"My peers who publish in Top Journal all start their papers this way - it's the only way to make it past the editor." 

"Unless I frame my proposal in terms of this Grand Challenge it will not be funded."

"I have to play this game until I get tenure, and then I will be free to do honest old-fashioned research."

"I just need this grant so I can extend my postdoc's contract..."

The end result: Paper introductions and grant applications written by large language models, because they can sell the science in a more exciting way (weasel words can be inserted to smooth over overt factual errors). Seminars where the speaker boldly claims application X in the introduction, only to backtrack when questioned after the talk (lucky there was an expert present to point out a key flaw known by specialists in the topic). Researchers wasting months on ideas that were already tried and didn't work (no rewards for publishing negative results).

It doesn't need to be this way.

 

If you think there is not enough scientific integrity nowadays, you can help by participating in peer review and questioning unsubstantiated claims and excessive hype in the right way.

You should be curious and respectful, not belligerent and dismissive. Recommending rejection on the basis of how the broader context of the results are sold (rather than the results themselves) rarely leads to a constructive outcome - either the authors will ask for your opinion to be dismissed, or they will publish the offending claims unaltered in another venue. Instead you could ask the authors to explain in more detail how approach X is expected to help goal Y and possible flaws to better put the work in context. 

The same approach is also useful for Q&A sessions after talks. Often, the speaker is well aware of certain gaps in the logic of the presentation but didn't have the time to elaborate on them.  Questions in this vein help them to better convey the important unanswered questions in their research topic and are valuable to both the speaker and the audience.

The system has too much inertia to change immediately, but by putting the broader context and salesmanship behind the research under closer scrutiny you can help to diminish the influence of cargo cult science.

Beyond "Oppenheimer"

Some reading for those who found the events of the film "Oppenheimer" interesting:

The Manhattan Project (and Before)

This is a concise timeline of the Manhattan project, from the initial atomic bomb patent (!) to the test of the Gadget. Beyond the science and the scientists at Los Alamos, the Manhattan Project was an incredible feat of engineering involving 129,000 workers at its peak, requiring the development of the first industrial-scale processes for the enrichment of uranium, generation of plutonium in nuclear reactors, and extraction and processing of enough fissile material required to build the bombs, all in less than 3 years! In wartime delays are much harder to stomach (compare with the case of ITER today).

The Taming of Plutonium: Pu Metallurgy and the Manhattan Project

This arXiv preprint outlines one of the unanticipated challenges arising during the Manhattan Project, that "plutonium would prove to be the most complex
element on the periodic table." At ambient pressure, plutonium exhibits six distinct solid allotropes (crystalline structures), more than any other element! The differing densities of the allotropes greatly complicated the processing of the plutonium into a bomb core, requiring the development of a suitable alloy to stabilize the plutonium into a single phase.

Trinity, by K. T. Bainbridge

A technical report outlining the organization of the first nuclear weapon test, practical challenges that emerged, and the data that was obtained.

Los Alamos and ‘‘Los Arzamas’’

A brief comparison between the American and Soviet nuclear weapons programs, the latter headed by Yulii Khariton who was sometimes called the Soviet Oppenheimer by his colleagues. But unlike Oppenheimer he remained the scientific director of the institute for more than 40 years.

Physics models that are wrong but useful

 "All models are wrong, but some are useful" is a saying usually attributed to statistician George Box. In physics we are often tempted to create a model that might be correct, but ends up being hopelessly useless. 

For example, the multi-particle Schrodinger equation in principle can give us an exact description of the energy levels of any molecule we would like to study, underlying the field of ab-initio quantum chemistry. But it cannot be solved except for the simplest of molecules. Heuristic approximation schemes which may not rigorously justified are essential to obtain useful predictions for large problems of practical interest. String theory is another example, with some arguing it is not even wrong.

There are many neat examples of models that, while wrong, lead to useful predictions and progress in our understanding:

• The Drude model of electrical conductivity. In the original paper there was a fortuitous cancellation of two big errors yielding agreement with experimental data for the specific heat. Nevertheless, the model remains a very good approximation for the frequency-dependent conductivity of metals.
• Conductivity at low temperatures: Before 1911 there were various predictions for the resistivity of metals cooled to zero temperature: zero, a finite value, and even infinite (argued by Lord Kelvin). Efforts to determine which prediction was correct led to the unexpected discovery of superconductivity.
• The Quantum Hall effect: quantization of the Hall conductivity was originally predicted in the absence of scattering, and thus the quantization was expected to only hold to a finite precision. Effects to measure a finite accuracy of the quantization led to the Nobel Prize-winning experiments.

A good model doesn't need to be 100% correct. A good model needs to give an actionable prediction.

arXiv highlights

Here are some papers that caught my eye over the past month:

Germain Curvature: The Case for Naming the Mean Curvature of a Surface after Sophie Germain

This essay argues that the intrinsic curvature of a surface, aka the Gaussian curvature, should be named instead the Germain curvature, since Gauss was not the first to study it.

I remember attending a lecture by Sir Michael Berry (of Berry phase fame) where he made a compelling argument against naming new objects or effects after people, on account of the three "Laws of Discovery":

"1. Discoveries are rarely attributed to the correct person

2.Nothing is ever discovered for the first time

3. To come near to a true theory, and to grasp its precise application, are two very different things, as the history of science teaches us. Everything of importance has been said before by someone who did not discover it."

Indeed, versions of the Berry phase had been previously decades before Berry, by Pancharatnam, Rytov, and others. For this reason he prefers the name "geometric phase." Similarly, intrinsic curvature is perhaps a more suitable alternative to Gaussian curvature.

The problem with naming effects after people is that the nature of the effect becomes opaque unless one already knows what it means. The situation becomes even worse when different groups decide to name the same effect after different people. On the other hand, simple yet descriptive names including geometric phase and intrinsic curvature reveal some sense of what is meant to the outsider. The absence of a simple-sounding name may indicate that we don't really understand the effect.

An Aperiodic Monotile

The authors discover a family of shapes that can tile the 2D plane, but only aperiodically. The shapes are non-convex mirror-asymmetric polygons. Tiling the plane involves placing a mixture of the polygon and its reflection, but the two can never be arranged to form a regular pattern. Can this kind of aperiodic tiling lead to novel physical properties of some system or model? For example, tight binding lattices can be obtained from tilings by identifying corners as "sites", with coupling between sites linked by edges of the tiling shapes.

Spectral localizer for line-gapped non-Hermitian systems

The localizer theory I have discussed previously (here and here) is now generalized to non-Hermitian systems! This is relevant to understanding the properties and robustness of certain topological laser models.

A quantum spectral method for simulating stochastic processes, with applications to Monte Carlo

This preprint shows that the quantum Fourier transform can be used to efficiently simulate random processes such as Brownian motion. In contrast to previous "digital" quantum Monte-Carlo approaches, here the authors consider an encoding in which the value of the random variable is encoded in the amplitude of the quantum state, with different basis vectors corresponding to different time steps. Since Prakash's earlier work on quantum machine learning using subspace states was the inspiration of our recent quantum chemistry work I think this paper is well worth a closer read!

Photonic quantum computing with probabilistic single photon sources but
without coherent switches

 If you want to learn more about the photonic approach for building a fault tolerant quantum computer (being pursued by PsiQ), you should read Terry Rudolph's always-entertaining papers. Even though the approaches presented in this manuscript (first written in 2016-2018) are now obsolete this is still well worth a read as a resource on the key ingredients of potentially-scalable methods for linear optical quantum computing.

An Improved Classical Singular Value Transformation for Quantum Machine Learning

The field of quantum machine learning has seen two phases. The first phase was sparked by the discovery of the HHL algorithm. HHL and its descendants promised an exponential speedup for certain linear algebra operations appearing widely-used machine learning techniques, arguably triggering the current boom in quantum technologies. However, running these algorithms on any useful problem will require a full fault-tolerant quantum computer.

Consequently, novel quantum algorithms for machine learning have attracted interest as a possible setting for achieving useful quantum speedups before a large scale fault-tolerant quantum computer can be developed. The power of these newer algorithms is much less certain and still under intense debate. Nevertheless, researchers could find solace in the hope that, even if these NISQ-friendly algorithms do not end up being useful, eventually we will achieve a quantum advantage using HHL-based algorithms.

The dequantization techniques pioneered by Ewin Tang and collaborators are starting to suggest that even a quantum advantage based on fault-tolerant algorithms such as HHL may turn out to be a mirage. This paper presents a new efficient classical sampling-based algorithm that reduces the potential quantum speedup for singular value transformations from exponential to polynomial. This affects a variety of quantum machine learning algorithms, including those for topological data analysis, recommendation systems, and linear regression.