I just happened to finish it today. I already knew throughout the book that it would be a very important read, in several aspects. The very end is about the cargo cult in science, and it looks like he put this collection of anecdotes of his life in the sole point of illustrating the cargo cult problem, only exposed in the end. It's almost the sole time he talks extensively about science in this book (and it's why it's so awesome). Cargo cult is the result of a trap that our own brain seem to be wired to fall in. We all know or knew at some point how to avoid the trap, but it gets another perspective when reading about someone that put it in practice his entire life.
Question everything, discover everything, take nothing for granted.
If it's seems unrealistic, just read the book, it'll just teach you how it happens. If it's intimidating, it's by its ruthless simplicity.
> Cargo cult is the result of a trap that our own brain seem to be wired to fall in
That's because we're over-sensitive pattern matching machines. This is good because often recognising and learning patterns quickly is extremely important, far more so than some false positives. Our amygdala, for example, responds very quickly to simple patterns ("AAAAH A SNAKE"). Slower processing can then tell us it's actually a stick. The cost of a false positive VS a false negative here are quite clear.
"Cargo Cult X" is always about us recognising a pattern and just blindly following it without understanding the causal links. Our poor (innate) understanding of probabilities means we can't really combine evidence well, but we can pattern match. Worse, because we start to feel like we understand what's happening we don't delve more into the problem. Even worse than that, we start to ignore evidence that doesn't fit the pattern since those examples look like outliers, and we quickly forget how often we've seen them.
As situations get more subtle than "Snake or stick" the costs of being wrong in different ways change massively. A poor understanding of statistics in healthcare can kill significant numbers of people, similarly with economic or political interactions.
Our brains are awful at complex statistics, but to be fair to them if our ancestors had waited for a double blind study on how likely a tiger was to eat you we would be here. Good work brains, but we need to stop listening to you now.
Information on fear responses and the role the amygdala plays:
Over the years I've come to the conclusion that "science" is all about the customs designed to defeat the brain's imperfections. (It's pretty impressive that we can build such a reliable system on top of a such an unreliable system.)
One important conclusion is that scientists are not more believable than other people. "You should believe me because I am a scientist" is not acceptable answer. That scientist is still a meat-bag, just like you. The entire field of science is built up so we can trust the results of science despite it being done by meat-bags, not because we have created high priests that are more believable.
Cargo cult is the result of a trap that our own brain seem to be wired to fall in.
Yeah, tell me about it. I fell into the trap a couple of months ago. I was tasked with porting my company's browser's 2D graphics backend onto Android OpenGL. My first attempt allowed the browser to draw into graphics buffers that I would load into a texture using glTexImage2D.
A few co-workers felt that this was less efficient than the old 2D APIs, where we would draw into a buffer that would be directly read by the blitter hardware, without having to do a load step. I agreed - on the face of it, if we could arrange things to draw into the texture buffer directly we could save memory and processor cycles, so I started hunting on the Internet to see if anyone else had come to the same conclusion.
It turns out that the guys that were developing Firefox for Android had had exactly the same problem, and they hacked together a solution where they used some internal Android APIs for graphic buffers, that they accessed through dlopen/dlsym. They then linked the buffer to an eglImage, and created a texture from the eglImage. I figured that hey, if the Firefox guys are doing it, then it's probably not to bad an idea.
Unfortunately I should have taken the next step in my investigations, and ask myself exactly what that code that created an eglImage and linked it to a texture was actually doing. If I had investigated that more closely, I would have realised that behind the scenes there's an implicit equivalent of glTexImage2D being done - the problem being that the internal format of a texture is not a simple bitmap. The image is tiled so that OpenGL can draw a texture to the screen with any rotation and have roughly the same performance - if you use a standard bitmap layout, memory paging makes drawing tall thin rectangles much slower than drawing wide short rectangles. So, for OpenGL (or at least OpenGL ES) to be able to draw the texture to the screen, it first reads the data in from the eglImage, tiling it as it goes. This is exactly the same workload that you have in glTexImage2D, and it requires you to keep two copies of the buffer - the one you draw into, and the one that has been tiled for OpenGL.
Anyway, it took me about a month to get the eglImage solution to work (apart from everything else, you are exposed to odd behaviour by the graphics libraries of each Android tablet that your code runs on, that you have to write special code to handle, ie it's not very portable as a solution). Once I realised that I wasn't actually gaining anything from the exercise, I had to go and rip out all of that native graphic buffer code, and put back in the original standard texture code. I probably wasted 6 weeks stuffing around with all of this.
Lesson learned - if you are worrying about optimising code, you need to understand what is being done by the compiler / hardware, and it is pointless changing code until you have that understanding. Don't just accept stuff that you read on the Internet, where possible you need to go and look at source code, or official documentation. Anything else is running the risk of cargo-culting.
Question everything, discover everything, take nothing for granted.
If it's seems unrealistic, just read the book, it'll just teach you how it happens. If it's intimidating, it's by its ruthless simplicity.