Sceptical Doubts about Meta-Analyses By Brian Simpson

     My son was looking at the use of the statistical measure called meta-analysis, which is a statistical technique which analyses data from different studies, either identifying some measure common to studies, or showing why variation exists. There are a whole lot of formulas to learn and apply, although computers do most of the heavy lifting nowadays, which is just as well, given how challenged social scientists are mathematically. What struck me, was the fundamental philosophical point, that in almost every field, say perception, the theory of consciousness, and even mundane things like diet, statistically significant sturdies are in logical contradiction. So, the idea is to use statistical measures to synthesise data across all of the studies. Forget about the formulae, which tend to mind-numb things. In any areas, the studies are all mutually contradictory. How can a synthesise across them be possible, and if it is, why trust the meta-analysis because isn’t it a question of garbage in, garbage out? Thus, I suppose that a number of IQ studies could be subjected to a meta-analysis, but that rides over the very issue of the conceptual tenability of the tests in the first place, something the early fathers of psychometric, such as Spearman, pondered deeply:
  https://www.nature.com/articles/116436a0.pdf

     I have a really bad feeling about this whole field, and when I get time I hope that I can smash it, not that academics care what a high school teacher thinks, we being sub-standard, and proud of it. Long live cranks, highly useful devices.