On the World
Aspect / Aggregation via Hypothesis Tests
NEVATHIR
August 24, 2018
Hypothesis test forms a corner stone of modern sciences as a method to make explicit distinction between parameters. There is little qualification and condition on parameters, in order to accommodate as many sciences as possible. Aggregates from samples are the means of evaluation. Because many microstate configurations form identical aggregates, evaluation on aggregates generally can not reveal system structure. Thus, hypothesis tests provide dubious conclusions if parameters are structural, which suggests that lack of restriction on parameters and general applicability are both questionable.
Consider a hypothesis test on a coin, with equal probability for aggregate heads and tails. Modern scientific SOP claims that the coin is fair. However, a coin capable of switching probability parameters for heads between p and (1-p) may give fair aggregate results provided with equal draws for both parameters. Thus, the coin may not be fair, given fair aggregates.
Conclusions about structural parameters in almost every science are likely fraudulent, if hypothesis test is the exclusive source of evaluation. Areas like quantum mechanics are mostly unaffected because statistical aggregates are principal theoretical predictions and experiment collections. Economics, medicine, and psychology may experience more unreliable structural findings with arbitrary hypothesis tests.
Hard sciences are not exceptions. Quantum field theory makes structural premises about field quantities that entail cosmological constant catastrophe.
While many economists certainly recognize the similarity between the argument here and the Lucas critique, it has to be emphasized that quantum mechanics constitutes a successful theory based on statistical aggregates. Lucas critique is useful, but there is nothing wrong building theories based on aggregates rather than structural parameters, provided that scientific predictions are valid. Contrary to popular perception, Lucas critique does not follow from the discussion above. Building models with structural microstates or statistical aggregates is a question of taste, not a logical principle. What's wrong is to draw conclusions on structural parameters exclusively from naive evaluations on aggregates.
Details about Type-I and Type-II errors, along with other sorts of errors, won't appear here, but a caution is provided that error rate may be close to 100% with p-value 0.00000000005. Under extreme configurations, hypothesis tests may provide junk rather than useful exploratory guidance.
Some problems, like those in game theory, are easier to answer with structural microstates, others, like those in Keynesian macroeconomics, with statistical aggregates. Complexity is a natural basis for efficient model development. The power of rationalism together with hypothesis tests may rule out structural parameters in disagreement with aggregate measurements, but determination of system structure generally requires more advanced evidentiary methods beyond rationalism.
Naive mathematical models based on structural parameters may be as fallacious as hypothesis tests due to dubious aggregation mechanism modeling, like several past macroeconomics models for financial crises and business cycles. Thus, we conclude with the recognition of rationalist fallacies and encourage further development of better empiricism.