Psychology really needs to clean up it's act. They should be the best at this stuff since they know all the mental games we play on ourselves.
[www].npr.org/sections/health-shots/2018/08/27/642218377/in-psychology-and-other-social-sciences-many-studies-fail-the-reproducibility-te
Agreed. The sample sizes are usually too small and the study designs are not as rigorous as they should be.
I have read about the replication issues that have popped up in Psychology over the last couple of years. While methodology used in some of these experiments was flawed, many of these experiments are quite legitimate.
The problem (as I understand it) has to do with the actual replication in terms of the subjects used and the locals where these subjects live. These two areas can cause divergent results depending on the subject being studied. I think any psychological study trying to replicate results must try in so far as it is possible to replicate not just the questions being asked or the actions being requested, but must try to replicate the subjects that were studied, taking into account their comparative socioeconomic situation, as well as the regional differences.
The purpose of the studies are to show something about human behavior. If it cannot be replicated, then maybe it doesn't say what the original authors thought it said about human behavior.
@JeffB
Harvard 2016
"In an attempt to determine the replicability of psychological science, a consortium of 270 scientists known as The Open Science Collaboration (OSC) tried to replicate the results of 100 published studies. More than half of them failed, creating sensational headlines worldwide about the "replication crisis" in psychology."
Daniel Gilbert (Edgar Pierce Professor of Psychology at Harvard University) said the following:
"Readers surely assumed that if a group of scientists did a hundred replications, then they must have used the same methods to study the same populations. In this case, that assumption would be quite wrong. Replications always vary from originals in minor ways of course, but if you read the reports carefully, as we did, you discover that many of the replication studies differed in truly astounding ways--ways that make it hard to understand how they could even be called replications."