The Replicability Challenge in Social Sciences
Your editorial on social science research highlights the poor replicability of results, a problem sometimes exploited to dismiss the entire field. In the complex study of human behaviour, factors like methodology, statistical misuse, and sample variations contribute to this issue.
The Observation Deficit in Human Studies
A key, often overlooked factor is the lack of systematic observation of human behaviour in everyday environments. Unlike natural scientists who observe other species directly, social sciences often rely on cultural knowledge and terms that have evolved for practical living, not scientific understanding.
Dr John Richer notes that using these "active insider" terms, which are subjective and change over time, leads to research that fails to replicate. As Søren Kierkegaard said, life is understood backwards but lived forwards; social science must develop its own "outsider" terms from observation to improve accuracy.
Investing in Data and Tools for Progress
Will Moy, chief executive of the Campbell Collaboration, emphasises the need for optimism and investment. He argues that learning about human behaviour and societies is a vast frontier, potentially more rewarding than studying stars or oceans. However, current social science tools are akin to Galileo's telescope, while modern observatories generate trillions of data points daily.
Data is crucial for natural sciences, social sciences, and AI. Upgrading to public data with better coverage, speed, volume, and detail is essential for scientific progress and effective governance. Language models must integrate with world and people models to fulfil their promise.
Enhancing Peer Review Incentives
Prof David Comerford suggests improving research robustness by shifting incentives. Currently, scientists are evaluated based on authored works, with peer review contributions overlooked. Recognising peer review activity through systems like Web of Science and ORCID could incentivise better reviews.
Editors could award points for excellent reviews and penalise poor ones, encouraging researchers to invest effort in spotting specious results. This would enhance the double-blind peer review system, helping identify replicable truths more effectively.
The social sciences stand at a critical juncture, requiring upgraded tools, better data, and reformed incentives to overcome replicability challenges and advance our understanding of human societies.
