The long list of things we do not know
“What we know here is very little, but what we are ignorant of is immense” — Pierre-Simon Laplace
The defining feature of the modern age is the depth of knowledge we have across a vast number of disciplines. Whereas the ‘renaissance man’ of the 1500s (and it usually was a man in the 1500s) could plausibly be across a large chunk of humanity’s collective wisdom, we now live in a world where millions of peer-reviewed scientific papers are published every year. Even researchers in a narrow specialty like orthopaedic surgery are confronted with nearly 50,000 new papers a year; one every 10 minutes.
One thing that fascinates me is that despite the explosion in science and research, there remains wide gaps in our knowledge. Let me provide a few examples.
The Herald recently reported that Kegwoth Public School reversed a decision to stop all homework. The article quotes the NSW Department of Educations’ homework policy, which starts ‘While there is little conclusive evidence of the learning benefits of homework in infants and primary schools, quality homework in these stages may help students to develop effective study habits…”. Primary school homework is an activity that millions of Australians, and billions of children around the world, have undertaken for decades, and yet the evidence base around its effectiveness is surprisingly thin.
Even in pharmaceutical research, possibly the paragon of evidence-based experimentation, limitations to our understanding persist. Paracetamol is one of the most common medications in the world for relief of mild to moderate pain. Most people can attest to its efficacy through personal experience. However, a recent meta-analysis has shown that there is only strong drug trial evidence for a small number of pain types and that very little research exists for repeated use (most drug trials are single-dose).
As a third example, some recent work I was involved in required an estimate for the deadweight loss of taxation. Labour supply curves are one of the fundamental building blocks of economic models, so it’s a heavily studied area. Despite this, some economists question whether we even know the sign of the elasticity when wages rise, and the ‘right’ answer depends on a whole host of factors such as household structure, gender, starting income and time. Even then, estimates will vary.
While there is obviously some cherry-picking in these examples, they are not isolated. There is a huge number of similar cases out there.
What are we to make of this? I think there are a few things.
First, it is worth recognising that, unsurprisingly, all the examples above are intrinsically human – they are about how people interact with education, medications and employment. People are complex and there are limits to how much we can run experiments on day-to-day life. The human element makes social sciences intrinsically tougher than hard sciences.
Second, it means humility in some of the problems we tackle. We cannot expect existing academic literature to provide all the answers for policies and programs. And even solid research findings may not fully translate to different times and places. Where research is uncertain, we can do worse than ask the opinions of people affected by a particular issue. Increasing use of customer voice and co-design recognises the value of lived experience in program development.
Third, it is a reminder of the important role that experts continue to play. Navigating literature and accumulated evidence is difficult; while there are uncertainties, there is a huge amount that we do know from literature and ensuring that important ideas are surfaced and used will remain key to improving practice.
Finally, it highlights the importance of ongoing work to grow the evidence base. In some cases, this is better-designed programs that facilitate measurement and evaluation. The Commonwealth Government’s proposed evaluation-general office would encourage more and better evaluation of programs. And as lovers of data and analysis, actuaries are in a position to help in the important task of building the evidence base in areas that we work in.
CPD: Actuaries Institute Members can claim two CPD points for every hour of reading articles on Actuaries Digital.
