The LERN event earlier this month, part of which I summarised here, picked up on the LETR call for more and better research into legal education. The LETR report was tasked to be evidence-based, and thus to inform the second phase of LETR which we are now in. As we pointed out, though, the literature needs improving, and below the fold below I try to outline some of the issues and some tentative approaches to improving matters.
First, here’s a non-LETR example of the problem. I’ve just finished a chapter for a book on simulation, co-written with Emma Nicol. We wanted our chapter to be a meta-review of the literature on digital sims in law in the last 42 years, in common law jurisdictions. We identified over 350 items, and brought this down to a dataset of 123 relevant items. But not a single item contained detailed statistical analysis. A meta-analysis was impossible: we set out a systematic review instead. In that review it was clear that the quality of the literature was highly uneven. The evidence that sim activities in legal education actually improved learning, for instance, needed to be clarified and systematised. Our chapter ended with recommendations to improve the quality of the research literature in the sub-domain. For example, Baernstein et al suggested that a rigorous methodology should contain the following characteristics: [1]
- Greater number of participants
- Multi-institutional focus
- Control or comparison group
- Measure objective outcomes
- Measure validated outcomes
- Measure outcomes at least one month after the intervention
- Conduct the intervention more than once
- Estimate statistical power.
Now it might be argued that the triangle of sims, tech & law is a small corner of the HE educational field, and a fairly specialised one at that. The same is true, though, of many aspects of research into technology-use more generally in legal education. In our webcast project a decade ago, for example (published here amongst other places), we had to rely on systematic reviews and meta-analyses from medical educational literature. There were none in Law. In other words, as early as 2004, which is when we carried out the research, there was already extensive research in the field of education & technology into the use of audiovisual channels via the internet. This had been taken up and built upon in the field of medical education. And in addition, medical education researchers had carried out systematic reviews of those research results for future research activity. Nothing of this had even started in legal education. A decade later, field is still very uneven.
During LETR, and particularly the literature review phase, we found the same again and again in almost every sub-domain of the field. Much of the literature was patchy, there was a lack of overviews, systematic reviews, and almost no meta-reviews. As a research community we urgently need to address the gaps in and improve the quality of our research of legal education.
LETR, of course, is only saying what has been said in various ways in the past. More broadly in Law there has been unease about fundamental research practices. The Law in Context initiative was one example of legal research striking out in a new direction (and with educational implications, of course). William Twining has given a detailed account of its genesis: it was born, he said, of a ‘dissatisfaction with the state of legal education, of legal research and, in particular, of legal literature in England’.[2] The quote comes from Twining’s original paper, given at a seminar held in 1967 to discuss such dissatisfaction, and republished in 1997. The paper is eloquent on the then reductive and narrow formalism of the legal textbook tradition in England; and the seminar was an attempt to give intellectual underpinning to the new type of textbook, and to say what it might bring about. As he points out, the relatively new research focus of law in context was not just a challenge to orthodoxy, but the basis of a new approach to legal research, legal writing, and legal education. In his biography of Karl Llewellyn Twining had of course been immersed in the quite different Realist experiments to change the nature of legal scholarship; and he drew at least one lesson from that tradition for the Law in Context series:
it was clear what [the Law in Context series] was not, but one of the lessons of the history of the American Legal Realist Movement was that it had been more successful in its iconoclastic and negative aspects than in providing a basis for constructing coherent alternatives. (37)
The Law in Context initiative was not the only one to address the nature and quality of research literature. Around the same time the Nuffield Foundation was concerned at the quality and quantity of rigorous research being carried out in law and its infrastructure. In 1971 set up a Legal Advice Research Unit and Social Science Research Fellowships for Law Teachers.[3] More recent concerns were (re-)articulated in the substantial Nuffield Inquiry on Empirical Research in Law.[4]
At the same time, Michael Adler’s Nuffield report on empirical sociolegal research (2007) focused attention on research training. Adler suggested some possible solutions to the problem of decline in numbers of researchers with the skills and capacities to carry out such research:
- Involve the leading socio-legal scholars
- ESRC itself to provide subject-specific training in socio-legal research methods
- Encourage collaboration between law schools and other subject areas and disciplines
- Introduce a dedicated programme of collaborative studentships in socio-legal studies
- Establish a new postdoctoral fellowship scheme
All could easily apply to legal education research. It’s pretty clear that we need to address not just research output, but our research infrastructure: our training in basic research, its funding, its initiatives, the place of our professional bodies in directing research (what roles have they played to date?), the roles of regulators (see LETR Recommendation 25), the roles that our legal education centres can play in implementing research programmes, and analysing and drawing upon the essential interdisciplinary reach of educational research. Nor should this be limited to England and Wales. The problems of legal educational research are global and we can look globally for solutions. What’s happening in Australia, for example, and what can we learn from that? More in later posts.
- [1]Baernstein, A., Liss H, Carney, P., Elmore J. (2007). Trends in study methods used in undergraduate medical education research, 1969-2007. Journal American Medical Association. 298(9), 1038-45. Quoted in McKimm, J., Preston-Shoot, M. (2010). Teaching, Learning and Assessment of Law in Medical Education. Coventry and Newcastle: The UK Centre for Legal Education (UKCLE) and The Subject Centre for Medicine, Dentistry and Veterinary Medicine (MEDEV).
McKimm & Preston-Shoot comment of their own field, medicine and law, that ‘much published research focuses on local processes and relies on student satisfaction and short term acquisition of knowledge’ (2010, 14). They also mention the problem of ‘insider research’, namely that many effects were observed and recorded by staff who were already involved in the educational intervention, rather than by trained outsiders (2010, 14). This was problematic in our dataset too.↩
- [2]Twining, W. (1997). Law in Context: Enlarging a Discipline. Oxford, Oxford University Press, 38.↩
- [3]Cited in Adler, M. (2007). Recognising the Problem: Sociolegal Research Training in the UK. Nuffield Foundation, 1.↩
- [4]Genn, H., Partington, M, Wheeler, S. (2006). Law in the Real World: Improving our Understanding of How Law Works. London, Nuffield Foundation.↩