LETR conference: reflections

I said in my first conference post that I was hoping for the conference to help me understand LETR’s continuing significance, if any.   I left with more questions in my mind about LETR’s purpose, but also a sense that what we co-authors made of it was at least in parts enduring beyond the five-year mark.  We presented our own interpretations of aspects of the report in the initial panel session – the slides at the slide tab above gives a sense of what we said.  With regard to my section, on regulatory relationship, I’ll say more in the next post or two.  Both Jane and Julian have written about their sections in more detail, too, and their work along with mine that was published in the last five years can be found in this reference list.

LETR lasted two years, and in that time saw several changes of personnel at the SRA and in other regulators.  It comprised a 350-page report, a literature review of 276pp, multiple forms of research reporting (qualitative and quantitative, questionnaire, online questionnaire, interviews, group interviews – more of that in another post), and a methodology that was new to legal educational reporting.  In size, types of research undertaken, methodology and content it was quite new.  Nothing like it had ever appeared before.  It set new standards for the quality, depth and range of reporting on social and educational issues arising from professional legal education which as we pointed out in the panel session had been taken up by other jurisdictions internationally.  Indeed it seemed to us authors that LETR was at least as well read internationally as in England and Wales.  As I described in a previous post, the responses in E+W were mixed: readings, misreadings and unreadings that ranged from the peevish, to uncomprehending, to mixed, to acute critique and future-oriented thinking that springboarded off the data and analyses of the report.  The last were the best we could hope for, and were by far the most sophisticated and challenging readings of both LETR and the current situation for legal education in E+W.

Many of the papers in the conference dealt at least in part with the upcoming SQE. I predicted that the assessment will act against our concept of a shared space in legal education.  That’s already happening: there’s a silence about what law schools will be doing on this score, as each of them prepares financial and educational strategy, each in their silos.  It was also pretty clear that the SQE would increase the marketisation of legal education, and in her presentation Julie Brannan pointed to this as being an outcome of the assessment.  We’re in agreement; but where we disagree entirely is in the effects that that will have upon the nature and culture of legal education.

Our recommendations were accepted by the SRA on communications and outcomes, but all of us need to focus more on the standards and implementations required.  We could do worse than take exemplars from other disciplines.  A good example is the excellent work of the Canadian rhetorician, Lorelei Lingard, and her colleagues. See for example her work on rhetorical relevance in referral and consultation letters.^[1]  These are letters that are passed between health care professionals, giving information about patients and their conditions to other professionals.  They involve the writer in analysing the patient’s condition, and writing a reader-centred text that takes account of the complex patient and healthcare professional nexus.  After expert analysis of 104 letters by 16 physicians across three specialties, six factors were found to influence physicians’ decisions about what material is relevant: educational, professional, audience, system-institution, medical legal, and evaluative.  Trainees with no training could only recognise three of these factors at best.  The framework provided a base for novices to understand how to compose such letters, meet the expectations of professional audiences in this genre, and of course assist patients in their paths to wellness.  It gave teachers a sense of what was important in such communications, and how medical comms could be improved; and also a sound basis for evaluation of novice communications.

Compare that with how we assess professional legal communications – do we have a body of work such as that undertaken by Lingard in a single professional genre of letter, in professional legal communications?  Should it not be the responsibility of both legal educators and the regulators to ensure that such work is undertaken?  The point raises further questions.  If this can be done for professional comms, can it be extended to forms of thinking and writing about law, such as statutory interpretation, and legal reasoning, that are more foundational in legal education?  Can it also be done for skill sets that are notoriously difficult to learn and evaluate.  Lingard has also done excellent work in this field, on what she calls ‘collective competence’ – see this brief YouTube interview introducing the idea.

But if LETR was a conference focus, and the professional panel explained their work, and how and why they were doing it, all of us needed to address the long-term failure of the academy to improve the quality and range of assessment methodologies. In LETR we pointed to some fundamental challenges:

• reliance on relatively shallow, vague or narrow conceptions of competence;
• a failure generally to adopt robust methods for deriving outcomes;
• the widespread absence of standardised assessment processes;
• reliance on assessment practices that possess conformity rather than practice validity. (4.141)

We dealt with assessment, inter alia, in chapter four of LETR, with issues such as validity (4.130ff), and methods of assessing the range of outcomes.  By comparison with innovative methods of assessment adopted in other disciplines, legal education has a poor record, on the whole.  Even successful transplants, such as the work on simulated clients (SCs) has not been broadly taken up by the academy.  SQE will now change that, but regulators should not be leading on this – we should all be part of a shared space of thinking, analysis, research and practice; and academic resistance to change opens up a vacuum that is being filled by regulatory action.^[2]  Internationally, there is more interest – see the work of John Garvey at the University of New Hampshire Law School, and that of Wilson Chow and colleagues at Hong Kong University Faculty of Law.^[3]

Finally, in our LETR panel we mentioned the need for the academy to improve its organisation of legal research.  I’ve written about this elsewhere on this blog (eg toward the end of this post) so won’t go into detail, except to say that this conference on LETR was no better illustration of the pressing need for such organisation.  In front of a panel of professional regulators from E+W, who was quoting meta-reviews, systematic reviews, summaries of research, policy papers, and the like?  No one, because there are none.^[4]  The evidence base is scattered, ill-sorted, hard to find, highly variable in quality – a situation that makes us inarticulate as a discipline.  LETR was required to be evidence-based, and for us authors it was a huge challenge just to organise such an evidence base because the work of sorting, reflecting and retrospective analysis had not been done by the academy.  You can see the results in the rendering of our Zotero reference lists at the LETR website, around 2,000 references covering the topics under review, stretching back over 40 years to Ormrod.  That was up to date back in 2013.  Has it been updated since then, by professional bodies such as ALT, SLS, or regulators, or a group of academics?  Of course not.  This is a culture of research neglect that needs to change; and some of us hope to begin to address that in the next couple of months.  Stay tuned…

My thanks to Jessica Guth and colleagues for such an interesting conference, and for looking after us so well in Leeds.  It was a great idea, well executed in the design of the conference programme, and with lots of challenging ideas from both plenary and paper sessions for all participants to take away and think about.  Its timing couldn’t have been better – a retrospective on a key legal education report, but a conference that also critically examined our new legal educational future under the SRA’s SQE, and the effect that that will have on not just professional education (the focus of LETR), but on undergraduate and part-time degrees as well.

1. ^[1]Lingard, L. et al (2004).  Expert and trainee determinations of rhetorical relevance in referral and consultation letters.  Medical Education, 38, 168-76.↩
2. ^[2]For the initial correlational work on SCs, see Barton, K., Cunningham, Jones, C.G., Maharg, P. (2006). Valuing what clients think: standardized clients and the assessment of communicative competence. Clinical Law Review, 13, 1, 1-65.  The work on SCs, particularly at the Glasgow Graduate School of Law, was the basis for much of the simulation activities developed for the QLTS.↩
3. ^[3]John’s work was the focus of a very positive independent report by the Institute for the Advancement of the American Legal System (IAALS) – see Gerkman, A., Harman, E. (2015). Ahead of the Curve: Turning Law Students into Lawyers.  IAALS, University of Denver.↩
4. ^[4]With once exception – Maharg, P., Nicol, E. (2014). Simulation and technology in legal education: a systematic review and future research programme.  In Grimes, R., Phillips, E., Strevens, C. (eds), Legal Education: Simulation in Theory and Practice, Ashgate Publishing, Emerging Legal Education series, 17-42.↩