Conference final thoughts

by Paul Maharg on 30/01/2015

The sessions were pretty short — 15 mins, no questions.  Julian mentioned he was bursting to ask questions, make comments, and that was my experience too.  But interestingly, the constant roll of presentations allowed us to see the comparisons and contrasts, the common points between the presentations.  There were some stand out approaches to assessment, and these were invariably the work of experience in working with students, a learning about what students need in specific contexts — a Deweyan approach to learning that dwells upon the transactional (in the epistemological sense) relationship between students and knowledge and skills, and the many other factors that are involved in the multi-factorial environment of legal educational assessment.

There was also a dearth of real assessment and evaluation of the methods.  Very little statistical analysis, very little sense of the literature, too much description of projects, much enthusiasm to be sure, plenty of hard thinking and planning.  But on the whole, insufficient analysis of the results, and insufficient long-term follow up of student feedback, results, etc.

But I can’t end on that note.  This was a really enjoyable and very focused and intense one-day conference, raising lots of issues that got to the heart of assessment issues in legal education.  Many thanks to Alison Bone for organising and pulling together the conference – great work – and to the presenters for presenting their ideas.  Before the end, Nigel Duncan mentioned the possibility of development of papers and how we could do it, based on a paper that we produced a while back.  More to follow later on that.



Post-lunch now, and Penny English, Anglia Ruskin U first up, on ‘Using posters as a means of summative assessment’.  She defined it from Handron 1994 as ‘an experiential learning activity that stimulates curiosity and interest, encourages exploration and integration of concepts and provides students with a novel way of demonstrating understanding.’  As she pointed out, the tightness of the format forces clarity and precision.  Bit like Twitter?  She cited the benefits as being the following: engaging and satisfying; develops comms skills; works well as a group exercise; requires selection, synthesis and evaluation of information; it’s novel and challenging; and it’s hard to plagiarise.  Downsides: novel, so students need very clear guidance on expectations; initial resistance; can lead to focus on visual impact rather than content; expense (of printing a poster).  She showed some good posters, on the subject of International Law and the use of white phosphorus as a weapon, and cluster munitions.

Rory O’Boyle (Northumbria U) and Frieda Grealy (Law Soc of Ireland) next, on ‘Ethics in Practice: The use of SCs’ methodology to teach and assess professional ethics and communications skills’.  The study was part of the Certificate in Legal Ethics and Lawyering Skills.  The aim of the project was to teach a substantive topic – solicitors’ regulation and complaints procedures, and to highlight issues of professional responsibility and ethics.  This, as Freda said, is usually taught in a narrow defensive way, and based on flawed assumption of inherent values, and is a test of moral courage and commitment.  It reinforces the values of the profession and the Society.  Socialisation occurs, for legal trainees, in the firm, where there are communities of practice that affect morals and ethics.  Rory pointed out how important comms skills are.  They drafted the scenario — ethical issues were based on actual ethical experiences of solicitors.  There was a client interview role play, instantaneous formative feedback and debrief, a client letter, reinforcing learning, and assessment of their overall performance. Multi-layered.  Students loved it.  Great presentation.  Good to see them both again, and to see SCs being developed by the Law Society of Ireland.

Lucy Yeatman and Sarah Crofts, U of Greenwich, on ‘Assessing research skills’.  As they point out, LETR recommended adopting the BIALL Legal Information Literacy Statement.  They described what they did at Greenwich.  eg staff hand out coursework brief, then a library seminar, then students begin assessment preparation record (APR), then interim feedback on APR, then students hand in coursework plus the completed APR and research certificate.  They have us their assessment criteria, that were incorporated into the Library assessment.  The assessment was neatly tied together, in a core subject of the curriculum.  Their message: you can’t assess research skills in an exam.  It’s about encouraging the students to take responsibility for their own learning.  Can be assessed in different ways: short answer tests, reflective logs, annotated bibliographies, clinical work, pbl, opinion writing, moots, etc.  Good presentation, sound, perceptive points on teaching & learning legal research.

Lynn Cherril-Teesdale, Oaklands College, on ‘A reflection on teaching law to non-A level law students: assessing law units on (some) vocational courses or, what the majority of FE students actually do’.  Lots of data on FE Colleges.  Vocational law is taught by non-lawyers, eg Hearlth and Safety law for construction student is taught by construction lecturers.

Nick Johnson now.  He’s the Former Rector ILPD Rwanda, talking on ‘Assessing Work based learning with judges – Ex Africa semper aliquid novi’.  Out of Africa, always new things — from Pliny the Elder?  He started by saying, importantly, that globalisation is also about law schools that are non-common law, not white, and in the developing world.  He made a good point that for students who start their careers, the erosion of legal categories is critical.  He spoke of the recovery of law after the genocide, and the extraordinary story that that is.  He described a curriculum that was part CL, part civilian.  There’s a transition from French to English; the legal tradition is civilian in origin; there’s initial professional training of lawyers, judges and prosecutors together.  The backlog of qualification was extreme.  Legal education was made compulsory for all three categories.  There was assessment of work based learning and a postgrad diploma in legal practice.  Compulsory from 2013 and three years for experienced practitioners to complete.  He refocussed on to legal skills with pathways for each profession.  There were four modes of delivery with common learning outcomes.  In the curriculum, formal learning was by lecturer, WBL by students, and mentoring of students on WBL.  There was a minimum level of professional experience stated.  Nick said that was essential.  In terms of learning outcomes, there was 50/50 formal learning and WBL – a conversion of traditional LOs to work based tasks.  He designed a learning log for each of the three categories of students — judges, lawyers, prosecutors.  Mentors were important.  So too was authenticity and situated learning.  Greater emphasis on result than process.  Very interesting paper.

Chris Maguire, Ishan Kolhatkar, Damian Smith & James Welsh, BPP Law School, on ‘Mind the gap: the limitations of assessment as an assurance of competence’.  Interesting question to start with: what’s a key question we ask now that we didn’t ask before 2000?  A: ‘where are you?’  The place of mobile comms.  Chris is worried about the place of knowledge, and how we assess it, and how our practices are changing.  Eg what is assessed is not intended to have been assessed, and that goes for teaching & learning as well.  Began by looking at the gap between undergrad and p/g vocational assessment.  How can assessments bridge the gap?  And can they satisfy the needs of students who are not going to go into the profession. How about aptitude tests, based on critical reasoning?  Now, apparently, according to the speakers, regulators are moving towards knowledge tests (sounds a bit like part of the US Bar Exam?  And sure, this doesn’t sound like a particularly good way of identifying good barristers, but of course since numbers are the game, it isn’t really meant to be, is it.).  Final speaker spoke of legal skills + pro bono clinic in LPC, and integration with knowledge, APL (accreditation of prior learning) should be part of the mix, too.

Final session of the day — Chris Hull, St Mary’s College University, on ‘Viva voce as an assessment method in undergraduate legal education’.  Not the norm in legal education.  His vivas can last 30 mins.  The rationale for: authentic assessment; they probe knowledge; promote a deep approach to learning; student learning styles; academic integrity.  There are concerns with: anxiety, impairments, anonymity, bias, assessment is time consuming, the novelty of the assessment, and recording of the viva assessment.  Other considerations included assessment design; the practicalities involved, the reliability and equity of the assessment; marking issues and responsibilities; and the guidance given to students and the preparation of students by students themselves and by staff for the assessment. From the questionnaire feedback (at last, someone gives us some stats, even if only aggregated percentages), some results were mixed, but there were great results too especially on transferable skills and depth of knowledge.  Chris thinks it a viable form of assessment design.  Guidance and practice is key for staff.  The approach develops a deeper approach to learning and develops additional skills, and in his opinion (and of many of his students) it is a viable alternative to written work.



Am liveblogging the conference as much as I can.  Julian and I up first, slides on the Slides tab.  Whirlwind tour of past & present on the theme of the title, ‘Of tails and dogs: Standards, standardisation and innovation in assessment’.

First up, Craig Newbury-Jones and Nigel Firth, Plymouth U Law School, on ‘Digital assessment for the YouTube generation: reflective practice in 21st century legal education’.  The presentation gave us an overview of innovation and technology-enhanced assessment, and contextualised it in ongoing curriculum changes.  Year 2 students operate as a virtual firm using f2f and online dialogue and meetings to progress their client’s case.  Allocated tutors act as clients and supervisor.  The module is a prelude to their work-based learning options in the Clinic.  The emphasis is on transferable skills rather than law/procedure (60% or so of their students don’t enter legal practice).  Blogs and file/firm updates lead to incremental development of their file that case tutors monitor and comment upon.  Part of the environment is the SANSspace, where students upload their reflections on their virtual practice.  Originally Sony language lab software —  Interesting use of this software for reflection online — but Craig said poor interface, not liked by students.  ELGG was a much better bridge between academic and practical legal learning.  As they say, oral reflection is an employability skill; and it helps to signpost to students what they are actually doing.  I agree.  It’s kind of link a constructed debrief outside the sim, which is so important for learning.  Students love the casework, combination of lecture and practical sessions has been engaged in by students, and excellent assessment results from students.

Next up, Dan Hill, head of learning & development, U of Law, on ‘Digital assessment evaluation’.  Assessment is labour-intensive for academics and admins: could digital help streamline the activities, eg marking process, and other pinch-points?  UoL used ES, ExamSoft, which gives students feedback, which was based on use of assessment criteria. Advantages — gives students indication of their performance and areas for improvement. Feedback is generated from comments made during the marking process, therefore no additional work for markers.  UoL also uses QM – QuestionMark.  It provides Coaching Reports, roughly same ideas as ES.  Also shows student’s metrics against other students.  That aspect is neat; but worrying, too.  There’s also an assessment summary report for faculty.  What did students think about both systems?  Liked ES better than QM — clearer feedback, and level of detail.  Students seemed to prefer working with paper-based assessments.  Most students agreed that both stores drained the batter life on their laptops, and wanted paper copies of the assessment, rather than having it in digital form only.  33% preferred to type, 67% preferred to write.  So issues there to think about.

Next, Rachel Dunn and Richard Glancey, Northumbria U, on ‘Assessing students through engagement with legal policy’.  No slides.  They talked about a Student Law Thinktank.  Composition of policy papers, consultation responses to Consultations put out by Parliament, Select C’ttees, and others (eg on whether we need a new Magna Carta).  Gives students skills such as legal research skills (different from client-based legal research skills); legal writing skills – more developed in a policy domain.  Different student groups work together vertically, eg undergrads, LPC students, doctoral students.  Richard wondered if assessment could be developed out of this experience.  So in Civil Liberties students were asked to compile a report, group written, 70% + oral assessment, 30% (if I have that right).  Eg on Terrorism Act, privacy, compensation for miscarriages of justice.  Students identify area of law, critique it and propose changes.  If the submissions are high quality, Richard sends them off to the relevant body.  Richard introduced it three years ago — disappointed because students couldn’t work together in groups.  He then introduced PBL to address the issue, as the basis of a group learning method.  The results were ‘unbelievable’ — all the submissions were good enough to send off to the bodies concerned (unlike the first year, where none were sent off by Richard).  This is a real assessment for students, said Richard.  When at Northumbria Richard and I talked about this — it’s an impressive project, and well worth being further developed and disseminated.

Alison Bone is keeping us all to time well.  Jessica Guth, U of Bradford now, on ‘Thinking critically about law: Book reviews in law and society’.  Her aim was to evaluate different theoretical approaches to the study of law.  Assessment via oral comms, written comms and advanced research & analytical skills.  She allowed the students to crucially examine a range of books, eg a textbook, an extract from a textbook, revision guide, etc.  There was an individual presentation of no more than five mins introducing the text they’d chosen to analyse and outline the core argument, and a written critical analysis of the same text that students analysed in their presentation.    Almost no module fails — just the non-attendees.  Eg of student work — review of Glanville Williams’ Learning the Law, entitled ‘Learning the Law: What Law’ (pointing out the narrow focus of Williams’ text).  Eg – an analysis of a landlord textbook called ‘This is not my house: Female Asians and land law’.  Nothing about extended families, granny flats, moving flexibly from one house to the next.  Students achieving good marks.

Nigel Duncan up next, on ‘Prepared for practice? Assessment for the Bar 1975-2015.  An overview of vintage Duncan stuff.  Legal knowledge, acknowledged by the Bar to be not enough — Hurrah.  He also analysed what was required for General Papers 1 & 2 — lots of cramming, lots of basic work.  3 hour closed book exam in 1975, unseen examinations, most require application of substantive and procedure law, very little application & skills.  Contrast with BPTC in 2015.  Closed book assessments are there, but are multiple choice and short answer questions, centrally-set, no question-spotting.  There are open-book assessments on Opinion writing and Drafting, 3.5 hours, dealing with uncertainty and contingency.  Quality of the drafting can be stunning.  Simulated assessments in Advocacy in Civil submissions, cross-examination in chief, Conference skills and two option assessments, eg Free Representation Units, where they can take specific types of caselaw.  Constructive alignment?  In 1975 there was a diet of lectures and tutorials, and compulsory practical exercises in Advocacy, and there was constructive alignment.  In 2014, there’s a varied diet of assessments, and considerable constructive alignment there too.  But the quality and differentiation between the two sets of work students did do in 1975 and are doing in 2015, is remarkable.  I like this, because it answers one of the criticisms I have of legal education research, namely the historical comparison of what we do now in legal education compared with what it was in the past.  Nigel has more confidence in what students are doing now as being quality learning and engaging in quality assessment.  Thoughtful, thorough piece.

Pat Feast next, from Portsmouth U., on appraisal: ‘Appraisal as an effective method of assessment’.  Based on a module called The Practical Lawyer, a 40 credit unit that replaces dissertation at level 6.  Students work in a number of settings offering legal advice, and trained over a two-year period, and students commit 100 hours to a particular project.  Before they go into the project, the students are assessed to ensure they have the necessary skills. Two assessments in the level 6 project: the work students do for clients, and then an appraisal that incorporates a reflective essay.  Appraisal replicates what the student can expect when employed, asks them to reflect on their performance and think about their future.

Now it’s Clare Sanford-Couch and Jonathan Bainbridge, from Northumbria, talking about a module on legal history, where students are involved in setting their assessment, and in marking those assessments.  The Legal History module is a half-module, for students in the upper school, mix of lectures & two hour seminars, assessments by coursework and oral assessment.  Students don’t like formulating the question.  Why was this done?  Because, quoting LETR, ‘it was widely recognised that legal research skills were not sufficiently acquired by the end of the academics stage’, p.44.  And for general transferable skills: analysis, synthesis, report writing, time management, self-monitoring and goal setting. The oral assessment carries 30% of mark, 15% from module tutor, 15% from students’ peers.  This assessment is about presentation skills.  Students seem to like this oral assessment — peer pressure, but also peer support, is part of the deal.  Why do this? Student is active ‘doer’, helps to install autonomy in learners, interactive classes, transparency in assessment.  But possible pitfalls: doubts about the validity and reliability of peer assessment; student dislike of innovation; student concerns about peer assessment, friends marking each others’ work; inconsistent/arbitrary marking; the role of the tutor.

Final morning session (puh, I’m wilting… but hey, I see from The Guardian live update blog from Melbourne that Andy Murray’s into the final…), Elle Dagilyte, Buckinghamshire New U. & Peter Coe, Aston U.  Developing professionalism via take-home exams: Assessment for learning in law studies.’  What has been developed and why?  Take-home assessment for LLM & LLB students.  Because Law is both an academic and practical discipline.  There’s an emphasis on professionalism: their institutions and LETR, too.  Their definition of professionalism was interesting, quite comprehensive, too.  Take-homes are used in Canada, McGill, Sweden, Uppsala, and Australia, Melbourne.  In the UK, only LSE use it.  Challenges and solutions — see table below. Small cohort in LLM –4.  Larger in LLB — 30 students, both international and home students.  Students comment that it feels like an exam, but the results showed better performance as compared to a prior mock exam.

And that’s the morning session.  I feel like the liveblogger on the Andy Murray game…  Lunch now then back into the blogging & think about the Lord Upjohn this evening.  Busy or what.


50 years of assessment in legal education

January 29, 2015

This is a conference hosted by the Association of Law Teachers at the Institute for Advanced Legal Studies, today, and part of their 50th anniversary celebrations (there’s a 50 Years of Legal Education conference later in the year), which are looking back as well as looking forward to the future(s) of legal education.  Maybe it’s […]

Read the full article →

New media and digital research literacies for legal educators

January 29, 2015

Second presentation to LERN on this subject, this time a seminar of three hours rather than 20 or so minutes.  Also gave a version to staff at Nottingham Trent U Law School.  The subject seems to be attracting interest, at last.  We so need to develop this as a discipline, and we also need to […]

Read the full article →

Simulated Clients @ Chinese University of Hong Kong

January 29, 2015

Been travelling recently, so not much posting.  To Hong Kong in early December, training Simulated Clients for the Chinese University of Hong Kong Faculty of Law, organised by Elsa Kelly.  Spent four intensive days on scenario and assessment standardisation, with 10 clients.  The sessions were attended by Matthew Cheung and Martin Doris.  Martin and I go […]

Read the full article →

LERN: digital media and legal education research

November 5, 2014

Am doing a brief slot for the Association of Law Teachers (ALT) Legal Education Research Network (LERN) today on social media and research, in a workshop designed to explore, in the words of Patricia Leighton, the ‘various ways that we can make people aware of our research and its findings, moving beyond the “traditional methods” […]

Read the full article →

Convergence and fragmentation

October 8, 2014

I’m giving a paper today at Melbourne Law School, by kind invitation of Gary Cazalet, title ‘Convergence and fragmentation: legal research, informatics and legal education’.  Slides up on the Slides page above.  The paper is a version of draft chapter five of a book I’m writing, Genealogies of Legal Education (interim chapter titles in the […]

Read the full article →

LETR on regulatory relationships

September 22, 2014

I was revisiting LETR on regulatory relationship for a paper I was giving here at Denver U Sturm College of Law.  A year or so on, how is it looking?  The responses of the main regulators were reasonably predictable though the future consequences of their actions are difficult to foresee.  But what of the report itself? […]

Read the full article →


September 22, 2014

MacDiarmid had the words for it: The rose of all the world is not for me. I want for my part Only the little white rose of Scotland That smells sharp and sweet – and breaks the heart. In truth the Referendum vote, Yes and No, is a continuum into the future.  In that future […]

Read the full article →