Testing across the pond

We have been reading with interest, from across the pond, the proposals by the Solicitors Regulation Authority for the solicitors qualifying examination, as well as some responses to those proposals. Such significant reform inevitably involves risk, and it is unsurprising that the proposals for the SQE have attracted both strong support and opposition.

It seems like an apt moment to reflect on the approach we have taken in the United States at the National Conference of Bar Examiners (NCBE) and consider how our examinations compare to the proposed SQE.The centralised assessment of legal knowledge, skills, and abilities has been accepted practice for decades in the US. Fifty-four of the 56 US legal jurisdictions use the multistate bar examination (MBE), created by the NCBE, in their admissions decisions.

The MBE is a 200-question multiple-choice test assessing the extent to which an examinee can apply fundamental legal principles and legal reasoning within a realistic scenario. It is a highly reliable test. Each question is carefully crafted by a group of subject-matter experts and then further scrutinised by a team of editors and psychometricians.

We have found that multiple-choice questions are a highly economical and unbiased way to assess examinees’ knowledge.

Still, over the years individuals have voiced concerns about the value of multiple-choice questions. Clearly, multiple-choice questions can’t test all relevant skills (such as, importantly, candidates’ ability to communicate in writing). Critics of our exam have also had concerns relating to face validity.

Partially in response to these concerns, NCBE has developed two additional examinations, the multistate essay examination and the multistate performance test. These tests are also in wide use in bar examinations and admissions.

Bar examining in the US is a major undertaking, requiring the cooperation and coordination of hundreds of subject-matter experts, NCBE staff, and administrators. And yet, in some ways, the US bar exam appears somewhat modest in scope compared with what has been proposed for the SQE.

Our examinations are typically conducted over only two days, whereas it seems that the SQE will require more time. Another major difference is that the SQE will test oral communication and client interviewing. We expect the SQE may prove a bit more difficult for both administrators and examinees given its greater complexity, although this will not become clear until after it has been administered – or at least pre-tested.

Despite these differences, ultimately there are many similarities between the SQE and US bar examinations. Like most US bar examinations, the SQE will use multiple-choice testing to measure legal knowledge. It will test research and writing skills through discursive assessments, such as drafting an extended memorandum of advice. The goals and purposes of the examinations are similar. The knowledge, skills, and abilities being measured are similar. Uniformity, standardisation, and best practices are at the core of both programmes. And the ultimate goal of both is the same: to protect the public from individuals unfit to competently practise law.

We at NCBE look forward to observing and learning from the SRA as it moves forward with the development and implementation of the SQE.

Joanne Kane is associate director of testing at the National Conference of Bar Examiners

www.ncbex.org