What's the value of a national registration examination?

As a former lay member of the Solicitors Regulation Authority’s education and training board, I have watched the debate about proposals to introduce a standard, national assessment – the solicitors qualifying examination (SQE) – in the legal sector with interest. In relation to the SQE it is worth considering lessons learnt from other sectors – specifically the insight we have gained from using such an examination for pharmacists.

In 1841 Jacob Bell founded the Pharmaceutical Society of Great Britain as a professional body for ‘chemists and druggists’. A year later he founded a school of pharmacy and established a national examining system for membership. Since then, except for the period from 1970 to 1992, pharmacists have always had to pass a national examination to practise. To register as a pharmacist with the General Pharmaceutical Council (GPhC) today, you must pass a four-year MPharm degree at university, a year of professional pre-registration training, and a national examination – the registration assessment – sat towards the end of the training year. The assessment is run by the GPhC.

The assessment is in two parts: one paper assesses pharmaceutical calculating ability, while the other assesses critical reasoning and applied clinical knowledge. The reasoning and application paper consists of single best answer questions (where candidates must use critical judgement to select the best clinical outcome for a patient) and extended matching questions (where a candidate’s broad knowledge of a particular topic is tested through multiple clinical scenarios). Both types of question centre on patients and replicate the sophisticated decisions pharmacists have to make repeatedly and safely in practice in a constantly changing environment.

Calculating ability, critical reasoning, and applied clinical knowledge are essential for practising pharmacists but so are other skills, and it is important to be clear what the registration assessment does and does not do. In particular, it does not, and has never claimed to, test communication skills or professionalism in practice (other than in a clinical sense), which is why it is one part of a broader diet of assessment. This includes four years of assessments at university and work-based assessments by a pharmacist tutor during the pre-registration training year.

The assessment is overseen by a board of assessors made up of ten pharmacists with educational expertise and two healthcare assessment specialists. Questions are written by a cadre of 60 practising pharmacists and then evaluated by a separate group of 30 standards setters, who are all practising pharmacists with direct experience of pre-registration trainee pharmacists. Requiring a multi-stage development process means that every question is evaluated by in excess of 20 experienced practitioners before it is included in a paper. Pharmacy practice varies slightly between sectors and countries (the GPhC regulates in England, Scotland, and Wales but not in Northern Ireland, where there is a separate regulator), so all permutations are represented on the board and among the question writers and standards setters.

While it may not be an explicit public expectation, there is an implicit public assumption that professionals in training are assessed to a uniform standard – something the registration assessment provides by being the only time in the five years of initial education and training where pharmacy students and trainees sit the same examination and are assessed to the same, independent standard.

Performance data

In recent years, candidates have been drawn from over 25 university schools of pharmacy and around 3,000 pre-registration training sites. Now that we have years of data, we have been able to build up a picture of consistently variable achievement. Some key findings are:

• Candidate performance by university school of pharmacy varies considerably and consistently, with the variation remaining comparatively stable year on year;

• Candidate performance by country of pre-registration training varies considerably and consistently, with Scottish candidates outperforming their English and Welsh counterparts (entry to the Scottish pre-registration training scheme is by open competition, the scheme is managed fully by NHS Education Scotland, and trainees and tutors are supported throughout the training year);

• Candidate performance by sector of training varies considerably and consistently, with hospital candidates outperforming their community (high-street) counterparts; and

• Candidate performance by ethnicity varies too. To illustrate the point, the performance of self-declared Black African candidates has been relatively low year on year, which led us to commission an external study to explore the issue further.

There are many reasons for positive and negatives performance in the registration assessment and we have been careful not to put forward simplistic cause-and-effect arguments. Nevertheless, given the amount of data we hold, we can say that there is an identifiable positive correlation with the following:

• Attending a university which selects rather than recruits; and

• Gaining a pre-registration training place through open competition. This includes places in Scotland, the devolved national health services, and the larger pharmacy multiples. Characteristics of these training environments include support and development for trainees and interaction with other trainees and pharmacy professionals during the training year.

Conversely, a training place not gained competitively and in an environment with limited exposure to a variety of pharmacy professionals and other trainees is likely to impact negatively on performance. For several years we have surveyed pre-registration trainee pharmacists and the characteristics of those who are dissatisfied include being mature, non-white, training in a community (high-street) pharmacy in England, and not having received adequate support.

Table 1: June 2015 first attempt candidate performance – performance by pre-registration training sector

Table 2: June 2015 first attempt candidate performance – performance by ethnicity (categories with >100 candidates)

Table 3: June 2015 first attempt candidate pass rate – performance by school of pharmacy

Legal context

The GPhC’s assessment is similar to the proposed SQE’s stage one assessment but there is no equivalent to stage two. However, there is such an examination in Canada, where national objective structured clinical examinations (OSCEs) are run for pharmacists by the Pharmacy Examining Board of Canada. Key to the success of the pharmacist qualifying examination is the standardisation of its main elements:

• Stations – the activities undertaken in the OSCE – are designed by experts to minimise potential bias and subjectivity as much as possible;

• The participants in OSCE stations – simulated patients – are trained with a script and performance advice to the same standard to give examinees a uniform experience;

• The layouts of stations are standardised;

• Practice examiners are trained in advance to benchmark assessments and their performance is analysed post hoc for consistency by validated statistical methods; and

• Threshold practice standards are set using validated statistical methods.

In short, while standardisation is time consuming and expensive, it can be achieved. (The additional complication in Canada that the SRA will not have to address is running standardised assessments across six time zones and in two languages.)

A legitimate consideration is whether the roles of pharmacist and solicitor can be considered sufficiently similar to apply what the GPhC has learnt about its registration assessment to a legal setting. We would argue they are because both professions require a period of professional training in practice during which both require trainees to demonstrate their ability to think critically, interpret information in complex and uncertain situations, and communicate clearly and appropriately with professionals and the users of services. That being said, it is important to remember that national examinations like the registration assessment and SQE test suitability for day-one practice, not the sophistication and complexity of an experienced professional’s practice.

Leaving readers to draw legal parallels, a junior pharmacist today would be expected to participate fully in the healthcare team alongside doctors, nurses, and others as a front-line clinician (the traditional medicines-dispensing role is shrinking as remote and automated alternatives take its place).

By means of illustration, on a routine basis a junior hospital pharmacist would be expected to interpret complex drug histories, counsel often confused and distressed patients about their medicines regime, decide on clinically appropriate revisions to that regime, and agree it with doctors. A junior pharmacist in the community would be expected to counsel patients in a pharmacy, visit them in care homes or their own home to monitor and modify medicine use, liaise with GPs about treatment, and, after a period in practice and some additional training, prescribe medicine independently – that is, prescribe medicine without first consulting a doctor.

Returning to the GPhC’s assessment, candidates can sit it three times only and those who fail on all three occasions cannot register as a pharmacist. Unfortunately for them it takes £36,000 in fees, student loans, and six to eight years to reach this point. It is important to note that candidates who fail ultimately have passed a university degree and a year of professional training but still cannot meet the national standard of the assessment. The outcome for these candidates is a personal blow, of course, but we feel it is justified on patient safety grounds.

National examinations have long been accepted as an objective gateway to some professions and are gaining ground: notwithstanding the SQE, early in 2017 the General Medical Council consulted on plans for a national registration examination for medics and the GPhC has been contacted by other regulators in the UK and abroad to share its experience of running its registration assessment.

While national examinations will never be loved, our experience is that when one can demonstrate that they contribute meaningfully to maintaining standards in a profession (and shine a light on education and training practice), they are certainly worth considering. I look forward to seeing what conclusion is reached about the SQE.

Damian Day is head of education at the General Pharmaceutical Council

@TheGPhC

www.pharmacyregulation.org

Copyright & permissions