UMAP, one organisation currently working to standardise medical student exam question content across medical schools

It’s essentially the organisation which is writing/ collecting a bank of medical student exam questions for use in ‘high stakes’ examinations, such as medical finals.

UMAP’s description of their activities can be found on their website: On the site they describe themselves as follows:

UMAP’s aim is to improve quality in high stakes written assessments across UK medical schools. UMAP runs best practice item writing sessions where clinical and academic staff come together to learn about item writing techniques and to represent their subject area in the developing question bank.

UMAP QA process

Once written, questions move on to be quality assured at UMAP question review meetings which are convened at our partner schools. Staff members with experience in assessment and who are familiar with UMAP style and technique check each question and amend as necessary to ensure the highest accuracy and conformity to question writing principles. Questions are then ready for use and are listed as part of selection documentation available to our partner medical schools.

Schools are invited to select the items they wish to use and then confirm their selections to us. Schools later return results data in an electronic format which is then analysed and uploaded into the UMAP bank. A summary of this data is displayed within question selection documentation to enable schools to make informed, evidence based, item choices.  [Accessed 4.03.09 from http://www.umap.org.uk/about/]

From their site they cover about 14 medical schools and they run an active recruitment of Specialist Registrars to write questions for their bank, most recently to our knowledge in Birmingham at a West Midlands General Internal Medicine training day.

UMAP currently publish on their site that as of October 2008 they have over 2500 questions for use in the these high stakes examinations. They acknowledge that they seem to be lacking in a few key areas, however they are currently working to address this.

What do you need to know about UMAP as a medical student? Well probably not that much. In fact all UMAP really are trying to do is generate sensible MCQ questions that are fair, and are well written. They have a reasonably complicated list of rules that a number of ME’s contributors have been talked through at a number of different times: overall though its not rocket science.

• you shouldn’t be able to answer a question just by using the investigation result or the stem on its own (e.g. a big intro and then showing an ECG with complete heart block etc.)
• The questions are aimed at core FY1 knowledge
• You should be able to guess the likely options for the answer (again straight forward)

In principle, when UMAP look at the answers for any given question, they check that these same answers seem reasonable. They also like the same level of detail for both (for example the answers should all be of similar length).

What about the stem’s themselves: again UMAP make some sensible judgements including avoidance of medical jargon, avoiding using the same words in the questions and the answers etc. They should be readable and comprehensible.

So is there any technique involved? Well yes! Look carefully at results an investigations. Its clear that UMAPs strategy will be not to spoon feed the reader. For example if an important feature is tachycardia the question may read as follows

Ther pulse rate was 124 – Rather than mentioning the tachycardia directly, or showing a picture of an ECG

The same goes for investigation reults: e.g. K+ 3.0 mmol/l (NR 3.5-5.5 mmol/l) rather than ‘hypokalaemia’

All this means is that you have to look carefully at the investigation answers, and then draw conclusions. The rest is guesswork? Not really. Simple mathematics state that you should rule out the maximum number of wrong answers, and maximise your chance of success.

Medical Educator or its wuestion writers have not contributed any questions to the UMAP quesiton bank and have no formal or informal association with the organisation. We’d be interested to hear students comments and opinions about the work done by UMAP.

Richard Marks, Consultant anaesthetist and representative of RemedyUK

A representative of RemedyUK, the key organisation behind the junior doctor movement speaks to Medical Educator. Dr Marks is a Consultant Anaesthetist, former training programme director, and spokesperson fro Remedy UK. Here he speaks to James Bateman, from Medical Educator.

Thanks for speaking to us Richard. You got involved with RemedyUK after you were disappointed with how the Government had handled the MTAS situation?

Yes, RemedyUK started in November 2006 and I joined it in January 2007. Of the key 6 people who started it, I was the last one in…..
When I came in I had spent the whole of 2006 trying to get our local training rotations to work under MMC [modernising medical careers], but I simply couldn’t make it work. The recruitment system looked like it was just going to be a big mess. I was feeling despondent, but then I went to a meeting of RemedyUK. They were planning a protest march, and I thought, if anyone’s going to sort it, then these guys will.

I think the march had a very large impact on doctors at the time. The vast majority of junior doctors in the UK were either aware of the march in 2007 about MTAS (Medical Training Application Service), or on it. Do you think the campaign has reached medical students?

Not really. We have medical students in the hospital that I work at. I don’t think that most of them are aware of what the problems are.

We tended to agree from our own experiences of contacts with students. We then asked Dr Marks to tell us a little about the current legal campaign. He highlighted the background to the MTAS enquiry following the march in 2007. He went on to describe the potential problems with dealing with a regulatory body (the GMC) and the existing organisation the BMA (British Medical Association) who had been perceived by many junior doctors to be less vocal in the defence of its members. He went on to say:

MTAS was technically just a computer system but it was the whole application process around the recruitment which was the disaster. The BMA had called for it to be stopped, but didn’t really do much more than that.

There was then some legal proceedings between thee two organisations which we will not cover in any more detail here. Dr Marks also commented:

The bad thing was that for the first time run-through training was being offered, which meant that the stakes for getting or not getting a job were higher than ever before.

Dr Marks makes a point here that resonates with colleagues who are junior doctors: the failure to get onto a training programme was almost seen as a “one shot” approach: failure meant that you were then destined to pursue a different speciality.

The process by which you would get or not get a job seemed to be less fair, and there were a lot of issues around the recruitment process. We thought it should never have been allowed to happen.

This is echoed by the grass roots support of RemedyUK by junior doctors. Want to see for yourself? Ask a doctor that you work or train with! Dr Marks then highlighted a series of reviews that cast a damning verdict on the MTAS process. Read more about them here, or listen to the podcast. The summary of the verdicts was as follows…

All the independent reviews said in various ways that was a complete disaster.

The role of the regulatory body for doctors in the UK, the GMC, was then discussed.

What we (RemedyUK) then thought was, why hasn’t the GMC taken a view on this? On two grounds…

Dr Marks went on to explain the rather poorly-defined concepts of professional misconduct and deficient professional performance.

At the end of last year we wrote to the GMC to say we thought they should hold an enquiry into this. There were 1600 signatures to a letter; but the  GMC wrote back and declined to hold an enquiry.

RemedyUK, acting on behalf of British Junior Doctors

As a result of this, and following legal advice taken by Remedy, the issue of the GMC’s role in the regulation of the management issues and application process is going to be scrutinised. The process is complicated but summarised on the Remedy Website as the following:

The pain, insensitivity and incompetence that MTAS created has not gone away. Neither have the people responsible for it, who continue to cling to their positions of power and responsibility. Last year we wrote on behalf of 1600 signators to the Fitness to Practice Directorate of the GMC, calling for an enquiry into whether those responsible were guilty of misconduct and/or deficient professional performance. The GMC refused our request, since it was apparently the wrong kind of misconduct.

We intend to challenge this decision. We intend to take the GMC to court, and seek judicial review to determine whether or not they have acted unlawfully in making this decision. Our lawyers believe there is strong case law and precedent which supports our view; we have already issued a Letter Before Action and are awaiting a reply.

Good lawyers are not cheap, and we aim to raise £20,000 to cover our legal costs. We are hoping to raise this from ordinary doctors – not only those directly affected by MTAS but from others who were appalled at the level of managerial incompetence, the lack of accountability of those in ivory towers and the failure of the GMC to grasp this nettle and show leadership. Should those entrusted with the highest responsibilities in medicine be immune from charges of deficient professional performance? Or are they as accountable as the rest of us in our own individual fields of practice? Do the concepts of accountability and revalidation apply to some but not to others?

We have mobilised our legal team and our fund-raisers. Many of our regular supporters have already contributed to our fighting fund, for which we are immensely grateful. But we have not yet raised enough. We are asking for a one-off donation of between £10 and £50 by you to make this happen. We also need help in bringing this campaign to the attention of consultants, GPs, retired practitioners and others who are not yet aware of what we are doing and who would be interested in helping.

We have just a few weeks to raise the money – time is of the essence.

But what about the medical Students: we asked Dr Marks about his perspectives on how students can get involved. His answers were as follows: awareness of the current situation can only empower medical students applying for jobs in the current market. Perhaps more worryingly, junior doctors may not be willing to come forward for fear of damaging their own career prospects. Dr Marks commented

RemedyUK publicises its campaing for further scruitiny of MTAS

We even get fairly regular emails saying, dear Remedy, can you do something about me because this something has happened……then they write back and say, actually, don’t because I don’t want to get into trouble, and I don’t want to make things any worse for myself.

Overall there are grey clouds and blue skies over the application processes for jobs. Advice for students? Dr Marks had the following advice for the current crop of specialist trainees: his words may seem frank, but reflect reality.

I say to them all- in 5 years time you are going to be competing against each other for consultant jobs – make sure that you’re better than the next person… We are moving into a competitive market.

Overall Medical Educator supports the goals of RemedyUK and we encourage all our readers to become informed about the organisation from  their website accessible here. You can also follow them on Facebook and Twitter. Donate to their legal challenge here.

To date, remedy Have raised £15 000 of the £20 000 needed for their legal challenge.

Listen to the remedy interview here