UMAP, one organisation currently working to standardise medical student exam question content across medical schools

It’s essentially the organisation which is writing/ collecting a bank of medical student exam questions for use in ‘high stakes’ examinations, such as medical finals.

UMAP’s description of their activities can be found on their website: On the site they describe themselves as follows:

UMAP’s aim is to improve quality in high stakes written assessments across UK medical schools. UMAP runs best practice item writing sessions where clinical and academic staff come together to learn about item writing techniques and to represent their subject area in the developing question bank.

UMAP QA process

Once written, questions move on to be quality assured at UMAP question review meetings which are convened at our partner schools. Staff members with experience in assessment and who are familiar with UMAP style and technique check each question and amend as necessary to ensure the highest accuracy and conformity to question writing principles. Questions are then ready for use and are listed as part of selection documentation available to our partner medical schools.

Schools are invited to select the items they wish to use and then confirm their selections to us. Schools later return results data in an electronic format which is then analysed and uploaded into the UMAP bank. A summary of this data is displayed within question selection documentation to enable schools to make informed, evidence based, item choices.  [Accessed 4.03.09 from http://www.umap.org.uk/about/]

From their site they cover about 14 medical schools and they run an active recruitment of Specialist Registrars to write questions for their bank, most recently to our knowledge in Birmingham at a West Midlands General Internal Medicine training day.

UMAP currently publish on their site that as of October 2008 they have over 2500 questions for use in the these high stakes examinations. They acknowledge that they seem to be lacking in a few key areas, however they are currently working to address this.

What do you need to know about UMAP as a medical student? Well probably not that much. In fact all UMAP really are trying to do is generate sensible MCQ questions that are fair, and are well written. They have a reasonably complicated list of rules that a number of ME’s contributors have been talked through at a number of different times: overall though its not rocket science.

• you shouldn’t be able to answer a question just by using the investigation result or the stem on its own (e.g. a big intro and then showing an ECG with complete heart block etc.)
• The questions are aimed at core FY1 knowledge
• You should be able to guess the likely options for the answer (again straight forward)

In principle, when UMAP look at the answers for any given question, they check that these same answers seem reasonable. They also like the same level of detail for both (for example the answers should all be of similar length).

What about the stem’s themselves: again UMAP make some sensible judgements including avoidance of medical jargon, avoiding using the same words in the questions and the answers etc. They should be readable and comprehensible.

So is there any technique involved? Well yes! Look carefully at results an investigations. Its clear that UMAPs strategy will be not to spoon feed the reader. For example if an important feature is tachycardia the question may read as follows

Ther pulse rate was 124 – Rather than mentioning the tachycardia directly, or showing a picture of an ECG

The same goes for investigation reults: e.g. K+ 3.0 mmol/l (NR 3.5-5.5 mmol/l) rather than ‘hypokalaemia’

All this means is that you have to look carefully at the investigation answers, and then draw conclusions. The rest is guesswork? Not really. Simple mathematics state that you should rule out the maximum number of wrong answers, and maximise your chance of success.

Medical Educator or its wuestion writers have not contributed any questions to the UMAP quesiton bank and have no formal or informal association with the organisation. We’d be interested to hear students comments and opinions about the work done by UMAP.

The 7 key reforms listed on the site are as follows:

• “Require insurance companies to cover pre-existing conditions so all Americans regardless of their health status or history can get comprehensive benefits at fair and stable premiums.”
• “Create a new Small Business Health Tax Credit to help small businesses provide affordable health insurance to their employees.”
• “Lower costs for businesses by covering a portion of the catastrophic health costs they pay in return for lower premiums for employees.”
• “Prevent insurers from overcharging doctors for their malpractice insurance and invest in proven strategies to reduce preventable medical errors.”
• “Make employer contributions more fair by requiring large employers that do not offer coverage or make a meaningful contribution to the cost of quality health coverage for their employees to contribute a percentage of payroll toward the costs of their employees’ health care.”
• “Establish a National Health Insurance Exchange with a range of private insurance options as well as a new public plan based on benefits available to members of Congress that will allow individuals and small businesses to buy affordable health coverage.”
• “Ensure everyone who needs it will receive a tax credit for their premiums.”

Some of these concepts will be unfamiliar to our British and European students. How much will this cost the US taxpayer? $65 billion is just the start. It will be interesting to see if the medical training comes under closer scrutiny under the new regime.