Medical Training Application Service

The Medical Training Application Service (MTAS) was an on-line application system used in 2007 for the selection of Foundation House Officers and Specialty Registrars, and allocating them to jobs in the UK. It was set up under the auspices of Modernising Medical Careers. The operation of MTAS has been heavily criticised and key people involved have resigned. There have also been serious security concerns.

Note that the term "junior doctors", used below, is a slight misnomer; the term is used in the UK to categorise a wide range of doctors, from those who have just graduated with no experience to those with many years experience who have not yet made it to Consultant level.

Suggested problems with the old system

Prior to the introduction of Modernising Medical Careers (MMC), junior doctors who had completed their initial training after medical school (formerly the Pre-registration house officer or PRHO grade) could apply for posts as a Senior House Officer (SHO). They could apply for as many posts as they wished, and would be selected by their future employers based on their CV / application form and interview. They could devise their own training programme or apply to be part of a training rotation - usually entailing changing jobs every six months, but staying within the same specialty and hospital. Whilst training, an SHO would be encouraged to take professional exams to enable them to become a member of a Royal College. Once this was achieved, they could apply for jobs as a specialist registrar. The lack of formal structure between the PRHO grade and the specialist registrar grade led to SHOs being labelled as a "lost tribe".

With increasing numbers of junior doctors in the UK, competition was fierce. It was not uncommon for 250 applications to be made for one popular post. The process was also perceived to lack transparency.

MTAS application process

Under MTAS, junior doctors who have completed the PRHO grade, and more recently the Foundation House Officer grades, along with those who already had up to several years of experience working at the old SHO (next grade up) level, were invited to submit an electronic application form on the MTAS website. Applications could be made to one specialty in four geographic areas called 'Units of Application' (UoAs), or to two specialties in two UoAs, or four specialties in one UoA. There were twelve geographical areas: one each for Scotland, Wales and Northern Ireland; one covering the whole of London, Kent, Surrey and Sussex; eight others. The completed application forms were used for selection for interviews. Candidates who were not eligible (for example, not having practiced medicine for a number of years, or not being registered with the UK General Medical Council) were rejected at the 'longlisting' stage, with the 'shortlisting' stage designed to pick out the best applicants.

The application form consisted of shortlisting questions with space for 150-word answers. One of the changes promoted by the new system was that little (25%) importance was given to past experience, achievements, or examinations passed in the specialties, while the majority of the weighting for selection was based on the 150-word answers in the electronic application form (75% weighting). This was in sharp contrast to the previous CV-led process where past experience, achievements and examinations passed in the specialties were used to shortlist candidates for interview.

The interviews were again designed to be unbiased, in that, once selected for interview, the application form would be ignored, and CVs would not be allowed at interview. Instead, defined questions with explicit marking schemes gave scores to the candidate's answers by looking for certain phrases and keywords. Again, the emphasis here was on removing possible bias due to a candidate's past achievements and experience, and focusing only on the performance at the standardised interview.

After the first round of interviews, there was a plan for a second round of applications. Candidates who were unsuccessful in both rounds would have no further opportunity to gain access to a training job in the UK for the year ahead.

Application forms were released in February 2007, with two weeks to complete and submit. The first round of interviews were due to be carried out in February, March, and April 2007 with applicants finding out whether they were successful in May. The second round was to take place after this, finishing in late June 2007.

Suggested benefits of the new application system

The new application system centralised the application process, reducing the workload for consultants in shortlisting candidates for interview and candidates applying multiple times for different posts.

Unfortunately, the centralised system failed (MTAS was withdrawn after the matching process was completed). Unfortunately, the workload for consultants was massively increased and continues to increase with every change in the system.

Concerns raised over the new application system

Problems with the theory behind the process

Theoretical concerns behind the process include:

1. The MTAS system was based on five academic papers all produced by Fiona Patterson. The papers were based on a very small sample size and also state that the correlation between successfully completing the MTAS form and having the competencies required to be a successful doctor is 0.35 i.e. the correlation is low
2. It was decided to make a 'clean break' with the old system of Senior House Officer training by making all SHOs currently in training apply through the system. These doctors were between six months and six years into their training in a particular speciality. Many had higher qualifications such as membership of the Royal College of Physicians (MRCP) or had taken part in research in their chosen speciality. It was a common misconception amongst more senior colleagues that SHOs would essentially be reapplying for their old jobs. Given the centralised nature of the selection process, this was not the case. In fact, when a table was produced showing the number of jobs at each level, it could be seen that there were many fewer posts in the upper ranks of training than there were doctors already at that stage, which will leave many doctors partially trained and possibly unemployed.
3. Since geographical units of application were so large, candidates had to be prepared to work in a huge area of the country, or to compromise on their speciality since they were warned that making less than four choices would place them at a disadvantage. This was extremely hard for those who had family or financial ties.
4. With interview and selection was taking place centrally, rather than by the institutions or hospitals the junior doctors would be working for, more specialist institutions have no say in more particular skills and attributes needed to work in particular posts.
5. An application via MTAS was seen as an endorsement for Modernising Medical Careers, which is controversial in itself, leading to a moral dilemma for some over whether to apply, or risk their jobs.

Problems with the execution of the process

1. the system shortlisted candidates by taking the responses from one 150 word answer and asking the markers to just mark this question. Unfortunately, candidates were not informed that this would occur. Candidates had assumed the entire form would have been marked. Consequently, candidates that cross referenced their answers across the form (e.g., by writing "as detailed in my answer above") failed to score marks compared to candidates who did not do this.
2. The online application system frequently became overloaded which meant that potential applicants were unable to log into the site, complete or submit their forms. The application deadline was extended by 48 hours to enable candidates to submit their forms
3. Some forms were "lost" in the online system. Over 1,000 West Midland deanery application forms were "lost" for a week by the system. They were "found" and graded. However, concerns were raised that other forms may have been "lost" by the system.
4. The "double-blind" system for marking failed. Each form should have been marked independently by two consultants. This system ensures that a mistake by one marker is covered by the second marker. Unfortunately, the MTAS system allowed the second consultant who marked the application form to see the mark the first consultant had given. "Anchoring" theory states that if this happens, there is a significantly increased probability that the second marker will give a very similar score to the first marker i.e. the reason for introducing a double- blind marking system is nullified. Further, the second marker had the ability to change the mark given by the first marker.
5. Not all forms were marked by consultants. Many deaneries asked non-medical staff to help them meet their deadlines. It is unclear at the present time whether these non-medical staff were appropriately trained to mark the papers nor whether applicants were disadvantaged or advantaged if their forms were marked by non-medical staff.
6. Exceptional Candidates receiving no interviews. A large proportion of candidates who had already passed professional exams, had PhDs or research experience in the relevant specialty and many with considerable clinical experience were not shortlisted for interviews. This was largely due to academics and clinical achievements not carrying weight and the "150-word" answers to questions on the application form being designed to test psychological characteristics rather than clinical skills.

Security concerns

There were serious concerns over the security of the system after personal details (including phone numbers, home addresses and sexual orientation) of junior doctors became publicly available for several hours to anyone with the right URL. The following day it was reported that applicants had been able to see each other's files by changing two digits in the personalised web address given to each individual, and the system was suspended.

Implementers

According to information on Doctors.net.uk, the contract for the IT implementation of this system was given to a £6.3 million joint venture between Methods Consulting Ltd and jobsite.co.uk, a company run by three brothers: Keith, Graham and Eric Potts.

Attempts to repair interview process

The allegations of widespread execution problems led review and a rolling program of sweeping changes. This was a time of great uncertainty for many. The review stated that everyone should be entitled to keep any previous interviews in hand and also have an interview for their first choice if this was in England, with Wales and Scotland deciding to interview all applicants for all posts. Applicants for English programmes were able to make any of their four initial choices into their new first choice if they wished, in order to maximise the number of interviews they would have. This extra round of interviews was called Round 1B. Estimates are that there were 30,000 extra interviews requiring at least 15,000 consultant hours of interview time.

The initial offers from Rounds 1 & 1B were made by 10^th June 2007. Round 2 started on 22^nd June and was organised locally as the MTAS system was offline. Round 2 continued until 31^st October 2007, three months after the process should have been completed. The guidance from MMC indicated that any junior doctor in substantive employment on 31^st July, and taking part in Round 2 will be guaranteed employment until the end of Round 2 in October, "Please note that all applicants who applied to MTAS who are in substantive NHS employment on 31st July will continue to have employment while they progress through Round 2".

There was, however, a general feeling amongst those involved in the system that it was beyond repair and further efforts to do so constituted an example of 'Irrational escalation'.

Consequent Events

The British Medical Association (the largest trade union body representing doctors) pulled out of the review panel after their latest announcement that each doctor would only be given one interview. Organisations like the Royal College of Surgeons have lost confidence in the system .

The BMA later rejoined the review and have since lent their support to the proposed changes ("Round 1B"). However, following an outcry by members and a controversial letter to The Times newspaper, Mr James Johnson, Chairman of the BMA resigned on 20^th May, 2007, citing his reasons for leaving as the contents of the letter .

On 30 March 2007 Professor Alan Crockard resigned as National Director of Modernising Medical Careers. In his resignation letter he stated that he has "responsibility but less and less authority" and that "the overriding message coming back from the profession is that it has lost confidence in the current recruitment system". It is as yet unclear what effect this resignation will have on the review process.

On 3 April 2007, during an interview on BBC Radio 4's Today programme, the Health Secretary Patricia Hewitt apologised to junior doctors over the crisis, saying that the application scheme had caused "needless anxiety and distress". The British Medical Association welcomed the government's acknowledgement of the problem but stated that "an apology isn't enough".Patricia Hewitt's apology was repeated to parliament on 16th April 2007, however she stated that, "the problems that have arisen relate in the main to the implementation process and not to the underlying principles of Modernising Medical Careers." Later that month Ms Hewitt was accused by the opposition of failing to express genuine regret over the fiasco.

The MTAS website was suspended on April 26 2007 after a Channel 4 News report stated that applicants had been able to see each other's files by changing two digits in the personalised web address given to each individual. At that point the Department of Health announced that this was a temporary suspension.

Ms Hewitt made another apology on 1 May 2007 in the House of Commons after the suspension of the MTAS website due to security breaches that she described as "utterly deplorable". . On 3 May, Hewitt appeared on BBC Question Time where she faced hostile questioning from a junior doctor over MTAS failures.

On May 15 2007 MTAS was shelved by ministers due to security breaches. Patricia Hewitt said that after the first round of recruitment, the system would only fulfil a monitoring role.

A judicial review of MTAS was requested by pressure group Remedy UK, who demanded that the appointments process should be scrapped altogether. The case was heard in the High Court from May 16-17 2007. On Wednesday May 23 2007 Mr Justice Goldring ruled against Remedy UK, stating that "although far from ideal", the Review Group's decision on amending the appointments process after the first round of interviews was "within the range of reasonable responses", and that the Review Group was "entitled to reach the decision it did given the circumstances facing it at the time". Although he ruled against invalidating the interviews that had already taken place, Mr Justice Goldring added that this judgement did not imply that junior doctors were not entitled to feel aggrieved, as "the premature introduction of MTAS has had disastrous consequences". Remedy UK have said they will not appeal the verdict, in order to avoid further uncertainty for junior doctors.