User:Lewiscatton/HILAS Questionnaires
Questionnaires
[edit]The electronic questionnaires should make the lives of participants as well as researchers easier. The web like interface that was used to present questionnaires in the current experiment is very easy to fill in for participants, and since the data is gathered electronically, this allows immediate distribution to other researchers in the consortium. In the current experiment two types of electronic questionnaires were used, questionnaires presented making use of Microsoft Office InfoPath, and questionnaires presented by Gwylio. Most questions focussed on the new technologies, acceptance of these technologies (Van Der Laan et al., 1997) and evaluation of usability.
RSME
[edit]Rating Scale Mental Effort (RSME) has been found to be a simple, reliable indicator of invested mental effort. It is a uni-dimensional scale on which the participants mark the vertical line (Zijlstra, 1993). In the current experiment, an electronic variant was used with a sliding bar. The RSME was intended to provide self-reported estimates of the pilot’s mental effort for use in the interpretation of the psychophysiological mental workload measures. Furthermore, the pilot ratings of RSME can be compared with workload ratings of an expert pilot who assessed the mental workload of the pilots.
CARS
[edit]The Crew Awareness Rating Scale (CARS) is a widely used situational awareness instrument developed by McGuinness & Foy (2000) at the BAE Systems Advanced Technology Centre. It may be used for self-assessment, rating the Situational Awareness (SA) of other team members and determining an objective “ground-truth” in a structured way. SA is the conscious representation of factors which are relevant to the achievement of operational goals in a complex situation. That is, information held to be represented in the mind. More specifically, it is the mental representation of one’s current task situation. It is “knowing what is going on so you can figure out what to do”. It is also “what you need to know so as to not be surprised”.
CARS identifies four common components of SA:
- Perception: detecting and recognising current facts and data;
- Comprehension: making appropriate interpretations of the facts and data;
- Projection: making realistic predictions of future developments;
- Appreciation: appreciating implications for goals, decisions and actions.
For each aspect of SA there are two rating scales, one addressing the perceived accuracy of the awareness / understanding achieved and the other addressing the perceived difficulty of the processing involved, which is an aspect of workload. The assessment of workload is considered to be a potentially important adjunct to that of SA. For instance, there could be situations in which a person has a high level of SA, but most of the attentional capacity is required to achieve that level of SA. This would leave little mental workspace left to allocate to other, perhaps equally critical processes, which would not be desirable.
Assessing the subjective difficulty associated with maintaining SA proves useful in evaluating the impact of new technologies on SA and mission performance. To enable this assessment, the questions used with the scale are framed so as to prompt the participant to consider whether or not they have “good” situational awareness or are finding it “easy” to maintain SA. For example, would you say that you have a good sense of the future course of events and what is likely to happen? The questions’ wordings sometimes required slight alteration to reflect unique aspects of the trial or special requirements of the researcher. Each question came with a rating scale, plus a “not applicable” response option.
Output
[edit]The output of the questionnaire program is a data file that is suitable for use in statistical programs such as SPSS, but which also can be imported into packages as MS Excel.
Depending on the type of questions included in the questionnaire, the output consists of text (e.g. for free text questions or for comments entered) or one or more values (e.g. rating on the RSME, rating scale value).
Methodology
[edit]The RSME was applied directly after each run. Pilots answered questions about the amount of effort they exerted during the run and scaled runs from 0 to 150, a score of 2 denoting “absolutely no effort”, a score of 58 “rather much effort”, and a score of 112 “extreme effort”.
Eight questions were asked as part of the CARS assessment at the end of each run.