Introduction

Together with the tremendous growth of civil aviation in the last decades, the importance of aviation security has dramatically increased. Despite of large technological progress in X-ray imaging such as high-resolution image quality and image enhancement features, the final decision whether a luggage is allowed to enter an aircraft or not is always made by a human operator. Therefore, human factors are still the essential key element in aviation security worldwide. Even if no technological equipment will replace human operators from the X-ray screening task in the near future, EU audits and covert tests have sometimes shown serious operational deficiencies at the checkpoints. This document concentrates on how aviation security can be largely improved by selecting the right people for X-ray screening tasks. Selection, or Pre-Employment Assessment (PEA), is one of the three main pillars of our integrative approach addressing the occurrence of operational deficiencies at the security checkpoints. Please refer to the documentations covering training and certification, the other two main pillars of our integrative quality management approach.

Why Selection is of Great Importance in Aviation Security Officers Recruitment

What our human visual system is capable of to perform in every-day visual information processing without any noticeable effort is hardly adequately appreciated. Human vision shows very reliable object recognition performance in every-day life. Under common circumstances, as for example under normal illumination, we hardly ever mistake known objects for something else.

In X-ray image interpretation the situation is actually quite different. Human X-ray image interpretation competency is very far from our every-day vision, where we constantly improve visual performance by interacting with our environment getting permanent feedback. X-ray images dispose of a different colour range, single objects superimpose each other independent of whether they are in front or behind each other, and the single objects look very different from what they look like under natural light.

Therefore, individual visual abilities such as mental rotation, figure-ground segregation, or visual search of specific patterns become much more essential in correctly interpreting X-ray images than in every-day life vision. Scientific literature could show that these visual abilities are relatively stable over time and that there are considerable differences among individuals (Schwaninger, Hardmeier & Hofer, 2005; Hofer & Schwaninger, 2005; Hardmeier, Hofer & Schwaninger, 2005). Visual abilities are individual properties that can not be easily changed by training. Training is considered to increase visual knowledge of which items are prohibited and what they look like in X-ray images. In contrast, visual abilities, like the ability to cope with superpositoin or bag complexity can be improved by training only to a limited extent. Therefore, it is valuable to use a scientifically based pre-employment assessment procedure to select screener candidates before employing them based on visual abilities. Training alone is not sufficient to compensate deficiencies regarding the basic required visual abilities.

How To Conduct a Reliable and Valid Pre-Employment Assessment

The Visual Cognition Research Group (VICOREG) at the University of Zurich has developed a series of X-ray image interpretation tests designed to assess the most important aspects of abilities and knowledge required in X-ray security screening. Some of these, like the X-Ray Competency Assessment Test for Cabin Baggage Screening (X-Ray CAT CBS), contain a large variety of prohibited items assessing visual knowledge of how the large amount of prohibited items look like. Other tests, like the Competency Assessment Test for Hold Baggage Screening (X-Ray CAT HBS), are especially designed to assess whether screeners can identify improvised explosive devices (IEDs).

The X-Ray Object Recognition Test (X-Ray ORT)

To specifically assess visual abilities required in X-ray image interpretation VICOREG developed the X-Ray Object Recognition Test (X-Ray ORT). This test measures reliably visual abilities that are relatively independent of visual knowledge acquired through training and experience. It only contains guns and knives as prohibited objects whose shapes are expected to be known from every-day life, or at least from every-day multimedia entertainment. Knowledge of how more specific prohibited items, like IEDs or tasers (electric shock devices), look like in X-ray images is not required to successfully solve this test. Furthermore, the images in the X-Ray ORT are displayed in grayscale to factor out the knowledge of how different materials are colour-coded in X-ray images. The most obvious knowledge-based factors threat type and colour-coding are eliminated from the test design.

In the X-Ray ORT three specific image-based factors closely related to visual abilities are systematically manipulated: view difficulty, superposition and bag complexity. These image-based factors are known to highly influence threat detection performance. So do the related visual abilities from a human factors perspective. View difficulty is closely related to the visual ability to mentally rotate suspicious objects. Superposition is closely related to the ability to differentiate between the shape of different physical objects in an X-ray image (figure-ground segregation) and bag complexity is related to the ability to visually search for specific patterns in complex backgrounds.

Figure 1: Image based factors relevant in X-ray screening: a) Bag Complexity, b) Superposition, c) View Difficulty.

The X-Ray ORT consists of 256 X-ray images of passenger bags that must be judged for whether they are OK or NOT OK to hypothetically enter an aircraft. One half of all images contain a threat item, those of the other half do not. The 128 threat trials of the test are given as follows: 8 guns and 8 knives are being projected into bags with low and high bag complexity (BC) with low and high superposition (SP) and in an easy and in a difficult view (view difficulty VD) each. Thus, all 16 threat exemplars are shown in all 8 factor combinations (2 VD x 2 SP x 2 BC; see Figure 1). In order to take into account the pressure of time screeners have to withstand on their job, the maximum display duration of the X-ray images in X-Ray ORT is limited to 4 seconds.

Reliability and Validity of the X-Ray ORT

The X-Ray ORT shows a very high reliability of Cronbach´s alpha values that range from .887 to .966 for professional screeners and from .907 to .970 for novices. Split-half reliability values of the X-Ray ORT range from .781 to .904 for professionals and from .778 to .939 for novices.

Large effects of bag complexity, superposition and viewpoint could be shown for aviation security screeners and novices and support high internal validity. Furthermore, convergent and discriminant validity could be shown based on all 453 screeners selected with the old selection procedure, correlating results in the X-Ray ORT with results in the PIT (r = .61, p < .001) and results in the computer-based questionnaire (CBQ) (r = .27, p < .001), respectively.

For more detailed information on validity and reliability please refer to Hardmeier, Hofer and Schwaninger (2005).

More Tests To Be Considered in Pre-Employment Assessment

The Ishihara Colour Vision Charts

Since the X-Ray Object Recognition Test does explicitly not assess colour vision, but colour perception is very important for interpreting X-ray images of state-of-the-art X-ray machines (colour-coding etc.) on the job, colour perception must be assessed separately in the pre-employment assessment. We recommend standard colour vision tests, such as the well-established Ishihara Colour Vision Test. Ishihara Colour Vision Charts are available with different set sizes of 10-35 charts. The different set sizes are related to the different types of colourblindness. The most widespread colour deficiency is the red-green indiscriminability, and can be diagnosed by the minimum set size. Colour vision tests are very easy to conduct in a short time. Therefore, we recommend using these tests at the beginning of each pre-employment assessment. Using computer-based tests for testing color vision can be problematic. Computer monitors display color information in different ways which can result in changed test scores. Therefore, we recommend to use original color plates (e.g. the Ishihara Color Vision Test).

Visual Acuity Test

As a matter of course also the visual acuity of future screeners must be normal or corrected-to-normal. Please contact professional opticians for further information.

Cognitive Test Batteries

In order to assess the visual abilities more abstractly and also to broaden the focus of cognitive testing beyond the scope of vision, we additionally recommend the use of appropriate cognitive tests in the pre-employment assessment. The Visual Cognition Research Group (VICOREG) at the University of Zurich reassembled a series of well-established commercial tests to assess figure-ground segregation, visual search, mental rotation, and spatial imagination as well as logical thinking, attentiveness, concentrativeness, and retentiveness (Hardmeier & Schwaninger, 2008). Cognitive tests allow getting a general idea of the strengths and weaknesses regarding cognitive abilities of a job applicant in an abstract way.

Pre-Employment Assessment Aside From the Vision Context

The field of activity of aviation security officers is not limited to X-ray image interpretation. Although this document concentrates on the visual cognition aspect, it is important to note that other human characteristics are worth being assessed as well in the pre-employment assessment. X-ray image inspection can be a fatiguing job when performed more than 20 minutes without breaks. Fortunately, aviation security screeners have at least two additional tasks at most security check-points. Examples are instructing passengers and placing bags on the X-ray belt, conducting manual search of luggage or conduct an additional test using a hand held metal detector. All these tasks ask for specific attributes. Aviation security officers must have substantial social and communication skills especially when dealing with certain passengers. Furthermore, they must have a certain constitution since they work about and must lift luggage and search it. Finally, the employer is interested in employing people with high degrees of conscientiousness, integrity, discretion, secrecy, work ethic, and service orientation. All these aspects should be taken in account when selecting new personnel for becoming an aviation security screener.

Scientific Findings Showing the Benefits of Using X-Ray ORT as a Selection Tool in Pre-Employment Assessment

The Effects of X-Ray ORT Use in Pre-Employment Assessment on Threat Detection Performance One Year Later

To test whether the X-Ray ORT is a useful pre-employment assessment tool, detection performance of screeners selected without the X-Ray ORT and screeners who were selected after having passed the X-Ray ORT was compared using the test results in the X-Ray CAT CBS. All screeners who were hired with the X-Ray ORT had completed a classroom training and about one year of working experience when taking the X-Ray CAT CBS. All test results were calculated using the "nonparametric" detection performance measure A´ (see Pollack & Norman, 1964; Grier, 1971; Pastore, 2003). A´ takes into account the hit rate (i.e. bags containing a prohibited item judged as NOT OK) as well as the false alarm rate (i.e. harmless bags judged as NOT OK).

Figure 2: Detection performance with standard deviations in the X-Ray CAT CBS for screeners selected without the X-Ray ORT (control group) on the left and screeners selected with the X-Ray ORT (experimental group) on the right.

In order to investigate whether the X-Ray ORT is a valuable tool for pre-employment assessment, the mean detection performance of both groups in the X-Ray CAT CBS was compared (see Figure 2). A significant difference in detection performance of prohibited items between screeners selected without the X-Ray ORT and the group hired with the X-Ray ORT can be shown. The job applicants who were selected with the X-Ray ORT are significantly better in detecting prohibited items in X-ray images, t(552) = 14.51, p < .001 one year after employment.

The Effects of the Image Based Factor as Implemented in the X-Ray ORT on Threat Detection Performance

Figure 3: Effects of image-based factors in the X-Ray ORT; error bars represent standard deviations: TOP: aviation security screeners, BOTTOM: novices.

Large main effects of image-based factors (bag complexity, superposition and view difficulty) were found both for professionals and novices. The large variances between individuals show that people differ remarkably with regard to how well they can cope with image difficulty resulting from these image-based factors. Interestingly, only small mean differences in A´ between professionals and novices were found. This is consistent with the assumption that the X-Ray ORT measures relatively stable visual abilities that are needed to cope with effects of bag complexity, superposition and view difficulty.

Individual A´ scores were subjected to a three-way analysis of variance (ANOVA) with bag complexity, superposition and view difficulty as within-participants factors. This analysis was done for both groups of participants (professionals and novices) separately. The main effects are displayed in Figure 3. ANOVA results of professional aviation security screeners showed highly significant main effects of Bag Complexity, Superposition and View. Similar results were observed for novices. There were again highly significant main effects with large effect sizes.

Further Scientific Literature

Take Home Message

Order Pre-Employment Assessment Tools for Free

If you are interested in implementing the VIA Project Quality Management Tools in your country/ airport please go to the Ordering Form.

Acknowledgements

European Commission Leonardo da Vinci Programme (VIA Project, DE/06/C/F/TH-80403, www.viaproject.eu) Belgian Civil Aviation Administration; Canadian Air Transport Authority (CATSA); Federal Office of Civil Aviation, Switzerland; German Ministry of the Interior; Ministry of Justice, The Netherlands; Amsterdam Airport Schiphol, Falck G4S, & ICTS; Berlin Airports Schönefeld and Tegel & Securitas; Brussels Airport & Securitas; Oslo Airport & Adecco; Zurich Airport & Zurich State Police, Airport Division.