Increasing Aviation Security X-Ray Image Interpretation Competency with Computer-Based Training

Introduction

X-ray technologies provide us with the inside view of a luggage which allows to see the content of a bag without opening it. Drastic growth in such technologies has been made in the last years. State-of-the-art X-ray machines provide automatic explosive detection algorithms, although still they have a relatively high false alarm rate. Hence the inspection task can of course not be solved by the machine itself; the last decision is always made by the human operator. Focusing only on image technology without really taking into account how the human brain processes visual information cannot answer the question how the best performance in threat detection can be achieved. The decision is determined by the level of human performance which needs to take into account perceptual and cognitive characteristics. The goal is to achieve a reliable, consistent, and effective performance which can only be achieved when screeners are well trained and familiarized with different kinds of threat items. To sum up, the most expensive equipment is of limited value if a screener who operates it is not trained appropriately.

Relevance of Training for Aviation Security Screeners

A screener needs to detect a threat item within a few seconds when visually inspecting an X-ray image of a passenger bag. This can be a challenge when the shape of a threat object is not similar to previously encountered objects. Please note also that some threat objects look completely different in X-ray images than in reality. Nevertheless, the perceptual learning capability an individual has is enormous when using the appropriate training. To train people in recognizing threat items, an individual adaptive training system helps to learn a large amount of threat objects in a short time, when the learning is adapted to the individual performance of each screener. Studies on how the human brain processes visual information to recognize objects can help to develop such an adaptive training system in the field of airport security. Although a training system should be very easy to use, but what is going on in the background of such a system should be scientifically well elaborated.

Threat Image Projection (TIP) is a software function of state-of-the-art x-ray machines. With TIP it is possible to project pre-recorded Fictional Threat Items (FTIs) into X-ray images of passenger bags during the routine baggage screening operation. TIP is a valuable tool to increase attention and motivation of screeners and it has some training value. Since with TIP screeners see only a few FTIs per day it is however not a very effective training tool. Note that when a small TIP library of a few hundred images is used, people can learn the FTIs quickly and substantial performance improvements in TIP data can be found. With a small image library there is however a risk that screeners focus on identifying the FTIs of the TIP library while their attention and imagination needed for identifying real threats might vanish. In order to gain from the main benefits of TIP (increased attention and motivation), it is essential to use a large FTI image library. In order to provide effective and efficient training it is recommended to use Computer Based Training (CBT).

Individually Adaptive Difficulty Levels

Figure 1: Image-based factors challenging the detection of threat objects in X-ray images.

Schwaninger (2003b) and Hardmeier, Hofer, and Schwaninger (2005) have identified three image-based factors that influence X-ray image interpretation (see Figure 1): It is harder to detect an object in a rotated view compared to the upright view (see Figure 1a, effect of viewpoint). The superposition of an object by others can impair the detection performance as well (see Figure 1b, effect of superposition). A luggage containing different objects and the type and number of these objects can distract visual attention and therefore also impair the detection performance (see Figure 1c, effect of bag complexity). These image-based factors (view, superposition, and bag complexity) should be taken into account when people get individually adaptive training. In other words, an individually adaptive training system should increase the difficulty regarding these image-based factors. For example, people should first be trained with objects in easy rotations.

Figure 2: Detection performance with standard deviations for the individually adaptive training group vs. the control group (not individually adaptive training) comparing first and second measurement of a standardized test.

Afterwards more difficult views of threat objects should be displayed. In a higher level threat objects should be more superimposed by other objects. Finally the bag complexity should be increased, individually adapted to the performance and to the difficulties a screener has to cope with these image-based factors, respectively. A study by Koller, Hardmeier, Michel, and Schwaninger (2007) could show that there is a very high increase of detection performance with an individually adaptive training system using a large image library compared to a training system which is not individually adaptive and has a small image library (see Figure 2). Michel, de Ruiter, Hogervorst, Koller, Moerland and Schwaninger (2007) found that screener increase their detection performances when using individually adaptive CBT. In addition Schwaninger, Hofer and Wetter (2007) found that computer-based training results in high detection performances at the security checkpoint, measured with an increase of the detection performances at the TIP system after CBT-training.

Image Library

What we see in real world is the result of visual processing in our brain. If a certain type of forbidden objects has never been seen before, no representation of it in visual memory could be formed and the object becomes difficult to recognize unless it is similar to stored views of another object. Therefore, different threat objects from different threat categories (e.g., guns, knives, IEDs and other) should be integrated in an adaptive training system to make sure that a screener will be able to detect a large number of different threat objects. It is essential that a large library of threat objects is implemented in an individually adaptive training system. Additionally, it is not necessary that harmless objects are integrated into the training image library because screeners are confronted with harmless objects all the time during work. Very important is not only that the image library is large, it is also important that the threat objects are displayed from different viewpoints. Additionally, fictional threat items (FTIs) should be displayed on a new position in the bag each time to avoid that people learn the FTI-bag combinations. The image library used for training can of course be adapted to the work environment. For example, the image library for a training system for passenger bags control is different from staff bags control. Of course for cabin baggage screening (CBS) another image library is used than for hold baggage screening (HBS) because for HBS IEDs are the most dangerous threat objects whereas missing a knife in a hold baggage is less dangerous. But for CBS all four categories of threat items (guns, knives, IEDs and other) should be integrated into the training system. The training library should therefore be optimally selected depending on the job requirements and must constantly being updated with new kinds of threat objects.

Feedback

A very important aspect of training is that screeners get feedback after each image. At least, feedback should be given if a threat was missed or detected (hit) or if there was a false alarm (harmless bag judged as NOT OK) or a correct rejection (harmless bag correctly judged as OK). A more improved feedback shows which object was missed and how it looks like as an X-ray image and under normal light conditions. Only then a representation of that threat object is built in the brain and makes it possible to recognize it in the future.

Monitoring Training Progress

Normally screeners can use the training system on their own without any supervision. To make sure that screeners really train, the training progress should be able to be monitored by training instructors or supervisors. For example the amount of training hours, the number of images that have been seen during training, and finally the training progress (e.g., difficulty levels) are important indications about training progress. For an overview about the usefulness of IEFs see Michel, Koller, Ruh and Schwaninger (2007).

Training Duration

Finally, a training session should not be too long. We recommend no more than three times 20 minutes per day at least 20 minutes training per week for an optimal training progress. Very important is that screeners conduct recurrent training in order to keep screeners up-to-date regarding new threats and in order to maintain their x-ray image interpretation competency.

How To Conduct Training

During training the screeners? task should then be to judge each bag if it is OK (harmless bag) or NOT OK (containing a threat item). Additionally, to reduce guessing, threat items can be identified by marking them in the bag with, for example, clicking on them using a computer mouse. All these answers given by a screener will then be stored and resultant, the next training session will be provided individually adapted to the performance of a screener?s previous training history. It is important that not only hits are measured and stored because the hit rate alone does not tell much. The hit rate refers to the proportion of all images containing a prohibited item that have been judged as NOT OK. Imagine that someone always gives the answer "NOT OK", this person reaches 100% hit rate but also 100% false alarm rate. The false alarm rate refers to the proportion of NOT OK judgments for harmless bags. Therefore, both measurements (hits and false alarms) should be taken into account. High false alarm rates result in long waiting queues at the security checkpoint because every baggage has to be controlled manually. The signal detection theory provides methods in which both (hits and false alarms) are considered. For example, a very good screener reaches a high hit rate with a low false alarm rate which results in a high d? or A' value. For more information on signal detection theory see Hofer and Schwaninger (2004). In addition, the response time (how long does it take to give an answer) is also a good indicator for the efficiency of a screener. During training, normally the response time is decreasing, especially for bags containing a threat object.

A training session can look like this:
A screener is sitting in front of a training computer and logs in with a user ID and password. Then the images for this training session are prepared, considering the previous performances of that person. The task a screener has to accomplish is to answer if a bag is OK or NOT OK. Important is that appropriate feedback, if the answer was correct or not, is given after each image. A more sophisticated feedback should also provide information about how the threat objects looks like in X-ray and in reality. After finishing a training session, a session feedback and also an overall feedback about the performance in all past training sessions should be given to the screener. This is important to increase the motivation for further using the training system.

Summary

In summary, training is one of the most important aspects in aviation security help obtaining reliable and effective recognition of threat objects in passenger bags. Without training it can be difficult to recognize threat items if they are not similar to objects that have been encountered previously. Important is that even if screeners are well trained, a reliable detection of threat items at the security checkpoint cannot be guaranteed when attention and motivation of screeners are insufficient. Therefore TIP can help to increase the attention and motivation of screeners on the job. A training system should be individually adaptive to train screeners optimally based on their performances. A large image library should be used and threat objects should be presented in different difficulties like easy and difficulty rotations, low and high superpositions, and low and high bag complexities. Very important is that feedback is given after each image about the correctness of the answer. In addition, performance feedback after each training session and also over all training sessions is an important motivation for users. Finally, recurrent training with an appropriate training duration (e.g., 1-2x 20 minutes per week) is important to maintain a high x-ray image interpretation competency.

Take Home Message

Even with the most expensive technology the human operator is the weakest link in the aviation security unless a screener is well trained with a computer-based training system.

Training is essential to guarantee a consistent, reliable, and effective recognition of threat objects.

The image-based factors view dificulty, superposition and bag complexity should be considered and therefore different difficulty levels should be integrated into an effective training system.

The image library should contain a large variety of threat items.

Different views of threat objects should be presented.

Harmless objects are not necessary to be integrated into the training image library because screeners encounter harmless objects during the routine x-ray screening operation all the time.

One of the most important aspects of a good training system is that feedback is provided after each image about the correctness of the answer. More sophisticated, the missed threat is shown in the feedback.

After each training session, feedback about the performance should be given.

It should be possible that training instructors or supervisors have the possibility to monitor the training progress of the screeners.

Finally, training should be recurrent to achieve reliable, effective and efficient recognition of threat objects.

References

Green, D. M., & Swets, J. A. (1966). Signal Detection Theory and Psychophysics. New York: Wiley.

Koller, S.M., Hardmeier, D., Michel, S., & Schwaninger, A. (2007). Investigating training and transfer effects resulting from recurrent CBT of x-ray image interpretation. In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th Annual Cognitive Science Society (pp. 1181-1186). Austin, TX: Cognitive Science Society.

Hofer F., & Schwaninger, A. (2005). Using threat image projection data for assessing individual screener performance. WIT Transactions on the Built Environment, 82, 417-426.

Michel, S., de Ruiter, J. C., Hogervorst, Koller, S.M., M., Moerland, R., & Schwaninger, A. (2007). Computer-based training increases efficiency in x-ray image interpretation by aviation security screeners. Proceedings of the 41st Carnahan Conference on Security Technology, Ottawa, October 8?11, 2007.

Schwaninger, A., Hofer, F., & Wetter, O.E. (2007). Adaptive computer-based training increases on the job performance of x-ray screeners. Proceedings of the 41st Carnahan Conference on Security Technology, Ottawa, October 8?11, 2007.

Michel, S., Koller, S.M., Ruh, M., & Schwaninger, A. (2007). Do "image enhancement" functions really enhance x-ray image interpretation? In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th Annual Cognitive Science Society (pp. 1301?1306). Austin, TX: Cognitive Science Society.

Schwaninger, A. (2003b). Evaluation and selection of airport security screeners. AIRPORT, 02/2003, 14?15.