Increasing Aviation Security X-Ray Image Interpretation Competency with Computer-Based Training
Introduction
X-ray technologies provide us with the inside view of a luggage
which allows to see the content of a bag without opening it. Drastic
growth in such technologies has been made in the last years.
State-of-the-art X-ray machines provide automatic explosive detection
algorithms, although still they have a relatively high false alarm rate.
Hence the inspection task can of course not be solved by the machine
itself; the last decision is always made by the human operator.
Focusing only on image technology without really taking into account
how the human brain processes visual information cannot answer the
question how the best performance in threat detection can be achieved.
The decision is determined by the level of human performance which
needs to take into account perceptual and cognitive characteristics.
The goal is to achieve a reliable, consistent, and effective
performance which can only be achieved when screeners are well trained
and familiarized with different kinds of threat items. To sum up, the
most expensive equipment is of limited value if a screener who operates
it is not trained appropriately.
Relevance of Training for Aviation Security Screeners
A screener needs to detect a threat item within a few seconds when
visually inspecting an X-ray image of a passenger bag. This can be a
challenge when the shape of a threat object is not similar to
previously encountered objects. Please note also that some threat
objects look completely different in X-ray images than in reality.
Nevertheless, the perceptual learning capability an individual has is
enormous when using the appropriate training. To train people in
recognizing threat items, an individual adaptive training system helps
to learn a large amount of threat objects in a short time, when the
learning is adapted to the individual performance of each screener.
Studies on how the human brain processes visual information to
recognize objects can help to develop such an adaptive training system
in the field of airport security. Although a training system should be
very easy to use, but what is going on in the background of such a
system should be scientifically well elaborated.
Threat Image Projection (TIP) is a software function of
state-of-the-art x-ray machines. With TIP it is possible to project
pre-recorded Fictional Threat Items (FTIs) into X-ray images of
passenger bags during the routine baggage screening operation. TIP is a
valuable tool to increase attention and motivation of screeners and it has some training value. Since
with TIP screeners see only a few FTIs per day it is however not a very
effective training tool. Note that when a small TIP library of a few
hundred images is used, people can learn the FTIs quickly and
substantial performance improvements in TIP data can be found. With a
small image library there is however a risk that screeners focus on identifying
the FTIs of the TIP library while their attention and imagination
needed for identifying real threats might vanish. In order to gain from
the main benefits of TIP (increased attention and motivation), it is
essential to use a large FTI image library. In order to provide
effective and efficient training it is recommended to use Computer Based Training (CBT).
Individually Adaptive Difficulty Levels
Figure 1: Image-based factors challenging the detection of threat objects in X-ray images.
Schwaninger (2003b) and Hardmeier, Hofer, and Schwaninger (2005)
have identified three image-based factors that influence X-ray image
interpretation (see Figure 1): It is harder to detect an object in a
rotated view compared to the upright view (see Figure 1a, effect of
viewpoint). The superposition of an object by others can impair the
detection performance as well (see Figure 1b, effect of superposition).
A luggage containing different objects and the type and number of these
objects can distract visual attention and therefore also impair the
detection performance (see Figure 1c, effect of bag complexity). These
image-based factors (view, superposition, and bag complexity) should be
taken into account when people get individually adaptive training. In
other words, an individually adaptive training system should increase
the difficulty regarding these image-based factors. For example, people
should first be trained with objects in easy rotations.
Figure 2: Detection performance with standard deviations for the
individually adaptive training group vs. the control group (not
individually adaptive training) comparing first and second measurement
of a standardized test.
Afterwards more difficult views of threat objects should be displayed.
In a higher level threat objects should be more superimposed by other
objects. Finally the bag complexity should be increased, individually
adapted to the performance and to the difficulties a screener has to
cope with these image-based factors, respectively. A study by Koller,
Hardmeier, Michel, and Schwaninger (2007) could show that there is a
very high increase of detection performance with an individually
adaptive training system using a large image library compared to a training system which is not
individually adaptive and has a small image library (see Figure 2). Michel, de Ruiter, Hogervorst,
Koller, Moerland and Schwaninger (2007) found that screener increase
their detection performances when using individually adaptive CBT. In
addition Schwaninger, Hofer and Wetter (2007) found that computer-based
training results in high detection performances at the security
checkpoint, measured with an increase of the detection performances at
the TIP system after CBT-training.
Image Library
What we see in real world is the result of visual processing in our
brain. If a certain type of forbidden objects has never been seen
before, no representation of it in visual memory could be formed and
the object becomes difficult to recognize unless it is similar to
stored views of another object. Therefore, different threat objects
from different threat categories (e.g., guns, knives, IEDs and other)
should be integrated in an adaptive training system to make sure that a
screener will be able to detect a large number of different threat
objects. It is essential that a large library of threat objects is
implemented in an individually adaptive training system. Additionally,
it is not necessary that harmless objects are integrated into the
training image library because screeners are confronted with harmless
objects all the time during work. Very important is not only that the
image library is large, it is also important that the threat objects
are displayed from different viewpoints. Additionally, fictional threat
items (FTIs) should be displayed on a new position in the bag each time
to avoid that people learn the FTI-bag combinations. The image library
used for training can of course be adapted to the work environment. For
example, the image library for a training system for passenger bags
control is different from staff bags control. Of course for cabin
baggage screening (CBS) another image library is used than for hold
baggage screening (HBS) because for HBS IEDs are the most dangerous
threat objects whereas missing a knife in a hold baggage is less
dangerous. But for CBS all four categories of threat items (guns,
knives, IEDs and other) should be integrated into the training system.
The training library should therefore be optimally selected depending
on the job requirements and must constantly being updated with new
kinds of threat objects.
Feedback
A very important aspect of training is that screeners get feedback
after each image. At least, feedback should be given if a threat was
missed or detected (hit) or if there was a false alarm (harmless bag
judged as NOT OK) or a correct rejection (harmless bag correctly judged
as OK). A more improved feedback shows which object was missed and how
it looks like as an X-ray image and under normal light conditions. Only
then a representation of that threat object is built in the brain and
makes it possible to recognize it in the future.
Monitoring Training Progress
Normally screeners can use the training system on their own without
any supervision. To make sure that screeners really train, the training
progress should be able to be monitored by training instructors or
supervisors. For example the amount of training hours, the number of
images that have been seen during training, and finally the training
progress (e.g., difficulty levels) are important indications about
training progress. For an overview about the usefulness of IEFs see
Michel, Koller, Ruh and Schwaninger (2007).
Training Duration
Finally, a training session should not be too long. We recommend no
more than three times 20 minutes per day at least 20 minutes training
per week for an optimal training progress. Very important is that
screeners conduct recurrent training in order to keep screeners
up-to-date regarding new threats and in order to maintain their x-ray
image interpretation competency.
How To Conduct Training
During training the screeners? task should then be to judge each bag
if it is OK (harmless bag) or NOT OK (containing a threat item).
Additionally, to reduce guessing, threat items can be identified by
marking them in the bag with, for example, clicking on them using a
computer mouse. All these answers given by a screener will then be
stored and resultant, the next training session will be provided
individually adapted to the performance of a screener?s previous
training history. It is important that not only hits are measured and
stored because the hit rate alone does not tell much. The hit rate
refers to the proportion of all images containing a prohibited item
that have been judged as NOT OK. Imagine that someone always gives the
answer "NOT OK", this person reaches 100% hit rate but also 100% false
alarm rate. The false alarm rate refers to the proportion of NOT OK
judgments for harmless bags. Therefore, both measurements (hits and
false alarms) should be taken into account. High false alarm rates
result in long waiting queues at the security checkpoint because every
baggage has to be controlled manually. The signal detection theory
provides methods in which both (hits and false alarms) are considered.
For example, a very good screener reaches a high hit rate with a low
false alarm rate which results in a high d? or A' value. For more information on signal detection theory
see Hofer and Schwaninger (2004). In addition, the response time (how
long does it take to give an answer) is also a good indicator for the
efficiency of a screener. During training, normally the response time
is decreasing, especially for bags containing a threat object.
A training session can look like this:
A screener is sitting in front of a training computer and logs in with
a user ID and password. Then the images for this training session are
prepared, considering the previous performances of that person. The
task a screener has to accomplish is to answer if a bag is OK or NOT
OK. Important is that appropriate feedback, if the answer was correct
or not, is given after each image. A more sophisticated feedback should
also provide information about how the threat objects looks like in
X-ray and in reality. After finishing a training session, a session
feedback and also an overall feedback about the performance in all past
training sessions should be given to the screener. This is important to
increase the motivation for further using the training system.
Summary
In summary, training is one of the most important aspects in
aviation security help obtaining reliable and effective recognition of
threat objects in passenger bags. Without training it can be difficult
to recognize threat items if they are not similar to objects that have
been encountered previously. Important is that even if screeners are
well trained, a reliable detection of threat items at the security
checkpoint cannot be guaranteed when attention and motivation of
screeners are insufficient. Therefore TIP can help to increase the
attention and motivation of screeners on the job. A training system
should be individually adaptive to train screeners optimally based on
their performances. A large image library should be used and threat
objects should be presented in different difficulties like easy and
difficulty rotations, low and high superpositions, and low and high bag
complexities. Very important is that feedback is given after each image
about the correctness of the answer. In addition, performance feedback
after each training session and also over all training sessions is an
important motivation for users. Finally, recurrent training with an
appropriate training duration (e.g., 1-2x 20 minutes per week) is
important to maintain a high x-ray image interpretation competency.
Take Home Message
Even with the most expensive technology the human operator is the
weakest link in the aviation security unless a screener is well trained
with a computer-based training system.
Training is essential to guarantee a consistent, reliable, and effective recognition of threat objects.
The image-based factors view dificulty, superposition and bag
complexity should be considered and therefore different difficulty
levels should be integrated into an effective training system.
The image library should contain a large variety of threat items.
Different views of threat objects should be presented.
Harmless objects are not necessary to be integrated into the
training image library because screeners encounter harmless objects
during the routine x-ray screening operation all the time.
One of the most important aspects of a good training system is
that feedback is provided after each image about the correctness of the
answer. More sophisticated, the missed threat is shown in the feedback.
After each training session, feedback about the performance should be given.
It should be possible that training instructors or supervisors
have the possibility to monitor the training progress of the screeners.
Finally, training should be recurrent to achieve reliable, effective and efficient recognition of threat objects.
References
Green, D. M., & Swets, J. A. (1966). Signal Detection Theory and Psychophysics. New York: Wiley.
Koller, S.M., Hardmeier, D., Michel, S., & Schwaninger, A.
(2007). Investigating training and transfer effects resulting from
recurrent CBT of x-ray image interpretation. In D. S. McNamara & J.
G. Trafton (Eds.), Proceedings of the 29th Annual Cognitive Science
Society (pp. 1181-1186). Austin, TX: Cognitive Science Society.
Hofer F., & Schwaninger, A. (2005). Using threat image projection data for assessing individual screener performance. WIT Transactions on the Built Environment, 82, 417-426.
Michel, S., de Ruiter, J. C., Hogervorst, Koller, S.M., M.,
Moerland, R., & Schwaninger, A. (2007). Computer-based training
increases efficiency in x-ray image interpretation by aviation security
screeners. Proceedings of the 41st Carnahan Conference on Security
Technology, Ottawa, October 8?11, 2007.
Schwaninger, A., Hofer, F., & Wetter, O.E. (2007). Adaptive computer-based training increases on the job
performance of x-ray screeners. Proceedings of the 41st Carnahan Conference on Security Technology, Ottawa, October 8?11, 2007.
Michel, S., Koller, S.M., Ruh, M., & Schwaninger, A. (2007). Do
"image enhancement" functions really enhance x-ray image
interpretation? In D. S. McNamara & J. G. Trafton (Eds.),
Proceedings of the 29th Annual Cognitive Science Society (pp.
1301?1306). Austin, TX: Cognitive Science Society.
Schwaninger, A. (2003b). Evaluation and selection of airport security screeners. AIRPORT, 02/2003, 14?15.
|