Using Pupil Diameter Changes for Measuring Mental Workload under Mental Processing

INTRODUCTION

Today, jobs requiring physical activity are gradually leaving their places to the jobs requiring much more mental processing. Job characteristics change from physical aspect to mental aspect, doing the job in person to inspect and control, or to thinking aspect. It is necessary to evaluate the workers not only with job characteristics which has inclined towards investigation and control but also with psychological and mental attributes as speed and correctness of perception, attention, memory, intelligence. It is needed comprehensive methods for analysis and evaluation of the jobs which requires mostly mental processing. A practical way to measure of mental workload that has the ability to evaluate operators’ mental effort in a multitask condition would be valuable in a natural working environment, because most such work is composed of multiple tasks. In this study, an approach is described that developed a combined measure of mental workload based on one physiological index one subjective index.

Systems designed which takes care of the results of mental workload measurements can help productivity and motivation. On the other hand, the fields like human-machine output quality, safety, personnel selection and recruitment could refer to the result of this kind of researches. It can be predicted that increasing mental workload can increase the mistakes related to the work and as a result of this, can multiply accidents. So, the mental workload information becomes vital to take actions and precautions related to accident prevention.

Avoiding excessive loading of brain is precedence; the least loading of brain is another important problem. In reality, it is intended that by avoiding mental workload to become excessive or the least, optimum mental workload should be provided which leads to personnel development and it keeps the person far from away upsetting.

Research headlines, defined according to predictions made by the researcher, which are the possible solutions of the problem, before testing without knowing whether they are valid or not (Kaptan, 1993).

The research headlines of this study are defined as, pupil diameter dilates or constricts when subject is performing thinking, time consuming, analyzing, synthesizing, recognizing tasks, with related psychomotor activities and there is relationship between the values measured during performing the related tasks and values found by applying NASA-TLX subjective measurement method in respect of discrepancy or not.

Basic assumptions, on which research is based: 1. Iris diameter, pupil diameter ratio (I/P) is the correct derivation of pupil diameter where the measurements are taken on each picture which makes up the video record (6 pictures sec^-1) captured with the head mounted camera, calculated by using the distances between the vertical lines drawn tangent to right and left sides of iris and the vertical lines drawn tangent to right and left sides of pupil. 2. Subjects did not communicate and influence each other on the experiment material and psychomotor task. 3. Subjects performed required activity as it is described on experiment directive. 4. The expert opinion on validity of the questions included in test tool is sufficient. 5. Subjects were motivated equally.

Mental workload concept is a multidisciplinary field including ergonomics, psychology, physiology, medicine, statistics, engineering etc. The research made in this field is a multidisciplinary study and the related publications in these fields were tried to be studied.

Mental workload has long been recognized as an important element of human performance in complex systems. Moray (1988) found that the optimization of mental workload could reduce human error, improve system safety, increase productivity and increase operator satisfaction. Mental workload has a direct influence over a user ability to perform tasks and can therefore impact the effectiveness and efficiency of interactions with computers (Vitense et al., 2003).

Cognitive (mental) workload has been defined as the interaction between the demand of a task that an individual experiences and his or her ability to cope with these demands. Hence, it arises due to a combination of the task demands and the resources that a particular individual has available (Noyes et al., 2004).

The mental workload of a task represents the level of attentional resources required to meet both objective and subjective performance criteria, which may be mediated by task demands, external support and past experience. One of the hurdles in deriving a subjective measure of MWL is validating it by correlation with other measures (such as physiological evidence) or objective demand (Young and Stanton, 2002).

Even though there are many environmental factors that can increase the mental workload, noise, temperature and light level are the most important. They have important influence on the performance, attention and effectiveness of the worker (Martinez de la Teja, 2001).

Mental workload is a structure assumed as a multidimensional reflection of mental effort and attention directing of an individual. Measurement of mental workload represents enumeration of mental process which occurred during performing a task or task group (Baldwin, 2003).

The response sensitivity to a mental task is different in each person. The physiological responses induced by the same task may differ from person to person. This is the individual difference problem. Furthermore, the physiological response pattern is different from task to task (Miyake, 2001).

Mental workload measurement techniques can be classified in three groups;

The question Which method of measurements is the best? is open to discussion in respect of aim of experiment, tasks performed and environment in which the experiment taking place with the benefits and limits of the indices (Baldwin, 2003).

Many of the research pertaining to mental workload, has relied on behavioral measures of response time and accuracy. Behavioral measures have enabled substantial progress in systems design. For example, response time and accuracy measures have demonstrated that mental workload increases when driving in urban as compared to rural settings. Some tasks that are time consuming are not very difficult. The monotony of these time-consuming, tedious tasks as well as tasks which demand sustained attention over long periods of time result in increased errors and delayed reaction time due to lapses of attention. Therefore response time and accuracy do not always or necessarily directly assess mental workload and may be insensitive measures of the difference between inherently difficult tasks versus difficult situations requiring sustained attention for extended periods of time (Baldwin, 2003).

Behavioral measurements are made by primary task measurement, two tasks or secondary task measurement methods.

Primary task approach: One of the attempts for quantification mental workload of a task defines workload simply as dividing the time period required to achieve the task to the time available. A few computer modeling program were developed to make such a calculation for a task period.

Secondary task approach: The logic behind the usage of secondary task measurements is that there is spare mental capacity, not directed to primary task, which can be used for performing the secondary task.

Measurement of spare mental capacity is based on the rationale that the operator has limited capacity to process information. One method to measure the mental workload is to give the operator a secondary task in addition to his/her main task the level of performance at secondary task will reflects the difficulty of main task. Secondary task is designed so that performance can be measured objectively by means of performance time and/or number of errors. The measure of secondary task performance can be used as an index of primary task load (Martinez de la Teja, 2001).

The logic behind the physiological indexes used for measuring mental workload is the activities created at central nervous system, on information processing of the brain, which has detectable indications. Heart rate, respiration, blood pressure, breathing, eye blinking, pupil diameter and brain activity level changes are some of the physiological indicators which are emerged in relation with mental workload. Recent advances in technological capabilities, analytical techniques and the increasing availability of equipment for non-invasive, real time assessment of human brain function have led to revolutionary advances in mental workload measurements.

Neuroergonomics, or examination of the changes in brain functioning and levels of brain activity associated with the performance of perceptual, cognitive and physiological tasks during work and leisure, provides an opportunity for a more direct assessment of mental of mental workload. Measurement of how hard the brain is working through neuroimaging techniques, such as Functional Magnetic Resonance Imaging (FMRI), electroencephalography (EEG) and Event-Related Potentials (ERPs), represent a major step forward in advancing proactive science-based system design (Baldwin, 2003).

Electroencephalography (EEG), recording of brain spontaneous activity, is recorded on the surface of haired head skin by surface electrodes or from deep brain structures by electrodes located with stereotactic method. If a physical stimulus (auditory, visual and electrical) applied to a person or an experiment animal from whom his/her EEG is obtained, a change lasting one second occurs in spontaneous activity of the brain. This reaction is called as Stimulus Potentials (SP). The stimulus potentials related with cognition, interpretation, memory and behavior are called as Event-Related Potentials (ERP) (Özesmi et al., 2000).

Event Related Oscillations (ERO) appear by decomposing of ERPs. The oscillations which are reliable events of the brain are the basic element of principles explaining brain processing. These oscillations displayed at different frequencies. The oscillations, underlined in respect of brain functioning, are consisting of delta, theta, alpha, beta and gamma. The findings show that frequency bands are divided in to lower level frequency bands as slow, medium fast gamma and slow and fast alpha. If these lower bands are taken into account, the total number of oscillations reaches to ten (Karakaş and Başar 2003).

Subjective mental workload can be defined as the subject’s direct estimate or comparative judgment of the mental or cognitive workload experienced at a given moment (Luximon and Goonetilleke, 2001).

Subjective assessment techniques have the advantage of being relatively easy to administer and interpret and they do not require extensive training or equipment. Despite their frequent use in aviation settings, subjective workload measures tend to be situation-specific and most often fail to take into account adaptivity, learning, experience, natural ability and changes in emotional state on the part of the operator. Further, when used alone, subjective measures reveal little in terms of the brain mechanism involved in task performance and, more importantly, from a systems design perspective, can not be used for continuous on-line monitoring of workload (Baldwin, 2003).

Several types of uni and multidimensional subjective scales exist. Examples are Cooper Harper scale, direct scaling, multidescriptor scale, the workload-compensation-interference/technical effectiveness scale, the overall workload scale, the consumer mental workload scale, SWAT and the National Aeronautics and Space Administration-Task Load Index (NASA-TLX). However, results from various studies have shown that NASA-TLX and SWAT are very popular and are widely used. Even though much effort has been made to develop objective measures of workload, subjective workload assessment techniques continue to be popular due to their ease of use, general non-intrusiveness, low cost, high face validity and known sensitivity to workload variations (Luximon and Goonetilleke, 2001).

NASA-TLX is a multidimensional scale for which the overall mental workload is a function of 1. Mental Demand (MD), 2. Physical Demand (PD), 3. Temporal Demand (TD), 4. Own Performance(OP), 5. Effort(EF), 6. Frustration Level (FR).

The degree to which each of the six factors contribute to the workload of the specific task to be evaluated, from the raters’ perspectives, is determined by their responses to pair-wise comparisons among the six factors. Magnitude ratings on each subscale are obtained after each performance of a task or task segment. Ratings of factors deemed most important in creating the workload of a task are given more weight in computing the overall workload score, thereby enhancing the sensitivity of the scale. The definitions of NASA-TLX basic factors are given at Table 1. NASA-TLX technique can be applied manually or on PC by filling up the related questionnaire.

Eye is one of the most important information sources to improve a basic methodology for enumerating the psychological data related with stress, fatigue and stimulation. Pupil activity is emerging as a basic field for the related research.

Occipital lobes take place at back of the brain and process the images coming from the eye by providing connection with the images stored at the memory.

Table 1:	NASA-TLX rating scale definitions

The reaction of pupil to the stimulus which causes general awakefulness, is controlled by autonomic nervous system. System is alerted by general awakefulness, excitements and some other situations. It is differentiated from the rest of nervous system by serving secretion glands, smooth muscles and cardiac muscle of the heart. The muscles which control the pupil dilation are smooth muscles.

Autonomic nervous system composed of sympathetic nervous system and parasympathetic nervous system. These two system acts antagonistically. For example, the action of sympathetic nervous system tends to increase activity, speed the heart and circulation, to dilate pupil and slow digestive processes. On the other hand parasympathetic nervous system tends to slow down activity in the glands and smooth muscles which the supply, to contradict pupil but promotes digestion. Sympathetic nervous produce noradrenalin and, to a lesser extend, adrenaline. Generally sympathetic nervous system provides general awakefulness, it prepares the organism to meet hasty situations. Parasympathetic nervous system resist to sympathetic nervous system, acts for preserving and saving stored energy (Morgan, 1980).

Associations have been found between human reflexes and measure of cognitive ability. Specifically, pupil reflexes have been related to a vast array of cognitive abilities including short term memory, long term memory, choice reaction time, language processing, attention, mathematical task complexity and individual differences in cognitive ability tests. These studies have consistently found that greater pupillary dilation is observed when cognitive task difficulty is increased and pupil dynamics is claimed to generally reflect human information processing load. The magnitude of pupil dilation has been related to the difficulty of mathematical task being performed and the differences in individual cognitive ability. Pupillary dilation latency has been related to the difficulty of mathematical tasks and inversely related to cognitive ability. Evidence suggest that pupillary constriction latency may also be related to mathematical task difficulty, and therefore, may also be related to cognitive ability (Bryan and Stone, 2003).

The eyes, considered as extensions of the brain to outer world is manifested the secrets of the brain.

Interest value of material viewed: Pupil dilation research initially focused upon emotional reactions and then shifted almost entirely to information processing. Hess and Polt (1960) found that the pupils of women dilated more to pictures of babies, mothers and babies and nude men, while the pupils of men dilated more to pictures of landscapes and nude women.

Hess had emphasized dilation in response to nude opposite-sex pictures (Hess, 1965), Aboyoun and Dabbs (1998) found that pupil dilation to nudity occurred irrespective of the sex of the subject. It appeared that the novelty of nude pictures of either sex produced dilation and the greater novelty of nude male pictures produced greater dilation (Aboyoun and Dabbs, 1998).

Information intake and processing: Kahneman (1973) and Beatty (1982) have utilized pupil diameter changes as reflectors of information processing requirements. Requiring subjects to commit a series of numbers to memory, hold that information in memory for a few seconds and then requiring them to repeat the numbers produced increases in pupil diameter proportional to the numbers of items in the memory set. Larger sets produced more dilation than smaller sets. Requiring subjects to manipulate the information by having them repeat the numbers in the reverse order in which they had been presented produced significantly greater pupil dilation as the numbers were presented with a further increase in pupil diameter during the delay period. Requiring subjects to commit more numbers to memory than they are capable of remembering produces patterns of pupil diameter change reflective of their strategy. Subjects who give up on the task and report that they could not remember the numbers show a different pattern of dilation and constriction than those who report at least some of the numbers in the memory set. The former demonstrate a pattern of dilation until their memory is taxed with a return to baseline levels at this point while those who are able to report some of the numbers show dilation which is maintained until called upon to report the numbers. Thus strategy differences are nicely reflected in pupil diameter changes. Other researchers have looked at variables such as sentence complexity as well as lexical complexity and demonstrated that greater complexity led to greater pupil dilation. These pupil dilation effects occur irrespective of mode of information presentation. Presenting the material verbally produces changes as great as those when the information is presented visually (and light intensity is held constant). The changes in pupil diameter are quite modest when compared to those associated with changes in light intensity. They generally are less than 0.2 mm in diameter (Stern, 1997).