Physiological Signal — Based Engagement Level Analysis under Fuzzy Framework

Abstract:

The paper presents real-time affective state detection, in particular, the engagement level detection by using physiological signals under fuzzy framework. In order to develop the fuzzy model, the engagement model is developed by using data collected from controlled design experiment. In measuring the level of engagement, the physiological signals; namely the Electrooculogram (EOG) is recorded using G-tec data acquisition system. In the experiment, the data collected are the average endogenous eye blinks and the average trajectory errors recorded from the trajectory that the subjects have to follow in completing specific tasks. For the tasks, the subjects are asked to track a set of prescribed paths within the allocated times and have to obey different speed constraints. Various shapes of trajectories are given to the subjects in order to study the level of engagement while performing the task. The data then are used to develop the fuzzy model to measure the level of engagement (LOE) of the subjects. Following the experiments, a series of questionnaires are given to the subjects to verify their engagement level when performing the experiments. Preliminary analysis on the data shows a good match between the experimental results and the questionnaire.

References

[1] T. Howard in: Eye blinks and eye movements in cognition, National Science Foundation St. Olaf College Department of Psychology (2009).

[2] K. Grauman, M. Betke, J. Gips, and G. Bradski in: Communication via eye blinks-detection and duration analysis in real time, Proc. IEEE Computer Society Conference on computer vision and pattern recognition, vol. 1, pp.1010-1017 (2001).

DOI: 10.1109/cvpr.2001.990641

[3] M. Sanjeeva, B. Narasimha, E. Suresh, and K. Subba in: Analysis of EOG signals using wavelet transform for detecting eye blinks, Wireless communications and signal processing (WCSP), International Conference (2010).

DOI: 10.1109/wcsp.2010.5633797

[4] R. Schleicher, N. Galley, S. Briest, L. Galley in: Blinks and saccades as indicators of fatigue in sleepiness warnings, Ergonomics, 51(7): 982-1010 (2008).

DOI: 10.1080/00140130701817062

[5] C. N. Karson, K. F. Berman, E. F. Donnelly, W. B. Mendelson, J. E. Kleinman, and R. J Wyatt in: Speaking, thinking, and blinking, Psychiatry Research, 5(3): 243-246 (1981).

DOI: 10.1016/0165-1781(81)90070-6

[6] L. Orchard, J. Stern in: Blinks as an index of cognitive activity during reading, Integr Physiol Behav Sci, 26: 108-116 (1991).

[7] B. Andreas, A. W. Jamie, G. Hans, and T. Gerhard in: Eye movement analysis for activity recognition, 11th International Conference on Ubiquitous Computing (UbiComp), Zurich, (2009).

[8] R. Heishman, D. Zoran, W. Harry in: Using eye region biometrics to reveal affective and cognitive states, Augmented Cognition, HCII, LNAI 4565, pp.84-93 (2007).

[9] D. Palomba, M. Sarlo, A. Angrilli, A. Mini, and L. Stegagno in: Cardiac responses associated with affective processing of unpleasant film stimuli, International Journal of Psychophysiology, 36(1): 45-57 (2000).

DOI: 10.1016/s0167-8760(99)00099-9

[10] F. VanderWorf, P. Brassinga, D. Reits, M. Aramideh, and B. Ongerboer in: Eyelid movements: Behavioral studies of blinking I human under different stimulus conditions, JNeurophysiol 89: 2784-2796 (2003).

DOI: 10.1152/jn.00557.2002

[11] A. Bentivoglio in: Analysis of blink rate patterns in normal subjects, Mov Discord, 12: 1028-1034 (1997).

DOI: 10.1002/mds.870120629

[12] S. Sarah, K. Ami, and J. Warren in: Inhibition of eye blinking reveals subjective perceptions of stimulus salience, Proceeding of the national academy of science of United States of America, 108(52): 21270–21275 (2011).

DOI: 10.1073/pnas.1109304108

[13] L. Bacher, W. Smotherman in: Spontaneous eye blinking in human infants: A review, Dev Psychobiol, 44: 95-102 (2004).

DOI: 10.1002/dev.10162

[14] C. Fogarty, J. Stern in: Eye movements and blinks: Their relationship to higher cognitive process, International Psyphysiol, 8: 35-42 (1989).

DOI: 10.1016/0167-8760(89)90017-2

[15] H. Ric, and D. Zoran in: Using image flow to detect eye blinks in color videos, Workshop on Applications on computer visions (WACV'07) (2007).

DOI: 10.1109/wacv.2007.61

[16] R. Pramila, S. Nilanjan in: Operator engagement detection and robot behaviour adaptation in human-robot interaction, Proceeding of the IEEE, International Conference of Robotics and Automation, Barcelona, Spain (April 2005).

DOI: 10.1109/robot.2005.1570415

[17] R. Pramila, S. Jared, B. Robert, S. Nilanjan in: Online stress detection using psycho physiological signal for implicit human-robot cooperation, Robotica, Volume 20(6), pp.673-686, Cambridge University Press, U. K (2002).

[18] B. Kosko in: Neural networks and fuzzy systems, Prentice hall, India (1994).

[19] S.K. Pal, S. Mitra in: Neuro-fuzzy pattern recognition methods in soft computing, Wiley- Interscience (1999).