Name: using electroencephalography measurements and high-quality video recording for analyzing visual perception of media content
Uploaded: 2018-05-26T14:00:00.000+00:00
Duration: 10 min 41 s
Description: Watch this Scientific Journal Video about Using Electroencephalography Measurements and High-quality Video Recording for Analyzing Visual Perception of Media Content at JoVE.com

The present study has been supported by a Spanish Ministry of Economy and Competitiveness (BFU2014-56692-R and BFU2017-82375-R) grants.

All methods described here have been approved by the Ethics Commission for Research with Animals and Humans (CEEAH) of the Universitat Aut&#242;noma de Barcelona.
1. Creation and Presentation of the Visual Stimuli
<ol>
	<li>Create the video stimuli according to the desired goals.
	<ol>
		<li>Decide which variables are of interest for the video stimuli. 
		NOTE: For example, in the present investigation, the main variable of interest is the editing style.
		<ol>
			<li>Decide the duration, the content, and the style of the video based on the interest of the experiment. To determine all of these variables, consider how many videos the participant is going to watch and how long the participant will be watching the screen, evaluate if the narrative of the video meets the interest, and decide the visual style of the video.</li>
		</ol>
		</li>
		<li>Write the script, paying attention to the chosen variables.</li>
		<li>Record the video with a high-quality video camera, and edit the video as needed. Seek professional help if necessary.</li>
	</ol>
	</li>
	<li>Design an appropriate context for the stimuli presentation.
	<ol>
		<li>Decide where the data acquisition will take place and design a comfortable space to display the stimuli. Participants should feel at ease.</li>
		<li>Select a high-quality screen for the presentation of the video. The higher the quality of the screen, the better.</li>
		<li>If there are several video films to be presented, select a method for the randomization of their presentation. 
		NOTE: It is convenient to randomize stimuli presentation across the subjects to avoid inconclusive results due to the order in which the participants perceive a stimulus.</li>
		<li>Choose a software package to present the videos. 
		NOTE: There are various options that allow the researcher to display different stimuli with inter-trial timers and instructions, or even a randomized presentation. When choosing the software, it is critical to consider its capability for synchronization with data acquisition devices. Since this protocol analyzes participants&#39; eyeblink rate, the synchronization between the visual presentation of the visual stimuli and the EEG device is crucial.</li>
	</ol>
	</li>
</ol>
2. Selection of Participants
<ol>
	<li>Establish a criterion for choosing participants. When a specific variable is going to be analyzed, a control group is needed. Write down the criterion and consult with colleagues to improve upon the study design.</li>
	<li>Select participants and seek approval from an ethics committee that follows relevant guidelines and regulations. 
	NOTE: Most universities and research centers have an ethics committee. To apply for approval, it will be necessary to inform them of the investigation goals. Common requirements include explaining the research methods, the benefits of the investigation to society, how informed consent will be obtained, whether participants will be recompensed financially, and how data collected from participants will be managed. Take time to design this request. It is very important to get the approval of an ethics committee to share the results with the scientific community. Some documents that can be useful in this respect are the Declaration of Helsinki, the Nuremberg Code, and the Belmont Report.</li>
	<li>Design an informed consent form for participants, and follow the criteria of the ethics committee.</li>
	<li>Explain the experiment to every potential participant. Let the participant know all the important information, but avoid information that is not important or can affect the results of the experiment. 
	NOTE: For example, in an eyeblink detection protocol, the participant cannot be told that eyeblinks are going to be detected, as this could modify the natural eyeblink rate. Ask for their approval to participate. Give each participant the document for informed consent and allow them enough time to read it, to understand the text, and, finally, to sign it.</li>
	<li>Make an appointment with each participant to conduct the experiment. Give the participant the following instructions: to arrive rested and with recently washed hair; chemical products such as hairspray, etc., should be avoided because they can affect EEG recordings. Give the participant all the necessary contact information.</li>
</ol>
3. Set-up Preparation
<ol>
	<li>Prepare the experimental session well in advance of the participant&#39;s arrival. Rehearse the session with a colleague. In case something incorrect is detected, make the corresponding modifications to the experimental session.
	<ol>
		<li>Prepare connections between all devices that are going to be used and check their synchronization (visual stimuli presentation and data acquisition). 
		NOTE: The connectivity between hardware and software is critical. It is desirable to perform tests to confirm synchronization. Discuss with the manufacturer or software developer, if needed.</li>
	</ol>
	</li>
	<li>Make the subjects feel comfortable in the experimental room: welcome the participant, and prepare a space for their things (bags and others).</li>
	<li>Explain the entire protocol to the participant: where the participant will be seated, how many visual stimuli are going to be shown, and the length of the session. Explain that the participant&#39;s cooperation is extremely important.</li>
	<li>Since a wireless EEG recording system is going to be used, explain how difficult these recordings are, and that in order to obtain proper electrophysiological signals with low artifacts, the participant should not move unnecessarily.</li>
</ol>
4. Data Acquisition
<ol>
	<li>Clean the surface of the skin of the participant to remove any traces of dirt or remnants of hair products.</li>
	<li>Measure the head of the participant. Measure the distance between the nasion and the inion, and mark the half-way point as a vertex.</li>
	<li>Prepare the electrodes in the EEG cap using the 10-20 system.</li>
	<li>Put the cap on the subject following the mark so that the vertex matches the Central zero (Cz) point of the cap.</li>
	<li>Apply conductive gel with the help of a syringe to the electrodes; make sure the cap fits well on the subject&#39;s head.</li>
	<li>Place the reference electrode and the EOG electrode on the subject&#39;s head.</li>
	<li>Make sure that the participant is comfortably seated and prepared for the session.</li>
	<li>Focus an HD video camera on the participant&#39;s face in close-up. This will be used to manually detect the participants&#39; eyeblinks.</li>
	<li>Check that all devices are synchronized.</li>
	<li>Check that the EEG control box is properly connected to the computer registering the data.</li>
	<li>Check that the signal for each electrode is correct; if not, check for the problematic electrode. 
	NOTE: Some common reasons for a bad signal from an electrode are a lack of conductive gel on the electrode, a bad position of the electrode on the head, or a bad connection of the electrode cable with its control box.</li>
	<li>When everything is ready, start recording the experiment (and playing the video).</li>
</ol>
5. Post-experimental Session
<ol>
	<li>Remove the cap from the participant&#39;s head and, if possible, give the participant a questionnaire related to the topics of interest.</li>
	<li>Save all the data to an external hard drive. 
	NOTE: It is recommendable to back up the data. A good option would be to save the data to two external hard drives and store them in different places. In designing the best procedure, it is important to consider the protocol for data storage approved by the ethics committee.</li>
	<li>Clean the electrodes and cap; most electrodes need to be cleaned immediately after use.
	<ol>
		<li>Clean the electrodes and cap with abundant water under a faucet and without chemical product (unless stated otherwise by the device manufacturer). Carefully dry the electrodes to prevent corrosion. If this protocol is insufficient to clean the cap, it can usually be washed in a washing machine (see the manufacturer&#39;s instructions).</li>
	</ol>
	</li>
</ol>
6. Data Analysis
<ol>
	<li>Organize the eyeblink detection data.
	<ol>
		<li>Import the EEG file into a suitable software package for data analysis.</li>
		<li>Select the EEG area of interest based upon the synchronization triggers used with the visual stimuli presentation software. Filter the data and apply an eyeblink filter to see how many and when eyeblinks are generated. 
		NOTE: Most commercial and free software packages have filters to detect eyeblinks in EEG.</li>
		<li>Select the HD video of participants&#39; faces and manually match eyeblinks with those found in the EEG data.</li>
		<li>Create a spreadsheet matrix with the final list of each participant&#39;s eyeblinks, and compute the rate per minute while the subject was watching the stimuli.</li>
	</ol>
	</li>
	<li>Perform statistical analysis.
	<ol>
		<li>Import the matrix into the corresponding software for statistical analysis.</li>
		<li>In a statistical software program, design the desired analysis to contrast the initial hypotheses and complete them.</li>
	</ol>
	</li>
</ol>

<ol>
	<li>Shapiro, K. L., Raymond, J. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+attentional+blink:+temporal+constraints+on+consciousness.">The attentional blink: temporal constraints on consciousness.</a> Attention and Time. , 35-48 (2008).</li><li>VanderWerf, F., Brassinga, P., Reits, D., Aramideh, M., Ongerboer de Visser, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eyelid+movements:+behavioral+studies+of+blinking+in+humans+under+different+stimulus+conditions.">Eyelid movements: behavioral studies of blinking in humans under different stimulus conditions.</a> Journal of Neurophysiology. 89 (5), 2784-2796 (2003).</li><li>Andreu-S&#225;nchez, C., Mart&#237;n-Pascual, M. &. #. 1. 9. 3. ;., Gruart, A., Delgado-Garc&#237;a, J. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eyeblink+rate+watching+classical+Hollywood+and+post-classical+MTV+editing+styles,+in+media+and+non-media+professionals.">Eyeblink rate watching classical Hollywood and post-classical MTV editing styles, in media and non-media professionals.</a> Scientific Reports. 7, 43267 (2017).</li><li>Bour, L. J., Aramideh, M., de Visser, B. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neurophysiological+aspects+of+eye+and+eyelid+movements+during+blinking+in+humans.">Neurophysiological aspects of eye and eyelid movements during blinking in humans.</a> Journal of Neurophysiology. 83 (1), 166-176 (2000).</li><li>Delgado-Garc&#237;a, J. M., Gruart, A., M&#250;nera, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neural+organization+of+eyelid+responses.">Neural organization of eyelid responses.</a> Movement disorders: official journal of the Movement Disorder Society. 17 Suppl 2, S33-S36 (2002).</li><li>Wiseman, R., Nakano, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blink+and+you'll+miss+it:+the+role+of+blinking+in+the+perception+of+magic+tricks.">Blink and you'll miss it: the role of blinking in the perception of magic tricks.</a> PeerJ. (4), e1873 (2016).</li><li>Fogarty, C., Stern, J. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eye+movements+and+blinks:+their+relationship+to+higher+cognitive+processes.">Eye movements and blinks: their relationship to higher cognitive processes.</a> International Journal of Psychophysiology official journal of the International Organization of Psychophysiology. 8, 35-42 (1989).</li><li>Andreu-S&#225;nchez, C., Mart&#237;n-Pascual, M. &. #. 1. 9. 3. ;., Gruart, A., Delgado-Garc&#237;a, J. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Looking+at+reality+versus+watching+screens:+Media+professionalization+effects+on+the+spontaneous+eyeblink+rate.">Looking at reality versus watching screens: Media professionalization effects on the spontaneous eyeblink rate.</a> PLoS ONE. 12 (5), (2017).</li><li>Nakano, T., Yamamoto, Y., Kitajo, K., Takahashi, T., Kitazawa, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Synchronization+of+spontaneous+eyeblinks+while+viewing+video+stories.">Synchronization of spontaneous eyeblinks while viewing video stories.</a> Proceedings. Biological sciences / The Royal Society. 276, 3635-3644 (2009).</li><li>Tadel, F., Bock, E., Mosher, J. C., Baillet, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Detect and remove artifacts. , (2015).</li><li>Jiang, X., Tien, G., Huang, D., Zheng, B., Atkins, M. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Capturing+and+evaluating+blinks+from+video-based+eyetrackers.">Capturing and evaluating blinks from video-based eyetrackers.</a> Behavior Research Methods. 45 (3), 656-663 (2013).</li><li>Pedrotti, M., Lei, S., Dzaack, J., Rotting, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+data-driven+algorithm+for+offline+pupil+signal+preprocessing+and+eyeblink+detection+in+low-speed+eye-tracking+protocols.">A data-driven algorithm for offline pupil signal preprocessing and eyeblink detection in low-speed eye-tracking protocols.</a> Behavior Research Methods. 43 (2), 372-383 (2011).</li><li>Adam, A., Ibrahim, Z., Mokhtar, N., Shapiai, M. I., Mubin, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Evaluation+of+different+peak+models+of+eye+blink+EEG+for+signal+peak+detection+using+artificial+neural+network.">Evaluation of different peak models of eye blink EEG for signal peak detection using artificial neural network.</a> Neural Network World. 26 (1), 67-89 (2016).</li><li>Zhu, Z., Ji, Q. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Robust+real-time+eye+detection+and+tracking+under+variable+lighting+conditions+and+various+face+orientations.">Robust real-time eye detection and tracking under variable lighting conditions and various face orientations.</a> Computer Vision and Image Understanding. 98, 124-154 (2005).</li><li>Lalonde, M., Byrns, D., Gagnon, L., Teasdale, N., Laurendeau, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Real-time+eye+blink+detection+with+GPU-based+SIFT+tracking.">Real-time eye blink detection with GPU-based SIFT tracking.</a> Proceedings - Fourth Canadian Conference on Computer and Robot Vision, CRV 2007. , 481-487 (2007).</li><li>Chang, W., Cha, H., Kim, K., Im, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detection+of+eye+blink+artifacts+from+single+prefrontal+channel+electroencephalogram.">Detection of eye blink artifacts from single prefrontal channel electroencephalogram.</a> Computer Methods and Programs in Biomedicine. 124, 19-30 (2016).</li><li>Tadel, F., Bock, E., Mosher, J. C., Baillet, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Brainstorm3: Function process_evt_detect.m. , (2018).</li><li>Hall, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+origin+and+purposes+of+blinking.">The origin and purposes of blinking.</a> The British Journal of Ophthalmology. 29 (9), 445-467 (1945).</li><li>Nakano, T., Kitazawa, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eyeblink+entrainment+at+breakpoints+of+speech.">Eyeblink entrainment at breakpoints of speech.</a> Experimental Brain Research. 250, 577-581 (2010).</li><li>Siegle, G. J., Ichikawa, N., Steinhauer, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blink+before+and+after+you+think:+Blinks+occur+prior+to+and+following+cognitive+load+indexed+by+pupillary+responses.">Blink before and after you think: Blinks occur prior to and following cognitive load indexed by pupillary responses.</a> Psychophysiology. 45, 679-687 (2008).</li><li>Shultz, S., Klin, A., Jones, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Inhibition+of+eye+blinking+reveals+subjective+perceptions+of+stimulus+salience.">Inhibition of eye blinking reveals subjective perceptions of stimulus salience.</a> Proceedings of the National Academy of Sciences of the United States of America. 108 (52), 21270-21275 (2011).</li><li>Wong, K. K. W., Wan, W. Y., Kaye, S. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blinking+and+operating+cognition+versus+vision.">Blinking and operating cognition versus vision.</a> BritishJournal of Ophthalmology. 86, 479 (2002).</li><li>Leal, S., Vrij, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blinking+during+and+after+lying.">Blinking during and after lying.</a> Journal of Nonverbal Behavior. 32 (2008), 187-194 (2008).</li><li>Murch, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> In the blink of an eye: A perspective on film editing. , (1995).</li><li>Holland, M. K., Tarlow, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blinking+and+thinking.">Blinking and thinking.</a> Perceptual and Motor Skills. 41 (2), 503-506 (1975).</li><li>Orchard, L. N., Stern, J. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blinks+as+an+index+of+cognitive+activity+during+reading.">Blinks as an index of cognitive activity during reading.</a> Integrative Physiological and Behavioral Science. 2, 108-116 (1991).</li><li>Faubert, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Professional+athletes+have+extraordinary+skills+for+rapidly+learning+complex+and+neutral+dynamic+visual+scenes.">Professional athletes have extraordinary skills for rapidly learning complex and neutral dynamic visual scenes.</a> Scientific Reports. 3 (1154), 1-3 (2013).</li><li>Kirk, U., Skov, M., Schram Christensen, M., Nygaard, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brain+correlates+of+aesthetic+expertise:+A+parametric+fMRI+study.">Brain correlates of aesthetic expertise: A parametric fMRI study.</a> Brain and Cognition. 69, 306-315 (2008).</li><li>Lotze, M., Scheler, G., Tan, H. R., Braun, C., Birbaumer, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+musician's+brain:+functional+imaging+of+amateurs+and+professionals+during+performance+and+imagery.">The musician's brain: functional imaging of amateurs and professionals during performance and imagery.</a> Neuroimage. 20 (3), 1817-1829 (2003).</li><li>Calvo-Merino, B., Glaser, D. E., Gr&#232;zes, J., Passingham, R. E., Haggard, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Action+observation+and+acquired+motor+skills:+an+FMRI+study+with+expert+dancers.">Action observation and acquired motor skills: an FMRI study with expert dancers.</a> Cerebral cortex. 15 (8), 1243-1249 (2005).</li></ol>

The authors have nothing to disclose.

A method for analyzing visual perception of media content with purpose-made video creation is described here. Many other studies attempt to analyze the perception of media content in narrative contexts with pre-existing films. The present method proposes to create visual content with a narrative construction following the criteria of interest, and is based on the suggestion that eyeblink rate is connected to the viewer's attention9. For that reason, the study detects participants' eyeblinks while ...

The Purpose of This Method
This method proposes a dual protocol for detecting eyeblinks. The aim is to analyze viewers&#39; visual perception of media content, specifically created for this investigation, by using EEG recordings and HD video recording systems.
The Rationale Behind the Development and/or Use of This Method
Each eyeblink hides visual flow for 150-400 ms1,2. Blinking has physiological3,4,5 and psychological6,7 functions. The connection between attention and eyeblink rate has been studied and proven in different studies8. A higher level of attention decreases eyeblink rate and according to previous studies, humans share a mechanism for controlling the timing of blinks that searches for the best moment to avoid loss of important visual information9. Thus, analyzing eyeblink behavior of viewers when watching screens could provide information regarding the level of attention given to media contents.
One method for detecting spontaneous eyeblink rate is by using EEG electrodes to record the electrical activity. Eyeblinks can easily be detected by prefrontal and electrooculogram electrodes connected to an EEG recording system. In most EEG analyses, eyeblinks are considered artifacts. For this reason, many software packages designed for analyzing EEG data have eyeblink detectors10. The advantage of using EEG for detecting eyeblinks is the high temporal resolution (in the order of milliseconds) and the possibility of registering brain effects of different narratives and cuts in movies synchronized with those eyeblinks - a matter open to further study. Recording participants&#39; faces with an HD camera can also be useful for matching/contrast purposes9.
The Advantages over Alternative Methods with References to Relevant Studies
There are multiple methods for counting eye blinks. Some dedicated instruments for detecting blinks are magnetic coils, infrared (IR) light beams, optoelectronic motion detectors with eye movement analysis such as eye-tracking techniques, and several techniques based on bioelectrical signals, e.g., electrooculography (EOG), electromyography (EMG), and EEG. Another more accurate, but time-consuming option is manually counting blinks from a frame-by-frame video recording11. The technologies today can be classified broadly into two groups: a) contact-free recording which includes two modalities, the direct blink detection using computer vision and offline blink detection using eye-tracking, and b) contact-based recording using biological signals through EOG and EEG devices12,13.
The eye-tracking system is a widely used technology, ranging from traditional image-based passive designs to the active near-infrared-based approaches mainly used today with a high-resolution camera. The latter exploits the reflective properties of the pupil under IR illumination14. The concept underlying modern eye-tracking methods is Pupil Center Corneal Reflection (PCCR), which involves a camera tracking the center of the pupil, where light reflects from the cornea. However, there is a lack of blink detection algorithms published for eye-tracking protocols. Moreover, although the different models of eye-tracking on the market provide integrated software with blink detection, the source code is not always provided by manufacturers, making it difficult to modify or know how the algorithms work12. Also, during experiments with eye-tracking there are events that cause data loss, such as tracking delays and significant head or gaze movements. The eye area is very small in video captures, which is a problem for calculating the duration of the blink, and which sometimes introduces various types of artifacts15.
In this experiment, EEG and EOG methods are used. EEG is not usually used alone to detect eyeblinks. However, analyzing eyeblinks recorded with EEG electrodes is a standard procedure for the study of eyelid displacements. This procedure enables researchers to have information of exactly when eyeblinks take place. The most common signal pattern for detecting blinks is that of peak points, representing vertical movement responses. There are several peak detection algorithms applicable to raw EEG, time-domain, or frequency-domain signals. Processes involved in peak identification are peak detection, feature extraction, and classification. Eyeblinks have a considerable effect on frontal channels of the EEG signal. Typically, eyeblinks are detected in EEG by using a pre-determined amplitude threshold16. The algorithms in the analysis software used in this experiment are based on the signals&#39; standard deviation and the root mean square (RMS) of the pre-filtered EEG signal; they are open source and available to the scientific community17. However, some eye movements not involving eyeblinks can provoke electrical activity that may be confusing. For that reason, a second method - recording viewers&#39; faces with an HD video camera - allows researchers to match eyeblinks by manually counting them. With such a double method, the investigator achieves a matrix of eyeblinks that can be easily analyzed with statistical tools.
Therefore, the proposed method performs a data triangulation with two different sources to validate the detected eyeblinks. This method is based on Nakano et al. indications9 for confirmation. At the same time, it also enables researchers to collect brain-activity and frequency-band information for further analysis. The experiment described here is part of a wider future investigation into the effects of editing-style cuts on occipital and prefrontal brain areas.
Determine Whether the Method is Appropriate for an Investigation
This experimental protocol enables viewers&#39; eyeblinks while watching video content to be studied under three experimental conditions. First, eyeblink rate is detected by using two complementary techniques: EEG and recorded HD videos. Here, we use a wireless EEG with 20 channels. Second, specific stimuli adapted to the experiment are created, so that the researcher can manage all the variables of the visual content. Here, three videos with the same narrative but different video-editing style were created. The narrative consisted of a man who entered a room, sat at a desk, juggled with three balls, opened a laptop, looked up information in some books, typed something on the laptop, closed it, ate an apple, looked directly into camera, and left the room. The three video stimuli last 198 s each. The first was a one-shot movie; the second was edited according to classical Hollywood-style rules with 33 different shots; and the third was edited following MTV-style rules with 79 shots. A fourth stimulus was also presented in which the narrative was identical, but the format was a real representation with an actor instead of a video. This fourth non-video stimulus was not used in an initial study of editing-style differences but was used in a different investigation to compare eyeblink-rate differences between real representation and screened media8. Third, different groups of participants are selected depending on their previous expertise in the visual analysis of videos. The purpose is to determine differences in eyeblink rates of subject groups watching the same visual stimuli. In this case, 40 subjects took part in the investigation. Half of them were media professionals (16 males and 4 females; age 30-56 years, with an average age of 44.15 &#177;7.15 years) and the rest were non-media professionals (15 males, and 5 females; age 28-56 years, with an average age of 43.25 &#177; 8.59 years). Media professionals were chosen with the criterion of more than 6 years of experience in making decisions related to media editing in their everyday work.

<table><tbody><tr><td>EEG Device</td><td>Neurolectrics</td><td>Enobio 20 EEG/EMG system&amp;nbsp;</td><td></td></tr><tr><td>Ag/AgCl Electrodes</td><td>Neuroelectrics</td><td>[NE022b] GelTrode</td><td></td></tr><tr><td>Recording EEG software</td><td>Neuroelectrics</td><td>NicOffline software</td><td></td></tr><tr><td>HD-video camera</td><td>Sony Corporation&amp;nbsp;</td><td>Sony HDR-GW55VE</td><td></td></tr><tr><td>Syringe</td><td>Monoject</td><td>Monoject 412, curved tip syringe, 50/box</td><td></td></tr><tr><td>Saline electrode EMG gel</td><td>Signa-Gel</td><td>X32-204: Signa Gel</td><td></td></tr><tr><td>Visual Stimuli Presentation Software</td><td>Paradigm Stimulus Presentation</td><td>Perception Research System Incorporated</td><td></td></tr><tr><td>EEG software analysis</td><td>Centre National de la Recherche Scientifique and Montreal Neurological Institute</td><td>Brainstorm3</td><td></td></tr><tr><td>EEG software analysis</td><td>The MathWorks Inc.</td><td>MATLAB 2013b</td><td></td></tr><tr><td>TV for video presentation</td><td>Panasonic Corporation</td><td>PanasonicTH- 42PZ70EA&amp;nbsp; - 50&quot;</td><td></td></tr><tr><td>PC for presenting stimuli</td><td>MacBook Air</td><td>Year 2013, running Mac OSX Mountain Lion</td><td></td></tr><tr><td>PC for recording stimuli</td><td>MacBook&amp;nbsp;</td><td>Year 2009 running Windows 7</td><td>With a bootcamp partition of the disk for providing Windows OS</td></tr><tr><td>Statistical Analysis</td><td>Systat Software Inc.</td><td>Sigmaplot 11.0&amp;nbsp;</td><td></td></tr></tbody></table>

using electroencephalography measurements and high-quality video recording for analyzing visual perception of media content

This article explores a method to detect differences in visual perception in humans. The method used is based on the psychological (or "cognitive") function of eyeblinks. Participants' eyeblinks are detected and acquired while watching videos specifically created for the investigation. The detection and acquisition of eyeblinks are carried out with the help of a 20-channel electroencephalographic (EEG) wireless device. The international 10-20 system for electrode placement is followed. A high-definition (HD) video camera is used to record participants' facial expressions, for contrast purposes. Instead of using pre-existing media content, purpose-made video content has been created following specific criteria of interest for this investigation, with stimuli enabling researchers to manage the precise parameters of interest. Otherwise, results could be contaminated with uncontrolled variables. The synchronization of the presentation of video stimuli with EEG recordings needs to be done in milliseconds. Analysis of collected data is performed with robust software for working with big matrices. Statistically significant differences in eyeblink rate related to media professionalization and editing style are found with the reported experimental procedures.

Using the procedure presented here, the eyeblink rate of 40 participants while watching three different video movies was analyzed. For a comparative analysis on media professionalization, half of the participants were media professionals. They were selected based on the criterion of a minimum 6 years of job experience in making decisions related to media editing and audio-visual cuts. For a comparative analysis on editing styles, the three video stimuli were created with exactly the same ...

Watch this Scientific Journal Video about Using Electroencephalography Measurements and High-quality Video Recording for Analyzing Visual Perception of Media Content at JoVE.com

Using Electroencephalography Measurements and High-quality Video Recording for Analyzing Visual Perception of Media Content

We present detection, acquisition, and analysis of eyeblink rates while watching media content.

This article explores a method to detect differences in visual perception in humans. The method used is based on the psychological (or "cognitive") ...

using-electroencephalography-measurements-high-quality-video

Universidad San Sebastian

Research

JoVE Journal

Behavior

6.8K Views.  Universitat Autónoma de Barcelona. We present detection, acquisition, and analysis of eyeblink rates while watching media content.

Video: Using Electroencephalography Measurements and High-quality Video Recording for Analyzing Visual Perception of Media Content

Functional Mapping with Simultaneous MEG and EEG

We use magnetoencephalography (MEG) and electroencephalography (EEG) to locate and determine the temporal evolution in brain areas involved in the processing of simple sensory stimuli. We will use somatosensory stimuli to locate the hand somatosensory areas, auditory stimuli to locate the auditory cortices, visual stimuli in four quadrants of the visual field to locate the early visual areas. These type of experiments are used for functional mapping in epileptic and brain tumor patients to locate eloquent cortices. In basic neuroscience similar experimental protocols are used to study the orchestration of cortical activity. The acquisition protocol includes quality assurance procedures, subject preparation for the combined MEG/EEG study, and acquisition of evoked-response data with somatosensory, auditory, and visual stimuli. We also demonstrate analysis of the data using the equivalent current dipole model and cortically-constrained minimum-norm estimates. Anatomical MRI data are employed in the analysis for visualization and for deriving boundaries of tissue boundaries for forward modeling and cortical location and orientation constraints for the minimum-norm estimates.

We use magnetoencephalography (MEG) and electroencephalography (EEG) to map brain areas involved in the processing of simple sensory stimuli.

We use magnetoencephalography (MEG) and electroencephalography (EEG) to locate and determine the temporal evolution in brain areas involved in the ...

Neuroscience

Cortical Source Analysis of High-Density EEG Recordings in Children

EEG is traditionally described as a neuroimaging technique with high temporal and low spatial resolution. Recent advances in biophysical modelling and signal processing make it possible to exploit information from other imaging modalities like structural MRI that provide high spatial resolution to overcome this constraint1. This is especially useful for investigations that require high resolution in the temporal as well as spatial domain. In addition, due to the easy application and low cost of EEG recordings, EEG is often the method of choice when working with populations, such as young children, that do not tolerate functional MRI scans well. However, in order to investigate which neural substrates are involved, anatomical information from structural MRI is still needed. Most EEG analysis packages work with standard head models that are based on adult anatomy. The accuracy of these models when used for children is limited2, because the composition and spatial configuration of head tissues changes dramatically over development3.&#160;
In the present paper, we provide an overview of our recent work in utilizing head models based on individual structural MRI scans or age specific head models to reconstruct the cortical generators of high density EEG. This article describes how&#160;EEG recordings are acquired, processed, and analyzed with pediatric populations at the London Baby Lab, including laboratory setup, task design, EEG preprocessing, MRI processing, and EEG channel level and source analysis.&#160;

In recent years, there has been increasing interest in estimating the cortical sources of scalp measured electrical activity for cognitive neuroscience experiments. This article describes how high density EEG is acquired and how recordings are processed for cortical source estimation in children from the age of 2 years at the London Baby Lab.

EEG is traditionally described as a neuroimaging technique with high temporal and low spatial resolution. Recent advances in biophysical modelling and ...

Combining Computer Game-Based Behavioural Experiments With High-Density EEG and Infrared Gaze Tracking

Experimental paradigms are valuable insofar as the timing and other parameters of their stimuli are well specified and controlled, and insofar as they yield data relevant to the cognitive processing that occurs under ecologically valid conditions. These two goals often are at odds, since well controlled stimuli often are too repetitive to sustain subjects' motivation. Studies employing electroencephalography (EEG) are often especially sensitive to this dilemma between ecological validity and experimental control: attaining sufficient signal-to-noise in physiological averages demands large numbers of repeated trials within lengthy recording sessions, limiting the subject pool to individuals with the ability and patience to perform a set task over and over again. This constraint severely limits researchers' ability to investigate younger populations as well as clinical populations associated with heightened anxiety or attentional abnormalities. Even adult, non-clinical subjects may not be able to achieve their typical levels of performance or cognitive engagement: an unmotivated subject for whom an experimental task is little more than a chore is not the same, behaviourally, cognitively, or neurally, as a subject who is intrinsically motivated and engaged with the task. A growing body of literature demonstrates that embedding experiments within video games may provide a way between the horns of this dilemma between experimental control and ecological validity. The narrative of a game provides a more realistic context in which tasks occur, enhancing their ecological validity (Chaytor &amp; Schmitter-Edgecombe, 2003). Moreover, this context provides motivation to complete tasks. In our game, subjects perform various missions to collect resources, fend off pirates, intercept communications or facilitate diplomatic relations. In so doing, they also perform an array of cognitive tasks, including a Posner attention-shifting paradigm (Posner, 1980), a go/no-go test of motor inhibition, a psychophysical motion coherence threshold task, the Embedded Figures Test (Witkin, 1950, 1954) and a theory-of-mind (Wimmer &amp; Perner, 1983) task. The game software automatically registers game stimuli and subjects' actions and responses in a log file, and sends event codes to synchronise with physiological data recorders. Thus the game can be combined with physiological measures such as EEG or fMRI, and with moment-to-moment tracking of gaze. Gaze tracking can verify subjects' compliance with behavioural tasks (e.g. fixation) and overt attention to experimental stimuli, and also physiological arousal as reflected in pupil dilation (Bradley et al., 2008). At great enough sampling frequencies, gaze tracking may also help assess covert attention as reflected in microsaccades - eye movements that are too small to foveate a new object, but are as rapid in onset and have the same relationship between angular distance and peak velocity as do saccades that traverse greater distances. The distribution of directions of microsaccades correlates with the (otherwise) covert direction of attention (Hafed &amp; Clark, 2002).

Procedures for recording high-density EEG and gaze data during computer game-based cognitive tasks are described. Using a video game to present cognitive tasks enhances ecological validity without sacrificing experimental control.

Experimental paradigms are valuable insofar as the timing and other parameters of their stimuli are well specified and controlled, and insofar as they ...

Investigating Social Cognition in Infants and Adults Using Dense Array Electroencephalography (dEEG)

Dense array electroencephalography (dEEG), which provides a non-invasive window for measuring brain activity and a temporal resolution unsurpassed by any other current brain imaging technology1,2, is being used increasingly in the study of social cognitive functioning in infants and adults. While dEEG is enabling researchers to examine brain activity patterns with unprecedented levels of sensitivity, conventional EEG recording systems continue to face certain limitations, including 1) poor spatial resolution and source localization3,4,2) the physical discomfort for test subjects of enduring the individual application of numerous electrodes to the surface of the scalp, and 3) the complexity for researchers of learning to use multiple software packages to collect and process data. Here we present an overview of an established methodology that represents a significant improvement on conventional methodologies for studying EEG in infants and adults. Although several analytical software techniques can be used to establish indirect indices of source localization to improve the spatial resolution of dEEG, the HydroCel Geodesic Sensor Net (HCGSN) by Electrical Geodesics, Inc. (EGI), a dense sensory array that maintains equal distances among adjacent recording electrodes on all surfaces of the scalp, further enhances spatial resolution4,5,6 compared to standard dEEG systems. The sponge-based HCGSN can be applied rapidly and without scalp abrasion, making it ideal for use with adults7,8, children9,10,11, and infants12, in both research and clinical4,5,6,13,14,15 settings. This feature allows for considerable cost and time savings by decreasing the average net application time compared to other dEEG systems. Moreover, the HCGSN includes unified, seamless software applications for all phases of data, greatly simplifying the collection, processing, and analysis of dEEG data.
The HCGSN features a low-profile electrode pedestal, which, when filled with electrolyte solution, creates a sealed microenvironment and an electrode-scalp interface. In all Geodesic dEEG systems, EEG sensors detect changes in voltage originating from the participant's scalp, along with a small amount of electrical noise originating from the room environment. Electrical signals from all sensors of the Geodesic sensor net are received simultaneously by the amplifier, where they are automatically processed, packaged, and sent to the data-acquisition computer (DAC). Once received by the DAC, scalp electrical activity can be isolated from artifacts for analysis using the filtering and artifact detection tools included in the EGI software. Typically, the HCGSN can be used continuously for only up to two hours because the electrolyte solution dries out over time, gradually decreasing the quality of the scalp-electrode interface.
In the Parent-Infant Research Lab at the University of Toronto, we are using dEEG to study social cognitive processes including memory, emotion, goals, intentionality, anticipation, and executive functioning in both adult and infant participants.

Investigating Social Cognition in Infants and Adults Using Dense Array Electroencephalography dEEG

Dense array electroencephalography is being used increasingly to study social cognitive functions in infants and adults. Here we present an established methodology that represents a significant improvement on conventional methodologies for studying EEG in infants and adults.

Dense array electroencephalography (dEEG), which provides a non-invasive window for measuring brain activity and a temporal resolution unsurpassed by any ...

Mapping Cortical Dynamics Using Simultaneous MEG/EEG and Anatomically-constrained Minimum-norm Estimates: an Auditory Attention Example

Magneto- and electroencephalography (MEG/EEG) are neuroimaging techniques that provide a high temporal resolution particularly suitable to investigate the cortical networks involved in dynamical perceptual and cognitive tasks, such as attending to different sounds in a cocktail party. Many past studies have employed data recorded at the sensor level only, i.e., the magnetic fields or the electric potentials recorded outside and on the scalp, and have usually focused on activity that is time-locked to the stimulus presentation. This type of event-related field / potential analysis is particularly useful when there are only a small number of distinct dipolar patterns that can be isolated and identified in space and time. Alternatively, by utilizing anatomical information, these distinct field patterns can be localized as current sources on the cortex. However, for a more sustained response that may not be time-locked to a specific stimulus (e.g., in preparation for listening to one of the two simultaneously presented spoken digits based on the cued auditory feature) or may be distributed across multiple spatial locations unknown a priori, the recruitment of a distributed cortical network may not be adequately captured by using a limited number of focal sources.
Here, we describe a procedure that employs individual anatomical MRI data to establish a relationship between the sensor information and the dipole activation on the cortex through the use of minimum-norm estimates (MNE). This inverse imaging approach provides us a tool for distributed source analysis. For illustrative purposes, we will describe all procedures using FreeSurfer and MNE software, both freely available. We will summarize the MRI sequences and analysis steps required to produce a forward model that enables us to relate the expected field pattern caused by the dipoles distributed on the cortex onto the M/EEG sensors. Next, we will step through the necessary processes that facilitate us in denoising the sensor data from environmental and physiological contaminants. We will then outline the procedure for combining and mapping MEG/EEG sensor data onto the cortical space, thereby producing a family of time-series of cortical dipole activation on the brain surface (or "brain movies") related to each experimental condition. Finally, we will highlight a few statistical techniques that enable us to make scientific inference across a subject population (i.e., perform group-level analysis) based on a common cortical coordinate space.

We use magneto- and electroencephalography (MEG/EEG), combined with anatomical information captured by magnetic resonance imaging (MRI), to map the dynamics of the cortical network associated with auditory attention.

Magneto- and electroencephalography (MEG/EEG) are neuroimaging techniques that provide a high temporal resolution particularly suitable to investigate the ...

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

When considering human neuroimaging data, an appreciation of signal variability represents a fundamental innovation in the way we think about brain signal. Typically, researchers represent the brain's response as the mean across repeated experimental trials and disregard signal fluctuations over time as "noise". However, it is becoming clear that brain signal variability conveys meaningful functional information about neural network dynamics. This article describes the novel method of multiscale entropy (MSE) for quantifying brain signal variability. MSE may be particularly informative of neural network dynamics because it shows timescale dependence and sensitivity to linear and nonlinear dynamics in the data.

Neuroimaging researchers typically consider the brain's response as the mean activity across repeated experimental trials and disregard signal variability over time as "noise". However, it is becoming clear that there is signal in that noise. This article describes the novel method of multiscale entropy for quantifying brain signal variability in the time domain.

When considering human neuroimaging data, an appreciation of signal variability represents a fundamental innovation in the way we think about brain ...

Stimulus-specific Cortical Visual Evoked Potential Morphological Patterns

This paper presents a methodology for the recording and analysis of cortical visual evoked potentials (CVEPs) in response to various visual stimuli using 128-channel high-density electroencephalography (EEG). The specific aim of the described stimuli and analyses is to examine whether it is feasible to replicate previously reported CVEP morphological patterns elicited by an apparent motion stimulus, designed to simultaneously stimulate both ventral and dorsal central visual networks, using object and motion stimuli designed to separately stimulate ventral and dorsal visual cortical networks.&#160; Four visual paradigms are presented: 1. Randomized visual objects with consistent temporal presentation. 2. Randomized visual objects with inconsistent temporal presentation (or jitter).&#160; 3. Visual motion via a radial field of coherent central dot motion without jitter.&#160; 4. Visual motion via a radial field of coherent central dot motion with jitter.&#160; These four paradigms are presented in a pseudo-randomized order for each participant.&#160; Jitter is introduced in order to view how possible anticipatory-related effects may affect the morphology of the object-onset and motion-onset CVEP response.&#160; EEG data analyses are described in detail, including steps of data exportation from and importation to signal processing platforms, bad channel identification&#160;and removal, artifact rejection, averaging, and categorization of average CVEP morphological pattern type based upon latency ranges of component peaks. Representative data show that the methodological approach is indeed sensitive in eliciting differential object-onset and motion-onset CVEP morphological patterns and may, therefore, be useful in addressing the larger research aim. Given the high temporal resolution of EEG and the possible application of high-density EEG in source localization analyses, this protocol is ideal for the investigation of distinct CVEP morphological patterns and the underlying neural mechanisms which generate these differential responses.&#160; &#160; &#160;&#160;

In this paper, we present a protocol to investigate differential cortical visual evoked potential morphological patterns through stimulation of ventral and dorsal networks using high-density EEG. Visual object and motion stimulus paradigms, with and without temporal jitter, are described. Visual evoked potential morphological analyses are also outlined.&#160;&#160;

This paper presents a methodology for the recording and analysis of cortical visual evoked potentials (CVEPs) in response to various visual stimuli using ...

Detecting Pre-Stimulus Source-Level Effects on Object Perception with Magnetoencephalography

Pre-stimulus oscillatory brain activity influences upcoming perception. The characteristics of this pre-stimulus activity can predict whether a near-threshold stimulus will be perceived or not perceived, but can they also predict which one of two competing stimuli with different perceptual contents is perceived? Ambiguous visual stimuli, which can be seen in one of two possible ways at a time, are ideally suited to investigate this question. Magnetoencephalography (MEG) is a neurophysiological measurement technique that records magnetic signals emitted as a result of brain activity. The millisecond temporal resolution of MEG allows for a characterization of oscillatory brain states from as little as 1 second of recorded data. Presenting an empty screen around 1 second prior to the ambiguous stimulus onset therefore provides a time window in which one can investigate whether pre-stimulus oscillatory activity biases the content of upcoming perception, as indicated by participants&#39;&#160;reports. The spatial resolution of MEG is not excellent, but sufficient to localise sources of brain activity at the centimetre scale. Source reconstruction of MEG activity then allows for testing hypotheses about the oscillatory activity of specific regions of interest, as well as the time- and frequency-resolved connectivity between regions of interest. The described protocol enables a better understanding of the influence of spontaneous, ongoing brain activity on visual perception.

This article describes how to set up an experiment that allows detecting pre-stimulus source-level influences on object perception using magnetoencephalography (MEG). It covers stimulus material, experimental design, MEG recording, and data analysis.

Pre-stimulus oscillatory brain activity influences upcoming perception. The characteristics of this pre-stimulus activity can predict whether a ...

Topographical EEG Recordings of Visual Evoked Potentials in Mice using Multichannel Thin-film Electrodes

Visual evoked potentials (VEP) allow the characterization of visual function in preclinical mouse models. Various methods exist to measure VEPs in mice, from non-invasive EEG, subcutaneous single-electrodes, and ECoG to fully invasive intracortical multichannel visual cortex recordings. It can be useful to acquire a global, topographical EEG-level characterization of visual responses previous to local intracortical microelectrode measurements in acute experimental settings. For example, one use case is&#160;to assess global cross-modal changes in VEP topography in deafness models before studying its effects on a local intracortical level. Multichannel epicranial EEG is a robust method to acquire such an overview measure of cortical visual activity.&#160;Multichannel epicranial EEG provides comparable results through a standardized, consistent approach to, for example, identify cross-modal, pathological, or age-related changes in cortical visual function. The current study presents a method to obtain the topographical distribution of flash-evoked VEPs with a 32-channel thin-film EEG electrode array in anesthetized mice. Combined with analysis in the time and frequency domain, this approach allows fast characterization and screening of the topography and basic visual properties of mouse cortical visual function, which can be combined with various acute experimental settings.

The present protocol describes a simple procedure to acquire and analyze the topography of epicranial visual evoked potentials with 32-multichannel thin-film electrodes in the mouse.

Visual evoked potentials (VEP) allow the characterization of visual function in preclinical mouse models. Various methods exist to measure VEPs in mice, ...

A Visuospatial Planning Task Coupled with Eye-Tracker and Electroencephalogram Systems

The planning process, characterized by the capability to formulate an organized plan to reach a goal, is essential for human goal-directed behavior. Since planning is compromised in several neuropsychiatric disorders, the implementation of proper clinical and experimental tests to examine planning is critical. Due to the nature of the deployment of planning, in which several cognitive domains participate, the assessment of planning and the design of behavioral paradigms coupled with neuroimaging methods are current challenges in cognitive neuroscience. A planning task was evaluated in combination with an electroencephalogram (EEG) system and eye movement recordings in 27 healthy adult participants. Planning can be separated into two stages: a mental planning stage in which a sequence of steps is internally represented and an execution stage in which motor action is used to achieve a previously planned goal. Our protocol included a planning task and a control task. The planning task involved solving 36 maze trials, each representing a zoo map. The task had four periods: i) planning, where the subjects were instructed to plan a path to visit the locations of four animals according to a set of rules; ii) maintenance, where the subjects had to retain the planned path in their working memory; iii) execution, where the subjects used eye movements to trace the previously planned path as indicated by the eye-tracker system; and iv) response, where the subjects reported the order of the visited animals. The control task had a similar structure, but the cognitive planning component was removed by modifying the task goal. The spatial and temporal patterns of the EEG revealed that planning induces a gradual and lasting rise in frontal-midline theta activity (FM&#952;) over time. The source of this activity was identified within the prefrontal cortex by source analyses. Our results suggested that the experimental paradigm combining EEG and eye-tracker systems was optimal for evaluating cognitive planning.

The study of cognitive planning combining EEG and eye-tracking systems provides a multimodal approach to investigate the neural mechanisms that mediate cognitive control and goal-directed behavior in humans. Here, we describe a protocol for investigating the role of brain oscillations and eye movements in planning performance.

The planning process, characterized by the capability to formulate an organized plan to reach a goal, is essential for human goal-directed behavior. Since ...