ETRA: Eye Tracking research & Applications (Conference)

Looking for interesting conferences where to submit the next papers I have found an ACM Conference named ETRA. It is not focused on HCI or IR or other topics about which I usually write, but on all the research concerned to the eye tracking technique, applied to any science.

ETRA will be held in March 2012 in Santa Barbara (CA):
http://www.etra2012.org/

The program of the last year conference, that was held in Austin are available at:
http://etra.cs.uta.fi/program.html

Thanks to this last page I have had news of Yvonne Kammerer, a researcher who is working on search. Her publications can be seen at:
http://www.iwm-kmrc.de/www/en/mitarbeiter/ma.html?dispname=Yvonne+Kammerer&uid=ykammerer

This is a list of those which are related to web search:

Kammerer, Y., & Gerjets, P. (2011). Searching and evaluating information on the WWW: Cognitive processes and user support. In K.-P. L. Vu & R. W. Proctor (Eds.), Handbook of human factors in Web design (2nd ed., pp. 283-302). Boca Raton, FL: CRC Press.

Gerjets, P., Kammerer, Y., & Werner, B. (2011). Measuring spontaneous and instructed evaluation processes during web search: Integrating concurrent thinking-aloud protocols and eye-tracking data. Learning and Instruction, 21, 220-231.

Kammerer, Y., & Beinhauer, W. (2010). Gaze-based Web search: The impact of interface design on search result selection. In C. Morimoto & H. Instance (Eds.), Proceedings of the 2010 Symposium on Eye Tracking Research & Applications ETRA ’10 (pp. 191-194). New York, NY: ACM. [pdf at ACM DL]

Kammerer, Y., & Gerjets, P. (2010). How the interface design influences users’ spontaneous trustworthiness evaluations of Web search results: Comparing a list and a grid interface. In C. Morimoto & H. Instance (Eds.), Proceedings of the 2010 Symposium on Eye Tracking Research & Applications ETRA ’10 (pp. 299-306). New York, NY: ACM. [pdf at ACM DL]

Kammerer, Y., Wollny, E., Gerjets, P., & Scheiter, K. (2009). How authority-related epistemological beliefs and salience of source information influence the evaluation of web search results – An eye tracking study. In N. A. Taatgen, & H. van Rijn (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp. 2158-2163). Austin, TX: Cognitive Science Society.

Kammerer, Y., & Gerjets, P. (2010, May). Objective, subjective, and commercial information: The impact of presentation format on the visual inspection and selection of Web search results. The Scandinavian Workshop on Applied Eye-Tracking (SWAET). Lund, Sweden.

Kammerer, Y., Werner, B., & Gerjets, P. (2008). What evaluation processes are performed during web search? An eye-tracking study. XXIX International Congress of Psychology (ICP). Berlin.

Métricas de usabilidad y UX

Althoug these readings are not exactly for measuring with eye tracking, all of them are focused on measuring usability and user experience:

• Evaluating UX by Measuring Task Efficiency, by Cathleen Zapata, article 686, 13 June 2011,
• Jeff Sauro's blog, http://www.measuringusability.com/blog.php
• Jeff Sauro, A practical guide to measuring usability. (2010)
• Jeff Sauro and James R. Lewis, Quantifying the user experience: Practical statistics for user research (2012) (In Amazon, available for Kindle)
• Jeff Sauro, Excel and R Companion to Quantifying the User Experience: Rapid Answers to over 100 Examples and Exercises (2012) (in Amazon "look inside" option available)

Here you are a list of reading about SUS:

• Jeff Sauro, post about SUS (2011)
• Jeff Sauro, A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices (2011) (in Amazon "look inside" option available)
• Jeff Sauro, SUS Guide and Calculator Package (Excel calculator, 150 euros)

Five second tests:

• Jeff Sauro, 5 seconds usability tests (2010)
• Christine Perfetti, 5-Second Tests: Measuring Your Site's Content Pages (2005)

(To be continued...)

Papers en los que se ha usado el eyetracker de Tobii

La empresa Tobii va recogiendo periódicamente los papers académicos que se publican y para los que se ha utilizado su dispositivo.

La lista contiene casi 300 documentos, algunos en PDF, está en esta dirección:
http://www.diigo.com/list/tobiieyetracking/hci-usability

Some readings about Western and Eastern cognitive styles

Today I have resumed contact with a professor in Zayed University, Dr. Emad Bataineh, who I knew in the EyeTracking UX Conference in London (June 2011). We planned to do a research together and compare users's behavior in SERPs in two countries: Spain and UAE.

Looking for a start point I have found the PhD Thesis of Jolin Adeeb Qutub (director Frederick J. Brigham) who assess that "[...] Recent studies have concluded that Western learners tend to have more analytical perceptual learning style whereas East Asians tend to have more holistic or contextual perceptual learning style (Nisbett & Norenzayan, 2002). Only a limited number of studies have examined cognitive perceptual differences between Middle Eastern and Western learners. This study aimed to help close this research gap by exploring cognitive perceptual differences among three groups who come from different cultural backgrounds: Saudi Arabians, immigrants living in the United States, and Americans [...] ".

I have collected some references from the thesis:

• Chua, H., Boland, J. and Nisbett, R. (2005). Cultural variation in eye movements during scene perception. Proceedings of the National Academy of Sciences, 102, 12629-12633
• Masuda, T., & Nisbett. R. (2001). Attending holistically versus analytically: Comparing the context sensitivity of Japanese and Americans. Journal of Personality and Social Psychology, 81, 922–934.
• Nisbett, R. (2004). The Geography of Thought: How Asians and Westerners Think Differently…and Why. New York: Free Press.
• Nisbett, R., & Miyamoto, Y. (2005). The influence of culture: Holistic versus analytic perception. Trends in Cognitive Science, 9, 467-473
• Nisbett, R.E., & Norenzayan, A. (2002). Culture and cognition. In H. Pashler & D. L. Medin (Eds.), Stevens Handbook of Experimental Psychology : Cognition (3d Ed., Vol. 2) (pp. 561-597). New York: John Wiley & Sons. [PDF]
• Nisbett, R.E., Peng, K., Choi, I., & Norenzayan, A. (2001). Culture and systems of thought: Holistic versus analytic cognition. Psychological Review, 108, 291-310. [PDF]
• Stevens, Handbook of Experimental Psychology, Vol 2. (3rd ed., pp. 561-597), New York: John Wiley and Sons, Inc.
• Norenzayan, E., Smith, E., Beom, J., & Nisbett, R. (2002). Cultural preferences for formal versus intuitive reasoning. Cognitive Science, 26, 653–684.

And one more that I might need for this and other studies:

• Hink, D. E., Wiersma, W., & Jurs, S. G., (1988). Applied Statistics For The Behavioral Sciences (2nd ed). Boston: Houghton Mifflin

If this study becomes a reality, I think the results can be very interesting. Let's see!

Are you starting to read about Eyetracking in UX?

Basic readings:

Bojko, Aga, http://www.rosenfeldmedia.com/books/eye-tracking/blog/ Blog]

Bojko, Aga. & Stephenson, A. It's All in the Eye of the User: How eye tracking can help answer usability questions. User Experience, 4:1, 2005, http://www.usercentric.com/publications/2005/03/18/its-all-eye-user-how-eye-tracking-can-help-answer-usability-questions

Bojko, Aga. Using Eye Tracking to Compare Web Page Designs: A Case Study. Journal of Usability Studies, 1:3, 2006, http://www.upassoc.org/upa_publications/jus/2006_may/bojko_eye_tracking.pdf

Duchowski, A. T. A Breadth-First Survey of Eye Tracking Applications, 'Behavior Research Methods, Instruments, & Computers (BRMIC), 34:4, 2002, 455–470.

Duchowsky, Andrew. Eye Tracking Methodology: Theory and Practice (2nd ed). Springer, 2007. Part 3 “Eye Tracking Methodology", http://andrewd.ces.clemson.edu/book/toc.pdf

Goldberg, J.H.; Wichansky, A.M. Eye tracking in usability evaluation: A Practitioner's Guide. En: Hyona, J., Radach, R., Duebel, H (Eds.). The mind's eye: cognitive and applied aspects of eye movement research. Boston, North-Holland / Elesevier, 2003, 573-605.

Hassan Montero, Yusef; Herrero Solana, Víctor. Eye-Tracking en Interacción Persona-Ordenador, No Solo Usabilidad, 2007, http://www.nosolousabilidad.com/articulos/eye-tracking.htm

Nielsen, Jakob; Pernice, Kara. Eyetracking web usability. New Riders, 2010, http://www.useit.com/eyetracking/

Pernice, Kara; Nielsen, Jakob. Eyetracking Methodology: 65 Guidelines for How to Conduct and Evaluate Usability Studies Using Eyetracking, http://www.useit.com/eyetracking/methodology/eyetracking-methodology.pdf (16 Mb)

Poole, A.; Ball, L.J. (2004). Eye Tracking in Human-Computer Interaction and Usability Research: Current Status and Future Prospects. In Ghaoui, Claude (Ed.). Encyclopedia of Human Computer Interaction. Idea Group, http://www.alexpoole.info/academic/Poole&Ball%20EyeTracking.pdf

Spool, J. Eyetracking: Worth The Expense?. UIE Brain Sparks Blog, 2006, http://www.uie.com/brainsparks/2006/06/13/eyetracking-worth-the-expense

Tullis, Tom; Albert, Bill. Measuring the user experience. Morgan Kaufmann, 2008, http://www.measuringux.com/

Usolab (2007). La aportación del eyetracking en el sector de usabilidad, 2007, http://www.usolab.com/articulos/eyetracking-usabilidad-comunicacion.php

What accuracy and precision means for eye trackers

Brilliant explanation from Aga -she again, sorry I admire her.

Accuracy and precision are not synonyms words. She explains both at http://www.rosenfeldmedia.com/books/eye-tracking/blog/the_most_precise_or_most_accur/:

"The accuracy of an eye tracker is the average difference between what the eye tracker recorded as the gaze position and what the gaze position actually was. We want this offset to be as small as possible but it is obviously unrealistic to expect it to be equal to zero. Accuracy is measured in degrees of visual angle. Typical accuracy values fall in a range between 0.5 and 1 degree."

"Precision (aka "spatial resolution") is a measure of how well the eye tracker is able to reliably reproduce a measurement. Ideally, if the eye is in the same exact location in two successive measurements, the eye tracker should report the two locations as identical. That would be perfect precision. In reality, precision values of currently available eye trackers range from 0.01 to 1 degree. These values are calculated as the root mean square of the distance (in degrees of visual angle) between successive samples.

Eyetracker simulation: EyeQuant, Feng-GUI, and Attention Wizard

These three software try to emulate the ocular behavior considering the elements showed on the screen.

Despite promising good results, some tests show that they are very far away from the reality. Aga shares their own results testing Ebay:
http://www.rosenfeldmedia.com/books/eye-tracking/blog/participant-free_eye_tracking/

A main error would be that "For example, the simulations predict a lot of attention on images (including advertising), whereas the study participants barely even looked at many of those elements"

Behavioral and Physiological Metrics

This Chapter 7. of Tullis and Albert, Measuring the user experience, it worth reading although it will not easy to apply in UX tests. A list of the metrics:

1. Just observing and coding behaviors taking notes in a form or in a open format, verbal and non-verbal ones (see form on page 170)
1.1. Verbal comments: positive/negative comments, doubts, questions...
1.2. Non-verbal behaviors to analyze: facial expressions and body language

2. Using specific devices
2.1. Facial expressions
- Video recording analysis (time consuming), and classify them, ex. FACSAID 2002
- Electromyogram sensors (EMG), specially for forehead and cheeks
2.2. Body language to measure stress-frustration and engagement
- Mouse gripping
- Chair pressure
- Skin conductance
- Heart rate
- Pupil dilation
2.3. Eye tracking
- Proportion of users looking at a specific element or region (AOI*)
- Time spent looking at a specific element or region (AOI)
- Time to notice a specific element or region
- Scan-paths and length of eye movements

*Tips about AOIs:
- do not leave any space undefined
- analyze the time as a percentage of total time spent on the page, not as an absolute amount of time, since it can vary widely between participants

Ventajas e inconvenientes de usar ET en usabilidad

Leo a Ania Rodríguez en el blog de Aga:

Pros:
• Augment usability findings
• Understand key performance indicators for visual elements on page such as photography which is not available with web analytic data
• Understand what elements users gaze/ dwell on during tasks
• Provides visual representations for hard to express behaviors
• Providing additional insights to Moderator so that they can ask more probing questing
• Observers can see what participants are fixating/dwelling on real time while users are performing the task
• Allows for compelling visualizations of usability problems for executive presentations
• Client teams and executive membership perceive this method to be cutting-edge

Cons:
• Just because a user fixates on an area does not mean they saw it
• Higher severity issues can be found without the use of eye tracking
• Misinterpretations of data likely if not adequately contextualized (e.g., heat maps)
• Interaction between moderator and participant changes
• Eye tracking equipment can be difficult to learn and use
• Eye tracking equipment is costly
• Dynamic sites are difficult to analyze (for Ajax and Flash heavy sites)
• Eye tracking is subject to technical problems

Ventajas del thinking aloud regresivo frente al concurrente

Las técnicas de thinking aloud combinadas con un test con ET pueden ser:

• CTA (Concurrent Think-Aloud): el usuario habla mientras se le graba con el ET
• RTA (Regresive Think-Aloud): el usuario, una vez grabada la sesión, ve el vídeo con las fijaciones marcadas (gaze-cued RTA) o sin ellas marcadas (video-cued RTA) y lo comenta

Leo en el blog de Aga Bojko:

Studies have shown that gaze-cued RTA provides more feedback than Concurrent Think-Aloud protocol (CTA) in terms of the number of words spoken and comments made by participants [source (pdf)]. Researchers have also noted that gaze-cued RTA feedback tends to be of a higher quality than feedback obtained through CTA, resulting in the discovery of more usability issues. However, when gaze-cued RTA is compared to video-cued RTA (with no gaze overlay), the differences in user feedback are minimal. [source (pdf: pp 15 - 19)].

A quién no perder de vista

Hay algunos autores que están trabajando actualmente con ET y que merece la pena seguirlos:

Aga Bojko, que está escribiendo el libro"Eye Tracking the User Experience" para Rosenfeld Media y se publicará en 2012 (espero!), va publicando posts sobre sus reflexiones a la vez que escribe. Además es profesora de Usability Evaluation Methods en la universidad DePaul.

Andrew T. Duchowsky, autor del libro "Eye Tracking Methodology: Theory and Practice" (2007) y de varios artículos. Del libro puede servirme en especial el apartado sobre "Experimental Design" en "Part III Eye Tracking Methodology" (ver tabla de contenidos). Otra publicación para leer de este autor, para tener una idea general de la investigación con ET:

• A Breadth-First Survey of Eye Tracking Applications (2002)
  This paper reports on eye tracking methodologies spanning
  three disciplines: neuroscience, psychology, and computer
  science, with brief remarks about industrial engineering
  and marketing.

Lecturas generales para comenzar desde el principio

Son varios cientos los documentos que se han escrito sobre el uso de técnicas de seguimiento de la mirada, muchos de ellos son resultados de investigación obtenidos con la aplicación de un eyetracker. Antes de abordar estudios concretos, estoy leyendo artículos y libros más generales, y otros que tengo anotados y pongo aquí, en mi tablón de anuncios, nada formal.

Nielsen, Pernice. Eyetacking web usability (2010) LIBRO
- Específico para el análisis de la usabilidad en sitios web
- Trata de forma muy ligera cómo funcionan los eyetrackers hoy en día
- Cuenta cómo han montado las sesiones de tests con usuarios
- Se centra en estos aspectos para el análisis: layout de la página web, navegación, elementos concretos como el login, el contacto, el selector de idioma, los logos, el carro de la compra, y dedica un capítulo a las imágenes y otro a la publicidad...
- Termina comentando los distintos comportamientos observados

Poole, Linden (2004). Eye tracking in Human-Computer Interaction aned Usability Research: Current status and future prospects. ARTÍCULO
- Específico para HCI
- Hace un vaciado de las métricas que se han utilizado en diversos estudios realizados con ET y las organiza en función de si se tratan de métricas basadas en las fijaciones, en las sacadas y en los scanpath
- Para cada métrica indica cómo intepretaron sus autores los resultados obtenidos
- Artículo de gran utilidad para aplicar métricas que otros ya han usado, pero no son 100% replicables las interpretaciones porque depende de qué se mide.

Primeros resultados

Hasta ahora había leído mucho -muchísimo- sobre estudios acerca del comportamiento visual de los usuarios en las páginas de resultados de los buscadores. De este tema salieron dos papers, el de EPI (Marcos, González-Caro 2010) y el de Hong Kong (González-Caro, Marcos 2011):

• Marcos, Mari-Carmen; González-Caro, Cristina. El comportamiento de los usuarios en la página de resultados de los buscadores: un estudio basado en la técnica de eye tracking. El Profesional de la Información, 19:4, julio-agosto 2010, 348-358. (2.3 Mb). [URL alternativa]

• González-Caro, Cristina; Marcos, Mari-Carmen. “Different Users and Intents: An Eye-tracking Analysis of Web Search”. In: King, Irwin; Nejdl, Wolfgang; Li, Hang. WSDM'11 Fourth ACM International Conference on Web Search and Data Mining (Kowloon, Hong Kong, February 09 - 12, 2011). New York: ACM, 2011.

También hemos enviado un paper con más resultados a Interact 2011. Estamos esperando la respuesta (cruzo los dedos)

¿Era necesario este blog?

Depende... para la humanidad, sin duda no. Para mis amigos y familia, tampoco, de hecho no sabrán que existe. Para mis contactos, seguidores, necesario no, pero posiblemente interesante ojearlo de ciento a viento. Para mi, creo que sí.

Si hasta ahora he estado dando la lata que si el eyetracker por aquí, el eyetracker por allá, ahora publico en EPI, luego me voy a Hong Kong... a partir de este mes de mayo de 2011 voy a darla mucho más porque tengo un eyetracker, sí, lo tengo, quiero decir que no se lo llevan dentro de un mes ni de dos, que es para mi -bueno, para nosotros, para la gente del grupo-

Hemos comprado un eyetracker Tobii T60 y una licencia del software. Aún no lo hemos visto en funcionamiento en espera de tener un portátil con puerto wirefire. Y como es de esperar, mi investigación va a estar muy vinculada a esta herramienta. Ya iré contando qué sale de todo esto...

De momento abro este blog para mi, para usarlo como tablón donde anotar lo que leo, lo que encuentro, lo que pienso. Podría tener un Google Docs y ya está, pero aquí me lo organizo un poco mejor con los tags. Manos a la obra, y...

... bienvenido Ms. Tracker!