Speaker
Description
Abstract. Mixed reality (MR) is playing an increasingly important role in teaching and learning natural sciences, as it can support learners especially during experimentation. We investigate the support with MR in experimental problem solving and use the probands' gaze data for a more detailed analysis. It is shown that experimental problem-solving success increases with the support of MR, especially for novices, and that this is accompanied by a significant shift of visual attention. Furthermore, it can be shown that novices can be supported with MR in such a way that they proceed more like experts during experimental problem solving.
1 Introduction
In recent years, mixed reality (MR) has become increasingly popular in science education [1, 2], as it opens up new opportunities that would otherwise not be possible. In physics education, MR can support experimentation by simultaneously displaying additional information related to the experiment and actual measurement data. The most comprehensive possibilities are offered by MR Head-Mounted Displays (HMDs) as they can not only superimpose virtual elements, but rather integrate them into real space. Another advantage of HMD is to be hands-free during experimentation as compared to tablet-based augmented reality. Recent studies have shown that the use of MR has a positive impact on conceptual knowledge as well procedural tasks [3–6].
Current HMDs are equipped with eye-tracking technology, providing built-in possibilities to evaluate MR applications based on visual attention and gaze patterns. Eye-tracking based evaluations are often used for usability evaluation studies in the context of human-computer interaction, but also in physics education in relation to problem solving or the use of multiple representations in reasoning [7–9].
The approach to experimental problem solving often differs completely between novices and experts. This is partly due to prior procedural knowledge in scientific work, but to a large amount it is also due to prior subject specific knowledge and the use of visual and mental representations [10]. Experts generally take their time in the beginning to form representations, while novices tend to rush through this phase and usually use more naive representations. MR can assist with experiment-specific representations to enhance prior knowledge and provide novices with similar opportunities as experts for subsequent problem solving.
Fig. 1 (a) Experimental setup and (b) MR learning environment
2 Experiment and Evaluation
For our study, we have chosen an electrical circuit with a certain complexity, so that neither experts nor novices will know how to solve the problem right away (s. Fig 1a). Participants are asked to first understand the circuit and then to maximize the time between the lighting of two lamps with additional cicuit modules. Based on the approaches of experts and novices, particularly their eye-tracking data, an MR application was created to best support the novices in problem solving on the experiment. In the MR learning environment (MRLE), representations such as the electron flow and the capacitor charge are visualized (s. Fig 1b). In the second part of our study, the experiment was performed by additional participants with the support of this MRLE.
For the study the participants were divided into two groups, one with high prior knowledge and one with low prior knowledge, on the basis of a subject-specific questionnaire. Subsequently, eye-tracking data was recorded during the experimental problem-solving process and additionally, self-reported cognitive load was collected questionnaire-based. Finally, semi-structured interviews were conducted. In addition, problem solving success was analyzed based on the experimental outcome.
The analysis shows that both novices and experts achieved a higher problem-solving success in the MRLE than the subjects without using MR. The visual attention of experts and noivces both with and without MR were analyzed and compared quantitatively to each other for selected areas of interest. It is shown that the improvement in problem-solving success using the MRLE is accompanied by a significant shift in visual attention for novices.
3 Conclusion and Outlook
The MRLE has resulted in novices applying expert-like strategies in problem solving based on their gaze data. Therefore, we propose to use MRLE supportively in experimental problem solving to direct visual attention in ways that enable novices to be more successful. Furthermore, eye-tracking data should be analyzed to gain insights into the effects of design considerations and visualisations, and to incorporate the results into the further development of MRLE.
References
[1] O. Bodensiek, D. Sonntag, N. Wendorff, G. Albuquerque, and M. Magnor, “Augmenting the fine beam tube: From hybrid measurements to magnetic field visualization,” The Physics Teacher, vol. 57, no. 4, pp. 262–263, 2019, doi: 10.1119/1.5095388.
[2] M. P. Strzys et al., “Augmenting the thermal flux experiment: A mixed reality approach with the HoloLens,” The Physics Teacher, vol. 55, no. 6, pp. 376–377, 2017, doi: 10.1119/1.4999739.
[3] B. Volmer et al., “A Comparison of Predictive Spatial Augmented Reality Cues for Procedural Tasks,” IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 11, pp. 2846–2856, 2018, doi: 10.1109/TVCG.2018.2868587.
[4] G. Westerfield, A. Mitrovic, and M. Billinghurst, “Intelligent Augmented Reality Training for Assembly Tasks,” in Artificial Intelligence in Education, 2013, pp. 542–551.
[5] M. P. Strzys et al., “Physics holo. lab learning experience: using smartglasses for augmented reality labwork to foster the concepts of heat conduction,” European Journal of Physics, vol. 39, no. 3, p. 35703, 2018, doi: 10.1088/1361-6404/aaa8fb.
[6] D. Sonntag, G. Albuquerque, M. Magnor, and O. Bodensiek, “Hybrid learning environments by data-driven augmented reality,” Procedia Manufacturing, vol. 31, pp. 32–37, 2019, doi: 10.1016/j.promfg.2019.03.006.
[7] C.-J. Wu and C.-Y. Liu, “Eye-movement study of high- and low-prior-knowledge students’ scientific argumentations with multiple representations,” Phys. Rev. Phys. Educ. Res., vol. 17, no. 1, p. 10125, 2021, doi: 10.1103/PhysRevPhysEducRes.17.010125.
[8] B. Ibrahim and L. Ding, “Sequential and simultaneous synthesis problem solving: A comparison of students’ gaze transitions,” Phys. Rev. Phys. Educ. Res., vol. 17, no. 1, p. 10126, 2021, doi: 10.1103/PhysRevPhysEducRes.17.010126.
[9] N. A. Rappa, S. Ledger, T. Teo, K. Wai Wong, B. Power, and B. Hilliard, “The use of eye tracking technology to explore learning and performance within virtual reality and mixed reality settings: a scoping review,” Interactive Learning Environments, pp. 1–13, 2019, doi: 10.1080/10494820.2019.1702560.
[10] M. T. H. Chi, P. J. Feltovich, and R. Glaser, “Categorization and representation of physics problems by experts and novices,” Cognitive science, vol. 5, no. 2, pp. 121–152, 1981, doi: 10.1207/s15516709cog0502_2.
