Speaker
Description
In large-scale scientific facilities such as the High Energy Photon Source (HEPS), users encompass diverse disciplines and must handle varied data processing tasks within a highly heterogeneous software ecosystem, leading to steep learning curves and substantial support demands. To address these challenges, we propose an AI-based software interaction method that reduces researchers’ operational burden through natural language dialogue, using a typical data preprocessing workflow as an example.
The system comprises three core modules: (1) an Intelligent Interaction Parsing Layer that uses large language models to translate natural language commands into structured operational semantics; (2) a Visual Perception and Mapping Layer that employs real-time interface understanding to identify software states and UI elements, mapping semantics to precise control actions; and (3) a Task Execution and Service Layer that invokes standardized backend methods to perform the specified data preprocessing.
This approach provides real-time visual feedback via a virtual cursor, establishing a transparent translation of language commands into interface responses. It not only lowers users’ cognitive load when operating complex scientific software but also reduces the development and support burden on software teams.
| I read the instructions above | Yes |
|---|