Wei Wang

Research Fellow

“We feel and know that we are eternal”

Baruch Spinoza

Easy Read Mode Off

Development of an Adaptive User Support System Based on Multimodal Large Language Models

Published:

1

Abstract

This paper presents a support system that uses multimodal large language models.

Some software is difficult to use. People may get stuck, feel frustrated, or not know what to do next.

The system looks at the user interface and helps create support messages that match what the user is trying to do.

The goal is to make software help more useful, more contextual, and easier to understand.

Open paper
Question Method Finding Contribution
2

Talks

Poster Presentations
Poster Presentations

🔍 Abstract

As software systems become more complex, some users find it challenging to use these tools efficiently, leading to frustration and decreased productivity. We tackle the shortcomings of conventional user support mechanisms in software and aim to create and assess a user support system that integrates Multimodal Large Language Models (MLLMs) for producing support messages. Our system initially segments the user interface to serve as a reference for selection and requests users to specify their preferences for support messages. Following this, the system creates personalised user support messages for each individual. We propose that user support systems enhanced with MLLMs can provide more efficient and bespoke assistance compared to conventional methods.

🎤 Talks

Poster Presentations

📝 Citation

Wei Wang, Lin Li, Shavindra Wickramathilaka, John Grundy, Hourieh Khalajzadeh, Humphrey O. Obie, and Anuradha Madugalla. "Development of an Adaptive User Support System Based on Multimodal Large Language Models." In 2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), pp. 344-347. IEEE, 2024. UPV