ScholarWorks@중앙대학교

초록

Multimodal machine learning has been widely studied for the development of general intelligence. Recently proposed Perceiver and its variant (Perceiver IO) have shown promising results in addressing diverse types of input modalities with a universal model architecture. However, they have mainly focused on image and text data modalities, and it is unclear whether this kind of universal architecture can also be effective for graph-structured datasets. As the graph data contains topological information, which is lacking in the image and text data, it is non-trivial to devise a universal architecture for graph and other data modalities altogether. In this study, we provide a Graph Perceiver IO (GPIO), a class of Perceiver IO models that addresses graph-structured datasets. We keep the main structure of the GPIO the same as with the Perceiver, which can already handle multiple data modalities, while focusing on how we can extend it to the graph domain. By leveraging positional encoding and output query smoothing, GPIO serves as a general architecture that handles graph-structured data as well as text and image data. Besides, we further propose GPIO+ for the multimodal few-shot classification that incorporates both images and graphs simultaneously. Through extensive experiments covering link prediction, graph classification, node classification, and multimodal text classification, we demonstrate that GPIO and GPIO+ outperform the representative graph neural network baseline models, while requiring lower computational complexity than them. © 2025 Elsevier Ltd

키워드