TopFusion: Using Topological Feature Space for Fusion and Imputation in Multi-Modal Data

A Myers, H Kvinge, T Emerson - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
Proceedings of the IEEE/CVF Conference on Computer Vision and …, 2023openaccess.thecvf.com
We present a novel multi-modal data fusion technique using topological features. The
method, TopFusion, leverages the flexibility of topological data analysis tools (namely
persistent homology and persistence images) to map multi-modal datasets into a common
feature space by forming a new multi-channel persistence image. Each channel in the
image is representative of a view of the data from a modality-dependent filtration. We
demonstrate that the topological perspective we take allows for more effective data …
Abstract
We present a novel multi-modal data fusion technique using topological features. The method, TopFusion, leverages the flexibility of topological data analysis tools (namely persistent homology and persistence images) to map multi-modal datasets into a common feature space by forming a new multi-channel persistence image. Each channel in the image is representative of a view of the data from a modality-dependent filtration. We demonstrate that the topological perspective we take allows for more effective data reconstruction, ie imputation. In particular, by performing imputation in topological feature space we are able to outperform the same imputation techniques applied to raw data or alternatively derived features. We show that TopFusion representations can be used as input to downstream deep learning-based computer vision models and doing so achieves comparable performance to other fusion methods for classification on two multi-modal datasets.
openaccess.thecvf.com
Showing the best result for this search. See all results