Trident Transformer for Light Field Image Super-Resolution

Z Wang, Y Lu, S Wang, W Xia, P Xia… - 2024 IEEE International …, 2024 - ieeexplore.ieee.org
Z Wang, Y Lu, S Wang, W Xia, P Xia, W Wang
2024 IEEE International Conference on Multimedia and Expo (ICME), 2024ieeexplore.ieee.org
Light Field (LF) image Super-Resolution (SR) requires leveraging the spatial-angular
relationship to super-resolve low-resolution LF images into corresponding high-resolution
counterparts. Recently, many Transformer-based methods have been proposed for LFSR.
However, these methods struggle to recover sharp edges and intricate structures due to the
Self-Attention (SA) mechanism's intrinsic defects of capturing high-frequency information.
Additionally, most of them fail to excavate the global spatial-angular information across all …
Light Field (LF) image Super-Resolution (SR) requires leveraging the spatial-angular relationship to super-resolve low-resolution LF images into corresponding high-resolution counterparts. Recently, many Transformer-based methods have been proposed for LFSR. However, these methods struggle to recover sharp edges and intricate structures due to the Self-Attention (SA) mechanism’s intrinsic defects of capturing high-frequency information. Additionally, most of them fail to excavate the global spatial-angular information across all views hindered by the expensive computational cost of SA on 4D LF data. To tackle these issues, we introduce Trident Transformer (TriFormer) with three parallel branches: the high-frequency branch, which utilizes convolution and max-pooling for recovering fine-grained textures; the low-frequency branch, which adopts vanilla SA to preserve the low-frequency component; and the interactive-frequency branch, which interacts the frequency information and enhances full-frequency feature, aiding in capturing global information across all angular views. A progressive feature fusion approach is then applied to integrate all distinct information. Experimental results demonstrate our TriFormer’s superiority over leading LFSR methods on five benchmarks, while maintaining a compact model size and computational efficiency. The code is publicly available at https://github.com/wziqi/TriFormer.
ieeexplore.ieee.org
Showing the best result for this search. See all results