research-article

Feature Pyramid Hashing

Authors:

Jian YinAuthors Info & Claims

ICMR '19: Proceedings of the 2019 on International Conference on Multimedia Retrieval

Pages 114 - 122

https://doi.org/10.1145/3323873.3325015

Published: 05 June 2019 Publication History

Abstract

In recent years, deep-networks-based hashing has become a leading approach for large-scale image retrieval. Most deep hashing approaches use the high layer to extract the powerful semantic representations. However, these methods have limited ability for fine-grained image retrieval because the semantic features extracted from the high layer are difficult in capturing the subtle differences. To this end, we propose a novel two-pyramid hashing architecture to learn both the semantic information and the subtle appearance details for fine-grained image search. Inspired by the feature pyramids of convolutional neural network, avertical pyramid is proposed to capture the high-layer features and ahorizontal pyramid combines multiple low-layer features with structural information to capture the subtle differences. To fuse the low-level features, a novel combination strategy, called consensus fusion, is proposed to capture all subtle information from several low-layers for finer retrieval. Extensive evaluation on two fine-grained datasets CUB-200-2011 and Stanford Dogs demonstrate that the proposed method achieves significant performance compared with the state-of-art baselines.

References

[1]

Yue Cao, Mingsheng Long, Bin Liu, Jianmin Wang, and MOE KLiss. 2018. Deep Cauchy Hashing for Hamming Space Retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1229--1237.

[2]

Zhangjie Cao, Mingsheng Long, Jianmin Wang, and S Yu Philip. 2017. HashNet: Deep Learning to Hash by Continuation. In Proceedings of the IEEE international conference on computer vision. 5609--5618.

[3]

Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. 2018. Cascaded pyramid network for multi-person pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7103--7112.

[4]

Aristides Gionis, Piotr Indyk, Rajeev Motwani, et al. 1999. Similarity search in high dimensions via hashing. In Proceedings of the International Conference on Very Large Data Bases, Vol. 99. 518--529.

Digital Library

[5]

Yunchao Gong, Svetlana Lazebnik, Albert Gordo, and Florent Perronnin. 2013. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, 12 (2013), 2916--2929.

Digital Library

[6]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[7]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[8]

Sheng Jin. 2018. Deep Saliency Hashing. arXiv preprint arXiv:1807.01459 (2018).

[9]

Tao Kong, Fuchun Sun, Chuanqi Tan, Huaping Liu, and Wenbing Huang. 2018. Deep feature pyramid reconfiguration for object detection. In Proceedings of the European Conference on Computer Vision. 169--185.

Digital Library

[10]

Brian Kulis and Trevor Darrell. 2009. Learning to hash with binary reconstructive embeddings. In Advances in neural information processing systems. 1042--1050.

Digital Library

[11]

Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3270--3278.

[12]

Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. 2015. Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855 (2015).

Digital Library

[13]

Wu Jun Li, Sheng Wang, and Wang Cheng Kang. 2016. Feature learning based deep supervised hashing with pairwise labels. In Proceedings of the International Joint Conference on Artificial Intelligence. 1711--1717.

Digital Library

[14]

Guosheng Lin, Chunhua Shen, David Suter, and Anton Van Den Hengel. 2013. A general two-step approach to learning-based hashing. In Proceedings of the IEEE international conference on computer vision. 2552--2559.

Digital Library

[15]

Kevin Lin, Jiwen Lu, Chu-Song Chen, and Jie Zhou. 2016. Learning compact binary descriptors with unsupervised deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1183--1192.

[16]

Kevin Lin, Fan Yang, Qiaosong Wang, and Robinson Piramuthu. 2018. Adversarial Learning for Fine-grained Image Search. arXiv preprint arXiv:1807.02247 (2018).

[17]

Tsung-Yi Lin, Piotr Doll�r, Ross B Girshick, Kaiming He, Bharath Hariharan, and Serge J Belongie. 2017. Feature Pyramid Networks for Object Detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, Vol. 1. 4.

[18]

Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. 2016. Deep supervised hashing for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2064--2072.

[19]

Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. 2012. Supervised hashing with kernels. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, 2074--2081.

Digital Library

[20]

Wei Liu, Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. 2011. Hashing with graphs. In Proceedings of the 28th international conference on machine learning (ICML-11). 1--8.

Digital Library

[21]

Mohammad Norouzi and David M Blei. 2011. Minimal loss hashing for compact binary codes. In Proceedings of the 28th international conference on machine learning (ICML-11). Citeseer, 353--360.

Digital Library

[22]

Adam Paszke, Sam Gross, Soumith Chintala, and Gregory Chanan. 2017. PyTorch.

[23]

Zhaofan Qiu, Yingwei Pan, Ting Yao, and Tao Mei. 2017. Deep semantic hashing with generative adversarial networks. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 225--234.

Digital Library

[24]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.

[25]

Ruslan Salakhutdinov and Geoff Hinton. 2007. Learning a nonlinear embedding by preserving class neighbourhood structure. In Artificial Intelligence and Statistics. 412--419.

[26]

Fumin Shen, Yan Xu, Li Liu, Yang Yang, Zi Huang, and Heng Tao Shen. 2018. Unsupervised deep hashing with similarity-adaptive and discrete optimization. IEEE transactions on pattern analysis and machine intelligence (2018).

Digital Library

[27]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[28]

Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. 2010. Semi-supervised hashing for scalable image retrieval. (2010).

[29]

Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. 2010. Sequential projection learning for hashing with compact codes. In Proceedings of the 27th international conference on machine learning (ICML-10). 1127--1134.

Digital Library

[30]

Jingdong Wang, Ting Zhang, Nicu Sebe, Heng Tao Shen, et al. 2018. A survey on learning to hash. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, 4 (2018), 769--790.

[31]

Qifan Wang, Zhiwei Zhang, and Luo Si. 2015. Ranking Preserving Hashing for Fast Similarity Search. In Proceedings of the International Joint Conference on Artificial Intelligence. 3911--3917.

Digital Library

[32]

Yair Weiss, Antonio Torralba, and Rob Fergus. 2009. Spectral hashing. In Advances in neural information processing systems. 1753--1760.

Digital Library

[33]

Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. 2010. Caltech-UCSD birds 200. (2010).

[34]

Chenxia Wu, Jianke Zhu, Deng Cai, Chun Chen, and Jiajun Bu. 2013. Semi-supervised nonlinear hashing using bootstrap sequential projection learning. IEEE Transactions on Knowledge and Data Engineering, Vol. 25, 6 (2013), 1380--1393.

Digital Library

[35]

Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. 2014. Supervised Hashing for Image Retrieval via Image Representation Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 1. 2156--2162.

Digital Library

[36]

Wei Yu, Xiaoshuai Sun, Kuiyuan Yang, Yong Rui, and Hongxun Yao. 2018. Hierarchical semantic image matching using CNN feature pyramid. Computer Vision and Image Understanding, Vol. 169 (2018), 40--51.

[37]

Gong Yunchao, S Lazebnik, A Gordo, and F Perronnin. 2013. Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE transactions on pattern analysis and machine intelligence, Vol. 35, 12 (2013), 2916--2929.

Digital Library

[38]

Xi Zhang, Hanjiang Lai, and Jiashi Feng. 2018. Attention-Aware Deep Adversarial Hashing for Cross-Modal Retrieval. In Proceedings of the European Conference on Computer Vision. 614--629.

Digital Library

[39]

Wanqing Zhao, Hangzai Luo, Jinye Peng, and Jianping Fan. 2017. Spatial pyramid deep hashing for large-scale image retrieval. Neurocomputing, Vol. 243 (2017), 166--173.

Digital Library

[40]

Bohan Zhuang, Guosheng Lin, Chunhua Shen, and Ian Reid. 2016. Fast training of triplet-based deep binary embedding networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5955--5964.

Cited By

Redaoui ABelalia ABelloulata K(2024)Deep Supervised Hashing by Fusing Multiscale Deep Features for Image RetrievalInformation10.3390/info1503014315:3(143)Online publication date: 5-Mar-2024
https://doi.org/10.3390/info15030143
Ma LHong HMeng FWu QWu J(2024)Deep Progressive Asymmetric Quantization Based on Causal Intervention for Fine-Grained Image RetrievalIEEE Transactions on Multimedia10.1109/TMM.2023.327999026(1306-1318)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3279990
Chen ZZhao LZhang ZLuo XXu X(2024)Characteristics Matching Based Hash Codes Generation for Efficient Fine-Grained Image Retrieval2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01635(17273-17281)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01635
Show More Cited By

Recommendations

Partial-Softmax Loss based Deep Hashing
WWW '21: Proceedings of the Web Conference 2021

Recently, deep supervised hashing methods have shown state-of-the-art performance by integrating feature learning and hash codes learning into an end-to-end network to generate high-quality hash codes. However, it is still a challenge to learn ...
Attention-Aware Feature Pyramid Ordinal Hashing for Image Retrieval
MMAsia '19: Proceedings of the 1st ACM International Conference on Multimedia in Asia

Due to the effectiveness of representation learning, deep hashing methods have attracted increasing attention in image retrieval. However, most existing deep hashing methods merely encode the raw information of the last layer for hash learning, which ...
HybridHash: Hybrid Convolutional and Self-Attention Deep Hashing for Image Retrieval
ICMR '24: Proceedings of the 2024 International Conference on Multimedia Retrieval

Deep image hashing aims to map input images into simple binary hash codes via deep neural networks and thus enable effective large-scale image retrieval. Recently, hybrid networks that combine convolution and Transformer have achieved superior ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '19: Proceedings of the 2019 on International Conference on Multimedia Retrieval

June 2019

427 pages

ISBN:9781450367653

DOI:10.1145/3323873

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada
,
Alberto Del Bimbo
University of Florence, Italy
,
Zhongfei Zhang
Binghamton University, State University of New York, USA
,
Program Chairs:
Alexander Hauptmann
Carnegie Mellon University, USA
,
K. Selcuk Candan
Arizona State University, USA
,
Marco Bertini
University of Florence, Italy
,
Lexing Xie
Australia National University, Australia
,
Xiao-Yong Wei
Sichuan University, China

Copyright � 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

the Research Foundation of Science and Technology Plan Project in Guangdong Province
National Natural Science Foundation of China under Grants

Conference

ICMR '19

Sponsor:

SIGMM

ICMR '19: International Conference on Multimedia Retrieval

June 10 - 13, 2019

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
267
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Redaoui ABelalia ABelloulata K(2024)Deep Supervised Hashing by Fusing Multiscale Deep Features for Image RetrievalInformation10.3390/info1503014315:3(143)Online publication date: 5-Mar-2024
https://doi.org/10.3390/info15030143
Ma LHong HMeng FWu QWu J(2024)Deep Progressive Asymmetric Quantization Based on Causal Intervention for Fine-Grained Image RetrievalIEEE Transactions on Multimedia10.1109/TMM.2023.327999026(1306-1318)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3279990
Chen ZZhao LZhang ZLuo XXu X(2024)Characteristics Matching Based Hash Codes Generation for Efficient Fine-Grained Image Retrieval2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01635(17273-17281)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01635
Zeng ZWang JChen BDai TXia SWang Z(2024)Pyramid hybrid pooling quantization for efficient fine-grained image retrievalPattern Recognition Letters10.1016/j.patrec.2023.12.022178:C(106-114)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.patrec.2023.12.022
Xu DWu QYue BZheng X(2023)A Fast Hash Image Retrieval Method Based on Dual LearningProceedings of the 2023 9th International Conference on Computing and Artificial Intelligence10.1145/3594315.3594328(81-87)Online publication date: 17-Mar-2023
https://dl.acm.org/doi/10.1145/3594315.3594328
Li X(2023)Non-Relaxation Deep Hashing Method for Fast Image RetrievalIEEE Access10.1109/ACCESS.2023.324481311(17684-17692)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3244813
Sun HLang WXu CLiu NZhou H(2023)Graph-based discriminative features learning for fine-grained image retrievalSignal Processing: Image Communication10.1016/j.image.2022.116885110(116885)Online publication date: Jan-2023
https://doi.org/10.1016/j.image.2022.116885
Tan XZou YGuo ZZhou KYuan Q(2022)Deep Contrastive Self-Supervised Hashing for Remote Sensing Image RetrievalRemote Sensing10.3390/rs1415364314:15(3643)Online publication date: 29-Jul-2022
https://doi.org/10.3390/rs14153643
Redaoui ABelloulata K(2022)Deep Feature Pyramid Hashing for Efficient Image RetrievalInformation10.3390/info1401000614:1(6)Online publication date: 22-Dec-2022
https://doi.org/10.3390/info14010006
Chen ZLuo XWang YGuo SXu X(2022)Fine-Grained Hashing With Double FilteringIEEE Transactions on Image Processing10.1109/TIP.2022.314515931(1671-1683)Online publication date: 2022
https://doi.org/10.1109/TIP.2022.3145159
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents