poster

On the Model Selection of Bernoulli Restricted Boltzmann Machines Through Harmony Search

Authors:

Joao Papa Papa,

Gustavo Henrique Rosa,

Kelton A. Costa,

Nilceu A. Marana,

Walter Scheirer,

David Daniel CoxAuthors Info & Claims

GECCO Companion '15: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation

Pages 1449 - 1450

https://doi.org/10.1145/2739482.2764628

Published: 11 July 2015 Publication History

Get Access

Abstract

Restricted Boltzmann Machines (RBMs) are amongst the most widely pursued techniques in deep learning-based environments. However, the problem of selecting a suitable set of parameters still remains an open question, since it is not straightforward to choose them without prior knowledge. In this paper, we introduce the Harmony Search (HS) optimization algorithm to find out a suitable set of parameters that minimize the reconstruction error of Bernoulli RBMs, which address binary-valued visible and hidden units. The results have shown the suitability of using HS for such task when compared to other optimization techniques.

References

[1]

J. S. Bergstra, D. Yamins, and D. D. Cox. Hyperopt: A python library for optimizing the hyperparameters of machine learning algorithms. In Python for Scientific Computing Conference, pages 1--7, 2013.

Crossref

Google Scholar

[2]

Z. W. Geem. Music-Inspired Harmony Search Algorithm: Theory and Applications. Springer Publishing Company, Incorporated, 1st edition, 2009.

Digital Library

Google Scholar

[3]

G. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1771--1800, 2002.

Digital Library

Google Scholar

[4]

J. Kennedy and R. Eberhart. Swarm Intelligence. M. Kaufman, 2001.

Digital Library

Google Scholar

[5]

M. Mahdavi, M. Fesanghary, and E. Damangir. An improved harmony search algorithm for solving optimization problems. Applied Mathematics and Computation, 188(2):1567--1579, 2007.

Crossref

Google Scholar

[6]

M. G. Omran and M. Mahdavi. Global-best harmony search. Applied Mathematics and Computation, 198(2):643--656, 2008.

Crossref

Google Scholar

[7]

J. Snoek, H. Larochelle, and R. P. Adams. Practical bayesian optimization of machine learning algorithms. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 2951--2959. Curran Associates, Inc., 2012.

Digital Library

Google Scholar

[8]

F. Wilcoxon. Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80--83, 1945.

Crossref

Google Scholar

Cited By

View all

De Oliveira AFigueiredo D(2022)Network Connectivity and Learning Performance on Restricted Boltzmann Machines2022 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN55064.2022.9892202(1-8)Online publication date: 18-Jul-2022
https://doi.org/10.1109/IJCNN55064.2022.9892202
Roder Mde Rosa GPapa JBreve F(2020)Fine-Tuning Temperatures in Restricted Boltzmann Machines Using Meta-Heuristic Optimization2020 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC48606.2020.9185505(1-8)Online publication date: Jul-2020
https://doi.org/10.1109/CEC48606.2020.9185505
Muhammad AAseere AChiroma HShah HGital AHashem I(2020)Deep learning application in smart cities: recent development, taxonomy, challenges and research prospectsNeural Computing and Applications10.1007/s00521-020-05151-8Online publication date: 15-Jul-2020
https://doi.org/10.1007/s00521-020-05151-8
Show More Cited By

Index Terms

On the Model Selection of Bernoulli Restricted Boltzmann Machines Through Harmony Search
1. Computing methodologies
  1. Artificial intelligence
    1. Philosophical/theoretical foundations of artificial intelligence

Recommendations

An improved adaptive binary Harmony Search algorithm

Harmony Search (HS), inspired by the music improvisation process, is a new meta-heuristic optimization method and has been successfully used to tackle the optimization problems in discrete or continuous space. Although the standard HS algorithm is able ...
Improved PSO algorithm with harmony search for complicated function optimization problems
ISNN'12: Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I

Improved particle swarm optimization algorithm with harmony search (IHPSO) is proposed in this paper. This algorithm takes particle swarm search direction estimation mechanism and harmony search (HS) approach to particle swarm optimization (PSO) ...
Hybrid harmony search particle swarm optimization with global dimension selection

This study presents a hybrid harmony search particle swarm optimization with global dimension selection (HHSPSO-GDS) for improving the performance of particle swarm optimization (PSO). In HHSPSO-GDS, a new global velocity updating strategy is introduced ...

Comments

Information & Contributors

Information

Published In

GECCO Companion '15: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation

July 2015

1568 pages

ISBN:9781450334884

DOI:10.1145/2739482

Editor:
Sara Silva
Universidade de Lisboa, Portugal
,
General Chair:
Anna I. Esparcia-Alc�zar
Universitat Polit�cnica de Val�ncia, Spain

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2015

Check for updates

Author Tags

Qualifiers

Poster

Funding Sources

Conference

GECCO '15

Sponsor:

SIGEVO

GECCO '15: Genetic and Evolutionary Computation Conference

July 11 - 15, 2015

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
125
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

De Oliveira AFigueiredo D(2022)Network Connectivity and Learning Performance on Restricted Boltzmann Machines2022 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN55064.2022.9892202(1-8)Online publication date: 18-Jul-2022
https://doi.org/10.1109/IJCNN55064.2022.9892202
Roder Mde Rosa GPapa JBreve F(2020)Fine-Tuning Temperatures in Restricted Boltzmann Machines Using Meta-Heuristic Optimization2020 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC48606.2020.9185505(1-8)Online publication date: Jul-2020
https://doi.org/10.1109/CEC48606.2020.9185505
Muhammad AAseere AChiroma HShah HGital AHashem I(2020)Deep learning application in smart cities: recent development, taxonomy, challenges and research prospectsNeural Computing and Applications10.1007/s00521-020-05151-8Online publication date: 15-Jul-2020
https://doi.org/10.1007/s00521-020-05151-8
Zhang YWang RHossain MAlhamid MGuizani M(2019)Heterogeneous Information Network-Based Content Caching in the Internet of VehiclesIEEE Transactions on Vehicular Technology10.1109/TVT.2019.293679268:10(10216-10226)Online publication date: Oct-2019
https://doi.org/10.1109/TVT.2019.2936792
Wang RYe YJiang CMa X(2019)HIN-VMReSys: Heterogeneous Information Network based Vehicle Music Recommendation System2019 IEEE International Conference on Signal, Information and Data Processing (ICSIDP)10.1109/ICSIDP47821.2019.9173063(1-6)Online publication date: Dec-2019
https://doi.org/10.1109/ICSIDP47821.2019.9173063
Silva LCosta KPapa JRosa GAlbuquerque V(2019)Fine‐tuning restricted Boltzmann machines using quaternions and its application for spam detectionIET Networks10.1049/iet-net.2018.51728:3(164-168)Online publication date: May-2019
https://doi.org/10.1049/iet-net.2018.5172
Chiroma HGital ARana NAbdulhamid SMuhammad AUmar AAbubakar A(2019)Nature Inspired Meta-heuristic Algorithms for Deep Learning: Recent Progress and Novel PerspectivePolish River Basins and Lakes – Part II10.1007/978-3-030-17795-9_5(59-70)Online publication date: 24-Apr-2019
https://doi.org/10.1007/978-3-030-17795-9_5
Passos LRodrigues DPapa J(2018)Fine Tuning Deep Boltzmann Machines Through Meta-Heuristic Approaches2018 IEEE 12th International Symposium on Applied Computational Intelligence and Informatics (SACI)10.1109/SACI.2018.8440959(000419-000424)Online publication date: May-2018
https://doi.org/10.1109/SACI.2018.8440959
Nakisa BRastgoo MRakotonirainy AMaire FChandran V(2018)Long Short Term Memory Hyperparameter Optimization for a Neural Network Based Emotion Recognition FrameworkIEEE Access10.1109/ACCESS.2018.28683616(49325-49338)Online publication date: 2018
https://doi.org/10.1109/ACCESS.2018.2868361
Passos LPapa J(2018)Temperature-Based Deep Boltzmann MachinesNeural Processing Letters10.1007/s11063-017-9707-248:1(95-107)Online publication date: 1-Aug-2018
https://dl.acm.org/doi/10.1007/s11063-017-9707-2
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

An improved adaptive binary Harmony Search algorithm

Improved PSO algorithm with harmony search for complicated function optimization problems

Hybrid harmony search particle swarm optimization with global dimension selection