Skip to main content
SpringerLink
Log in

Graph Computing System and Application Based on Large-Scale Information Network

  • Conference paper
  • First Online:
Space Information Network (SINC 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1353))

Included in the following conference series:

Abstract

Graph computing is more and more widely used in various fields such as spatial information network and social network. However, the existing graph computing systems have some problems like complex programming and steep learning curve. This paper introduces GRAPE, a distributed large-scale GRAPh Engine, which has the unique features of solid theoretical guarantee, ease of use, auto-parallelization and high performance. The paper also introduces several typical scenarios of graph computing, including entity resolution, link prediction, community detection and graph mining of spatial information network. In these scenarios, various problems have been encountered in the existing systems, such as failure to compute over large-scale data due to the high computation complexity, loss of accuracy due to the cropping of original data and too long execution time. In the face of these challenges, GRAPE is easy to support these computing scenarios with a series of technical improvements. With the deployment of GRAPE in Alibaba, both effectiveness and efficiency of graph computing have been greatly improved.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Alidata (2017). https://dt.alibaba.com/page21.htm

  2. Apache Kafka (2017). https://kafka.apache.org/

  3. Turi Create: simplifies the development of custom machine learning models (2019). https://github.com/apple/turicreate

  4. Aliyun dataworks (2020). https://help.aliyun.com/product/72772.html

  5. Aliyun maxcompute (2020). https://help.aliyun.com/document_detail/27800.html

  6. Aliyun odps (2020). https://cn.aliyun.com/product/odps

  7. Aliyun oss (2020). https://www.aliyun.com/product/oss

  8. PowerSwitch: Adaptive Prediction and Mode Switch on Graph-parallel Computation (2020). https://ipads.se.sjtu.edu.cn/projects/powerswitch.html

  9. Pregel+ (2020). http://www.cse.cuhk.edu.hk/pregelplus/

  10. Authors, J.: JanusGraph (2017). https://janusgraph.org

  11. Avery, C.: Giraph: large-scale graph processing infrastructure on Hadoop. In: Proceedings of the Hadoop Summit, Santa Clara, vol. 11, vo. 3, pp. 5–9 (2011)

    Google Scholar 

  12. Brizan, D.G., Tansel, A.U.: A survey of entity resolution and record linkage methodologies. Commun. IIMA 6(3), 5 (2006)

    Google Scholar 

  13. Callan, J., et al.: The clueweb12 dataset (2013)

    Google Scholar 

  14. Chen, R., Shi, J., Chen, Y., Zang, B., Guan, H., Chen, H.: PowerLyra: differentiated graph computation and partitioning on skewed graphs. ACM Trans. Parallel Comput. (TOPC) 5(3), 1–39 (2019)

    Article  Google Scholar 

  15. Ching, A., Edunov, S., Kabiljo, M., Logothetis, D., Muthukrishnan, S.: One trillion edges: graph processing at Facebook-scale. Proc. VLDB Endow. 8(12), 1804–1815 (2015)

    Article  Google Scholar 

  16. Fan, W., Wang, X., Wu, Y., Xu, J.: Association rules with graph patterns. Proc. VLDB Endow. 8(12), 1502–1513 (2015)

    Article  Google Scholar 

  17. Fan, W., Xu, J., Wu, Y., Yu, W., Jiang, J.: GRAPE: parallelizing sequential graph computations. Proc. VLDB Endow. 10(12), 1889–1892 (2017)

    Article  Google Scholar 

  18. Figueiredo, D.R., Ribeiro, L.F.R., Saverese, P.H.: struc2vec: learning node representations from structural identity. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, pp. 13–17 (2017)

    Google Scholar 

  19. Getoor, L., Diehl, C.P.: Link mining: a survey. ACM SIGKDD Explor. Newslett. 7(2), 3–12 (2005)

    Article  Google Scholar 

  20. Gonzalez, J.E., Low, Y., Gu, H., Bickson, D., Guestrin, C.: PowerGraph: distributed graph-parallel computation on natural graphs. In: 10th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 2012), pp. 17–30 (2012)

    Google Scholar 

  21. Gonzalez, J.E., Xin, R.S., Dave, A., Crankshaw, D., Franklin, M.J., Stoica, I.: GraphX: Graph processing in a distributed dataflow framework. In: 11th \(\{\)USENIX\(\}\) Symposium on Operating Systems design and Implementation (\(\{\)OSDI\(\}\) 2014), pp. 599–613 (2014)

    Google Scholar�

  22. Granville, K.: Facebook and Cambridge Analytica: what you need to know as fallout widens. New York Times 19, 18 (2018)

    Google Scholar 

  23. Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864 (2016)

    Google Scholar 

  24. Inc, A.: Euler: a distributed graph deep learning framework (2019). https://github.com/alibaba/euler

  25. Inc, A.: Graph Compute (2020). https://help.aliyun.com/document_detail/134189.html

  26. Inc, A.: Amazon Neptune (2018). https://aws.amazon.com/neptune/

  27. Jones, N.D.: An introduction to partial evaluation. ACM Comput. Surv. (CSUR) 28(3), 480–503 (1996)

    Article  Google Scholar 

  28. Khan, B.S., Niazi, M.A.: Network community detection: a review and visual survey. arXiv preprint arXiv:1708.00977 (2017)

  29. Khayyat, Z., Awara, K., Alonazi, A., Jamjoom, H., Williams, D., Kalnis, P.: Mizan: a system for dynamic load balancing in large-scale graph processing. In: Proceedings of the 8th ACM European Conference on Computer Systems, pp. 169–182 (2013)

    Google Scholar 

  30. Leskovec, J.: Graph representation learning with graph convolutional networks

    Google Scholar 

  31. Malewicz, G., et al.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 135–146 (2010)

    Google Scholar 

  32. McSherry, F., Isard, M., Murray, D.G.: Scalability! But at what \(\{\)COST\(\}\)? In: 15th Workshop on Hot Topics in Operating Systems (HotOS \(\{\)XV\(\}\)) (2015)

    Google Scholar 

  33. Neo4j, I.: Neo4j Graph Platform (2021). https://neo4j.com/

  34. Nielsen, A.: Nielsen global online consumer survey: trust, value and engagement in advertising. Ad week Media (2010)

    Google Scholar 

  35. Pearl, J.: Theoretical impediments to machine learning with seven sparks from the causal revolution. arXiv preprint arXiv:1801.04016 (2018)

  36. Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining pp. 701–710 (2014)

    Google Scholar 

  37. Ramalingam, G., Reps, T.: On the computational complexity of dynamic graph problems. Theoret. Comput. Sci. 158(1–2), 233–277 (1996)

    Article  MathSciNet  Google Scholar 

  38. Salihoglu, S., Widom, J.: GPS: a graph processing system. In: Proceedings of the 25th International Conference on Scientific and Statistical Database Management, pp. 1–12 (2013)

    Google Scholar 

  39. Tian, Y., Balmin, A., Corsten, S.A., Tatikonda, S., McPherson, J.: From “think like a vertex” to “think like a graph”. Proc. VLDB Endow. 7(3), 193–204 (2013)

    Article  Google Scholar 

  40. Valiant, L.G.: A bridging model for parallel computation. Commun. ACM 33(8), 103–111 (1990)

    Article  Google Scholar 

  41. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32, 4–24 (2020)

    Google Scholar 

  42. Yan, D., Cheng, J., Lu, Y., Ng, W.: Blogel: a block-centric framework for distributed computation on real-world graphs. Proc. VLDB Endow. 7(14), 1981–1992 (2014)

    Article  Google Scholar 

  43. Zhu, R., et al.: AliGraph: a comprehensive graph neural network platform. arXiv preprint arXiv:1902.08730 (2019)

Download references

Author information

Authors and Affiliations

  1. Peking University, Beijing, China

    Jingbo Xu

  2. Alibaba Group, Hangzhou, China

    Jingbo Xu, Zhao Li, Weibin Zeng & Jiaming Huang

Authors
  1. Jingbo Xu
    View author publications

    You can also search for this author in PubMed�Google Scholar

  2. Zhao Li
    View author publications

    You can also search for this author in PubMed�Google Scholar

  3. Weibin Zeng
    View author publications

    You can also search for this author in PubMed�Google Scholar

  4. Jiaming Huang
    View author publications

    You can also search for this author in PubMed�Google Scholar

Corresponding author

Correspondence to Zhao Li .

Editor information

Editors and Affiliations

  1. Institute of China Electronic Equipment, Beijing, China

    Quan Yu

Rights and permissions

Reprints and permissions

Copyright information

� 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xu, J., Li, Z., Zeng, W., Huang, J. (2021). Graph Computing System and Application Based on Large-Scale Information Network. In: Yu, Q. (eds) Space Information Network. SINC 2020. Communications in Computer and Information Science, vol 1353. Springer, Singapore. https://doi.org/10.1007/978-981-16-1967-0_12

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-1967-0_12

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-1966-3

  • Online ISBN: 978-981-16-1967-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation