Optimization of web search techniques using frequency analysis

Aishwarya, M.; Ilayaraja, N.; Periakaruppan, R.M.

doi:10.22075/ijnaa.2021.5852

Optimization of web search techniques using frequency analysis

Document Type : Research Paper

Authors

M. Aishwarya ¹

N. Ilayaraja ¹

R.M. Periakaruppan ²

¹ Computer Applications, PSG College of Technology, Coimbatore, India

² Applied Mathematics and Computational Sciences, PSG College of Technology, Coimbatore, India

10.22075/ijnaa.2021.5852

Abstract

The raw data obtained in the form of search results may be large for any particular problem, but is often a relatively small subset of the data that are relevant, and a search engine does not enable discovering the necessary subset of relevant text data in a large text collection. In this paper, a solution to a problem called conformity to truth, which studies how to find websites with the maximum amount of true facts, from a large amount of conflicting information on the user-defined topic, is proposed. Two algorithms called ParaSearch and FactFinder, which helps in identifying the best web links for searching general information and finding individual facts respectively are proposed. In ParaSearch, latent Dirichlet allocation (LDA) is used to identify the top 10 frequent terms using which we further construct a similarity matrix to identify the best web pages. In FactFinder, the usage of semantic processing is done to identify the best web pages, building upon the existing Page Rank Algorithm to further optimize the search results. The results prove that ParaSearch can identify web pages with the maximum number of facts conforming to the truth much better than popular search engines. The ambiguity of the individual facts is decreased to a great extent by using the FactFinder algorithm. Thus these algorithms will increase the accuracy of identifying possible web links for a given search word much better than most of the popular search engines.

Keywords

frequency analysis

Latent Dirichlet allocation

text mining

[1] B. Amento, L Terveen and W. Hill, Does Authority mean quality? predicting expert quality ratings of web
documents, Proc. ACM SIGIR ’00, Assoc. Comput. Machin. (2000) 296–303.
[2] I. Antonellis and E. Gallopoulos, Exploring term-document matrices from matrix models in text mining, Proc.
SIAM Text Mining Workshop, 6th SIAM SDM Conference, Maryland, 2006.
[3] D.M. Blei, A.Y. Ng and M.I. Jordan, Latent dirichlet allocation, J. Mach. Learn. Res. 3 (2003) 993–1022.
[4] A. Borodin, G.O. Roberts, J.S. Rosenthal and P. Tsaparas, Link analysis ranking: Algorithms, theory, and
experiments, ACM Trans. Internet Technol. 5(1) (2005) 231–297.
[5] E. Dragut, F. Fang, P. Sistla, C. Yu and W. Meng, Stop word and related problems in web interface integration,
Proc. VLDB Endow. (2009) 349–360.
[6] J. Han, X. Yin and P.S. Yu, Truth discovery with multiple conflicting information providers on the web, IEEE
Trans. Knowledge Data Engin.20(6) (2008) 796–808.
[7] R. Jizba, Measuring Search Effectiveness, https : //www.creighton.edu/f ileadmin/user/HSL/docs/ref /Searching −
Recall P recision.pdf, (2007).
[8] D. Jurafsky and J.H. Martin, Speech and language processing: An introduction to natural language processing,
Computational Linguistics, and Speech Recognition, Prentice Hall PTR, Upper Saddle River, NJ, 2000.
[9] Y.A. Kim, M.T. Le, H.W. Lauw, E.P. Lim, H. Liu and J. Srivastava, Building a web of trust without explicit trust
ratings, 2008 IEEE 24th Int. Conf. Data Engin. Workshop, (2008) 531–536.
[10] J. M. Kleinberg, Authoritative sources in a hyperlinked environment, J. ACM 46(5) (1999) 604–632.
[11] Y. Matsuo and M. Ishizuka, Keyword extraction from a single document using word co-occurrence statistical
information, Int. J. Artificial Intell. Tools 13(01) (2004) 157–169.
[12] L. Page, S. Brin, R. Motwani and T. Winograd, The Pagerank Citation Ranking: Bringing Order to the Web,
Technical Report, Stanford Digital Library Technologies Project, (1999).
[13] S. Ramachandran, S. Paulraj, S. Joseph and V. Ramaraj, Enhanced trustworthy and high-quality information
retrieval system for web search engines, Int. J. Comput. Sci. 5 (2009).