UGC Approved Journal no 63975(19)

ISSN: 2349-5162 | ESTD Year : 2014
Call for Paper
Volume 11 | Issue 12 | December 2024

JETIREXPLORE- Search Thousands of research papers



WhatsApp Contact
Click Here

Published in:

Volume 6 Issue 4
April-2019
eISSN: 2349-5162

UGC and ISSN approved 7.95 impact factor UGC Approved Journal no 63975

7.95 impact factor calculated by Google scholar

Unique Identifier

Published Paper ID:
JETIRBE06016


Registration ID:
206676

Page Number

84-87

Share This Article


Jetir RMS

Title

SMART CRAWLER-A TWO STAGE FRAME WORK FOR CRAWLING DEEP WEBSITES

Abstract

A web Search is a program, which automatically traverses the web by downloading documents and following links from page to page. They are mainly used by web search engines to gather data for indexing. Web Search is also known as spiders, robots, worms etc. In this proposed project the web search program acts as an smart crawler application where the crawling of the pages is done not by the overall page rank (i.e. Overall total page’s rank count visited by users),but the pages are crawled based on individual page count of individual URL’s. The proposed smart application is used to measure the individual page traffic accurately and is mainly useful for Web–Masters to maintain the traffic of each and every web page in very sophisticated manner. As this application requires internet connection, the internet connection should be of enough bandwidth for processing of the Web pages URL’s accurately. In this current application we can find out the count of successfully crawled URL’s as well as failed URL’s successfully based on the pages which were crawled by internet traffic. In this project BFS algorithm also known as “Breadth First Search” is used, for crawling the web pages based on the individual page rank and there will be no back traversing of URL’s, so this will give unique results. As an extension the monitoring of count of failed URL’s and count of successful URL’s crawled in this application has also been included, this gives more importance for the application of each and every URL.

Key Words

Web search, Web pages, Crawled URL’s, Breadth First Search, Traversing, etc

Cite This Article

"SMART CRAWLER-A TWO STAGE FRAME WORK FOR CRAWLING DEEP WEBSITES", International Journal of Emerging Technologies and Innovative Research (www.jetir.org), ISSN:2349-5162, Vol.6, Issue 4, page no.84-87, April-2019, Available :http://www.jetir.org/papers/JETIRBE06016.pdf

ISSN


2349-5162 | Impact Factor 7.95 Calculate by Google Scholar

An International Scholarly Open Access Journal, Peer-Reviewed, Refereed Journal Impact Factor 7.95 Calculate by Google Scholar and Semantic Scholar | AI-Powered Research Tool, Multidisciplinary, Monthly, Multilanguage Journal Indexing in All Major Database & Metadata, Citation Generator

Cite This Article

"SMART CRAWLER-A TWO STAGE FRAME WORK FOR CRAWLING DEEP WEBSITES", International Journal of Emerging Technologies and Innovative Research (www.jetir.org | UGC and issn Approved), ISSN:2349-5162, Vol.6, Issue 4, page no. pp84-87, April-2019, Available at : http://www.jetir.org/papers/JETIRBE06016.pdf

Publication Details

Published Paper ID: JETIRBE06016
Registration ID: 206676
Published In: Volume 6 | Issue 4 | Year April-2019
DOI (Digital Object Identifier):
Page No: 84-87
Country: -, -, -- .
Area: Engineering
ISSN Number: 2349-5162
Publisher: IJ Publication


Preview This Article


Downlaod

Click here for Article Preview

Download PDF

Downloads

0003068

Print This Page

Current Call For Paper

Jetir RMS