• Shreya V. Dhoke
• Anupama D. Sakhare
• Satish J. Sharma

Internet is an ocean of information spread across various websites, where it is categorized, interlinked and mostly freely available for everyone. A vast amount of data is being created every second. All this ‘Big Data’ is in heterogeneous formats. We need to access information fast and quickly. Data extraction can be done manually but it can be time-consuming and can also be a very complicated task, for this reason Web Scraping is used. Web Scraping is the technique of automating the process of navigating through links, and then navigating and collecting the relevant data from these relevant links. The proposed system is a method of extracting and restructuring information from web pages. It is a technique for targeted, automated extraction of information from websites. This system acquires non-tabular or poorly structured data from websites and converts it into a usable structured format. The main objective of the proposed system is to extract information from one or many websites and process it into simple structures such as CSV files. In this proposed system, Text Grepping technique is used, that offers insight into price data, market dynamics, prevailing trends, practices employed by various competitors, and the challenges they face. The result of this technique is to easily access relevant data from websites. The proposed system can be modified for scraping dynamic websites. This proposed system will be beneficial in many business and at education areas.

Web scraping, Big Data, CSV file, Structured and Unstructured data

"EFFICIENT SCRAPING OF DATA FROM WEBSITES USING SELENIUM", International Journal of Emerging Technologies and Innovative Research (www.jetir.org), ISSN:2349-5162, Vol.9, Issue 6, page no.353-357, June-2022, Available :http://www.jetir.org/papers/JETIRFM06063.pdf

"EFFICIENT SCRAPING OF DATA FROM WEBSITES USING SELENIUM", International Journal of Emerging Technologies and Innovative Research (www.jetir.org | UGC and issn Approved), ISSN:2349-5162, Vol.9, Issue 6, page no. pp353-357, June-2022, Available at : http://www.jetir.org/papers/JETIRFM06063.pdf