Thanks to visit codestin.com
Credit goes to github.com

Skip to content

erenonal/scraping-amazon

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 

Repository files navigation

SCRAPING FROM AMAZON

This is a code that gets the name of the product, the title of the review, the number of stars given, and the full comment from the review page of a product you want from Amazon.com. I took airpods 2 comments as an example here

Required libraries

  • requests
  • BeautifulSoup from bs4
  • pandas
After importing our libraries, we define an empty list to convert to dataframe structure after receiving our comments.
We use the header structure (User-Agent) so that the Amazon site does not consider us as robots and prevent us from pulling data.
We define the function that we send a request to the site (get_soup). We send a request by typing the link of the product we want into requests.get and adding headers to the end. With BeautifulSoup, we split the data from the lxml method. (can also be done in html)
Then, with the get_reviews function, we select the previously obtained data as product, title, rating and body part according to the html structure.
Finally, we take product, title, rating and body parts in each comment and put it in the empty directory (reviewlist) we defined at the beginning and save it as an excel file. Using the for loop with range, we determine from the beginning how many pages of comments we want to receive. The if part at the end of the for loop is to avoid an error when the last page is reached.

About

Review scraping from amazon .

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published