Web scraping | Scrape eCommerce Websites Without Coding

Octoparse

6 Jun 201904:36

EducationalLearning

32 Likes 10 Comments

TLDRThis tutorial demonstrates how to use Octoparse for web scraping on Aliexpress.com. It guides viewers through the process of setting up a task to scrape product information, starting from opening the website and entering a keyword, to creating pagination loops and item loops for detailed pages. The steps include selecting elements to extract data, such as product names and prices, and running the task for local extraction. The tutorial is designed to be clear and engaging, ensuring users can efficiently gather the needed product information.

Takeaways

🌐 Start by opening AliExpress and searching for the product category you want to scrape, such as 'laptop'.
🔗 Copy the URL of the product listing page to use in Octoparse for building a new task.
🛠️ In Octoparse, enter the URL in 'Invest Mode' to create a new task and save the URL.
🔄 Create a pagination loop in Octoparse to navigate through all listing pages by interacting with the 'next page' button.
🔍 The Octoparse interface is divided into three parts: the workflow box, settings area, and the website view.
🔁 To scrape all product information, set up a loop to go through each listing page and click through each detail page.
🎯 Select elements on the product listing to create a loop item, which allows Octoparse to identify and interact with similar items.
📊 Choose 'Extract the Text of the Selected Element' to gather specific data like product names and prices from the detail pages.
💼 After setting up the extraction rules, run the task by starting extraction and selecting 'Local Extraction'.
📈 Monitor the task's progress and check the data panel to ensure the scraped data is collected successfully.
💬 For any questions or issues, leave a comment in the tutorial for assistance or further guidance.

Q & A

What is the main topic of the tutorial?
-The main topic of the tutorial is how to scrape product information from Aliexpress.com using Octoparse.
Where should you start the tutorial from?
-You should start the tutorial by opening Aliexpress.com in your browser and searching for the product you want to scrape, in this case, laptops.
What is the first step in creating a new scraping task in Octoparse?
-The first step is to enter the URLs of the website you want to scrape and save the URL in Octoparse's Advanced Mode.
How does Octoparse divide its interface?
-Octoparse divides its interface into three parts: the workflow box on the left, the setting area on the right, and the view of the website at the bottom.
What is the purpose of creating a pagination loop in the scraping process?
-The purpose of creating a pagination loop is to allow Octoparse to go through each listing page and scrape all product information available across multiple pages.
How does Octoparse identify similar items for looping?
-Octoparse identifies similar items by selecting an element from the listing, and then it finds and puts other similar items in red color for the loop.
What action should you take to navigate through each detail page?
-You should create a loop item and select an element, like the product name, to click and then choose 'Loop click selected link' from the action tip.
How do you select the data to extract during the scraping process?
-To select the data to extract, you click on the element you want to extract, like the product name or price, and choose 'Extract the text of the selected element' from the action tip.
What is the final step in running the scraping task?
-The final step is to click 'Start extraction' and select 'Local extraction' to run the task, and then check the scripting status and data panel for the extracted data.
What is suggested for users who have questions about the tutorial?
-Users are encouraged to leave a comment down below the tutorial video if they have any questions.

Outlines

00:00

🌐 Introduction to Octoparse Web Scraping Tutorial

This paragraph introduces the Octoparse web scraping tutorial led by Ashley. The tutorial aims to demonstrate how to extract product information from AliExpress using Octoparse. It begins by instructing the user to open AliExpress in their browser and search for a product, using 'laptop' as an example. The user is then asked to copy the URL of the search results page, which will be utilized in the tutorial. The paragraph outlines the initial steps of using Octoparse, including entering the URL in the investment mode and saving it to proceed to the product listing page within the Octoparse building browser.

🔄 Setting Up Pagination Loop for Data Scraping

The second paragraph delves into the creation of a pagination loop for efficient web scraping. It explains that after the webpage finishes loading, the user should interact with the Octoparse interface, which is divided into three parts: the workflow box, the setting area, and the website view. The tutorial instructs the user to scroll down to the bottom of the page to find the pagination bar and click the 'next page' button. By doing so, the user enables Octoparse to navigate through each listing page, thereby creating a pagination loop that allows for the scraping of product information across all pages.

🔍 Clicking Through Each Detail Page with Loop Item

This paragraph focuses on creating a loop item to click through each product detail page and gather further information. The user is guided to select an element, such as the product name from the listing, and use the 'select all' function to create a looper item. By clicking 'Lube click selected link' from the action tip, a loop is established that enables Octoparse to click through each detailed page, mirroring the actions performed on the first page.

📋 Selecting and Extracting Required Data

The fourth paragraph emphasizes the selection and extraction of necessary data. Users are instructed to click on elements, like the product name or price, and choose to extract the text from the selected element using the 'extract the text' action tip. The tutorial suggests repeating this process for additional elements, thus ensuring comprehensive data extraction from the product listing pages.

🚀 Running the Task for Data Extraction

The final paragraph outlines the process of executing the data scraping task. After setting up the rules, users can initiate the task by clicking 'start extraction' and opting for local extraction. The tutorial mentions that users can monitor the scripting status and view the extracted data in the data panel. The video concludes with an invitation for viewers to leave comments and ask questions if they encounter any issues.

Mindmap

Keywords

💡Octoparse

Octoparse is a web scraping tool that enables users to extract data from websites. In the context of the video, it is the primary software used to demonstrate how to scrape product information from Aliexpress. The video outlines the steps to set up a task in Octoparse, highlighting its features such as creating a pagination loop and selecting data extraction elements.

💡Aliexpress

Aliexpress is an online retail service that belongs to the Alibaba Group. Known for its wide range of products at competitive prices, it is the target website from which the video tutorial aims to scrape product information. The keyword signifies the practical application of web scraping techniques in the realm of e-commerce and market research.

💡Web Scraping

Web scraping refers to the process of extracting data from websites. It is a technique used to collect information from web pages into a structured format for further analysis or use. In the video, web scraping is the core activity, with Octoparse as the tool facilitating the extraction of product details from Aliexpress.

💡Product Information

Product information encompasses all the details about a product, such as its name, price, description, and images. In the context of the video, the focus is on scraping product information from Aliexpress, which is critical for market analysis, price comparison, and e-commerce operations.

💡Pagination Loop

A pagination loop is a process in web scraping that automates the navigation through multiple pages of results. It allows the scraper to systematically collect data from all pages, not just the initial one. In the video, creating a pagination loop is a key step to ensure that all product listings across different pages are scraped.

💡Looper Item

A looper item in the context of web scraping is an element that is used to identify and iterate over similar elements on a webpage. It allows the scraper to go through each item, such as product listings, and extract the required information. The video demonstrates how to create a looper item to click through each product detail page and gather more detailed information.

💡Data Extraction

Data extraction is the process of collecting and retrieving specific data from a larger set of information. In the video, data extraction is the goal, where the scraped product information from Aliexpress is collected and saved in a structured format for further use.

💡Task

In the context of the video, a task refers to a specific web scraping operation set up within Octoparse. It involves defining the website to scrape, creating a pagination loop, and setting up looper items for data extraction. The task encapsulates the entire process from start to finish, including the rules and parameters for scraping product information.

💡Start Extraction

Start extraction is the command used to initiate the web scraping process once all the necessary rules and settings have been configured. It triggers the task to begin collecting data from the target website according to the defined parameters. In the video, this step marks the beginning of the actual data collection phase.

💡Local Extraction

Local extraction is a mode in web scraping where the data is extracted and saved directly on the user's local machine, rather than being processed on a remote server. This method ensures that the data is stored locally and can be accessed without an internet connection. The video mentions running a task in local extraction mode, which implies that the scraped data from Aliexpress will be saved on the user's computer.

💡Data Panel

The data panel is the section within a web scraping tool, such as Octoparse, where the extracted data is displayed and organized. It allows users to review and manage the collected information. In the video, the data panel is where the scraped product information from Aliexpress will be visible once the extraction process is complete.

Highlights

Ashley introduces an Octoparse web scraping tutorial.

The tutorial demonstrates scraping product information from Aliexpress.com.

Octoparse is used for the web scraping task.

The first step is to open Aliexpress.com and enter a product keyword.

The URL of the search results is copied for use in the tutorial.

A new task is created in Octoparse by entering the URL.

The tutorial explains how to create a pagination loop for multiple pages.

The Octoparse interface is divided into a workflow box, setting area, and website view.

The pagination loop is created by interacting with the next page button.

A loop item is created to click through each product detail page.

The tutorial shows how to select and extract data from the product listing.

Different elements like product name and price can be extracted.

The extraction rules are set up in the Octoparse task.

The task is run for local extraction to gather data.

The extracted data can be viewed in the data panel.

The tutorial concludes with an invitation for questions and comments.

Transcripts

Browse More Related Video

ParseHub Tutorial: Scraping Product Details from Amazon

ParseHub Tutorial: Scraping 2 eCommerce Websites in 1 Project

ParseHub Tutorial: Directories

Beautiful Soup 4 Tutorial #1 - Web Scraping With Python

Web Scraping with ChatGPT is mind blowing 🤯

Web Scraping Tutorial | Data Scraping from Websites to Excel | Web Scraper Chorme Extension