Web Scraping for Beginners: Extract a Website in 10 Minutes

Web scraping for beginners should produce data before theory. The practice site below is built for scraping, so you can learn selectors without violating somebody’s terms or hammering a real store.

Web scraping setup for beginners

python -m pip install requests beautifulsoup4

Web scraping code you can paste

import csv
import requests
from bs4 import BeautifulSoup

url = "https://quotes.toscrape.com/"
r = requests.get(url, timeout=30, headers={"User-Agent": "LearningScraper/1.0"})
r.raise_for_status()
soup = BeautifulSoup(r.text, "html.parser")

rows = []
for card in soup.select(".quote"):
    rows.append({
        "text": card.select_one(".text").get_text(strip=True),
        "author": card.select_one(".author").get_text(strip=True),
    })

with open("quotes.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["text", "author"])
    writer.writeheader()
    writer.writerows(rows)

print(f"saved {len(rows)} rows")

A CSS selector is the short pattern passed to select; it identifies elements by class, id, or structure. Use browser developer tools to inspect the item you need, then test the selector against one page before adding pagination.

When beginner web scraping breaks

An empty list means the selector does not match the downloaded HTML. If the browser shows content that requests cannot see, JavaScript rendered it later; ScrapingBee can return the completed page. A 403 means stop retrying and read the blocking guide. A timeout needs bounded retries with delays, never an infinite loop.

Keep the first crawler small, identify itself where appropriate, honor robots.txt, and store responses while debugging. The saved HTML tells the truth faster than guessing at the selector.