Files
pokemon-stock-checker/CLAUDE.md
T
mmcghen 8d382e723f feat: Implement Walmart scraper and integrate with existing architecture
- Added Walmart scraper to scrape product data from Walmart.com, including category pages and product details.
- Introduced a stealth browser module to handle bot protection and improve scraping reliability.
- Created a SQLite database for tracking product history, price changes, stock events, and user favorites.
- Developed a Discord bot for user interaction, allowing location setting and stock checking at local stores.
- Implemented a favorites system to manage priority products and categories with custom notification settings.
- Added news aggregation module to fetch and analyze Pokemon TCG news from various sources.
- Created tools for API discovery and monitoring, including a backend monitor for detecting new products.
- Added unit tests for database operations, product filtering, and API endpoints to ensure functionality.
- Enhanced existing modules with improved error handling and logging for better maintainability.
2026-03-27 23:08:09 -04:00

4.1 KiB

Pokemon Stock Monitor - Project Guide

Overview

A Pokemon TCG stock monitoring system that tracks retail sites (Target, PokemonCenter, GameStop, Best Buy, Walmart) for restocks and new drops, sending Discord notifications when products become available.

Architecture

User -> Chrome Extension (PokemonCenter API interception)
     -> Python Monitor (main.py) -> Scrapers -> ProductTracker -> Discord Notifications
     -> Flask Dashboard (localhost:5000) -> REST API

Directory Structure

Core Application

  • main.py - Entry point, scheduling loop, orchestrates scraping
  • config.py - All configuration (intervals, URLs, Discord webhook, browser settings)

Source Code (src/)

  • browser.py - Playwright/Chrome browser management
  • product_tracker.py - Detects new products and restocks, manages state
  • discord_notifier.py - Sends Discord webhook notifications
  • discord_bot.py - Interactive Discord bot commands
  • database.py - SQLite database for historical tracking
  • favorites.py - User favorites/watchlist management
  • scraper_state.py - Persists scraper state between runs
  • store_locator.py - Target store location lookup

Scrapers (scrapers/)

  • base.py - Product dataclass, BaseScraper abstract class
  • pokemoncenter.py - PokemonCenter scraper (needs stealth browser)
  • target.py - Target.com scraper
  • gamestop.py - GameStop.com scraper (light bot protection)
  • bestbuy.py - BestBuy.com scraper (moderate bot protection)
  • walmart.py - Walmart.com scraper (aggressive bot protection, may need stealth)

Chrome Extension (chrome-extension/)

  • manifest.json - Extension config (MV3)
  • background.js - Service worker, manages alarms and notifications
  • content.js - Injected into PokemonCenter pages
  • api-interceptor.js - Intercepts PokemonCenter API responses
  • popup.html/js - Extension popup UI

Dashboard (dashboard/)

  • app.py - Flask app, routes
  • api.py - REST API endpoints (/api/*)
  • templates/index.html - Main template
  • static/app.js - Frontend JavaScript

Data (data/)

  • products.json - Tracked products state
  • scraper_state.json - Persisted scraper state
  • known_skus.json - Known product SKUs
  • proxies.json - Proxy configuration
  • user_locations.json - User store locations

Tools (tools/)

Development and debugging utilities:

  • api_monitor.py, backend_monitor.py - API monitoring
  • stealth_browser.py - Anti-detection browser
  • capture_api_calls.py - HAR capture for debugging

Key Classes

Product (scrapers/base.py)

@dataclass
class Product:
    name: str
    url: str
    price: Optional[str]
    in_stock: bool
    image_url: Optional[str]
    site: str
    product_id: Optional[str]

ProductTracker (src/product_tracker.py)

  • process_products(products) -> (new_products, restocked_products)
  • get_stats() -> dashboard statistics

BaseScraper (scrapers/base.py)

  • scrape_category_page(url) -> List[Product]
  • check_product_stock(url) -> (in_stock, price)

Common Tasks

Add a new retailer scraper

  1. Create scrapers/newsite.py extending BaseScraper
  2. Implement scrape_category_page() and check_product_stock()
  3. Add to scrapers/__init__.py
  4. Add config in config.py (URLs, enable flag)
  5. Add check function in main.py

Modify Discord notifications

  • Webhook format: src/discord_notifier.py
  • Bot commands: src/discord_bot.py

Change check intervals or URLs

  • Edit config.py

Dashboard changes

  • Routes: dashboard/app.py
  • API: dashboard/api.py
  • Frontend: dashboard/static/app.js

Chrome extension changes

  • Popup UI: chrome-extension/popup.html, popup.js
  • API interception: chrome-extension/api-interceptor.js
  • Background logic: chrome-extension/background.js

Conventions

  • Python 3.13+
  • Logging via logging module (configured in main.py)
  • Dataclasses for data structures
  • Flask for web/API
  • Playwright for browser automation
  • SQLite for persistence (stats.db)

Running

# Main monitor
python main.py

# Dashboard only
python dashboard/app.py

# Discord bot
python src/discord_bot.py

Testing

python -m pytest tests/