Files
pokemon-stock-checker/docs/BOT_DETECTION_STRATEGIES.md
T
mmcghen 8d382e723f feat: Implement Walmart scraper and integrate with existing architecture
- Added Walmart scraper to scrape product data from Walmart.com, including category pages and product details.
- Introduced a stealth browser module to handle bot protection and improve scraping reliability.
- Created a SQLite database for tracking product history, price changes, stock events, and user favorites.
- Developed a Discord bot for user interaction, allowing location setting and stock checking at local stores.
- Implemented a favorites system to manage priority products and categories with custom notification settings.
- Added news aggregation module to fetch and analyze Pokemon TCG news from various sources.
- Created tools for API discovery and monitoring, including a backend monitor for detecting new products.
- Added unit tests for database operations, product filtering, and API endpoints to ensure functionality.
- Enhanced existing modules with improved error handling and logging for better maintainability.
2026-03-27 23:08:09 -04:00

4.9 KiB

Pokemon Center Bot Detection - Strategies & Solutions

Pokemon Center uses Imperva (Incapsula) bot protection, which is one of the most aggressive anti-bot systems. Here are practical approaches to work around it.

Current Problem

  • Direct API calls: 403 Forbidden
  • Sitemap.xml: Blocked by Imperva
  • Browser automation: CAPTCHA after a few requests

Strategy 1: Third-Party Stock Trackers (Easiest)

Instead of scraping Pokemon Center directly, integrate with existing stock tracking services:

Discord Bots/Webhooks

Several Discord servers track Pokemon Center in real-time:

  • Pokemon TCG Drops - Dedicated Pokemon TCG stock alerts
  • Stock Informer - General retail stock tracking
  • Search Discord for "Pokemon Center stock alerts"

You can join these servers and set up webhook forwarding to your own Discord.

Stock Alert Services

  • NowInStock.net - Has Pokemon Center tracking
  • Distill.io - Browser extension for change detection
  • Visualping - Monitors page changes

Implementation

# Forward alerts from a public tracker to your system
# Set up a Discord bot to listen to stock alert channels
# Then forward to your own notification system

Strategy 2: Undetected Chrome Driver

Use undetected-chromedriver which patches Chrome to avoid detection:

pip install undetected-chromedriver
import undetected_chromedriver as uc

driver = uc.Chrome(headless=False)  # Headless often gets detected
driver.get("https://www.pokemoncenter.com/category/tcg-cards")

# Add human-like delays
import time
import random
time.sleep(random.uniform(3, 7))

# Scroll like a human
driver.execute_script("window.scrollBy(0, 500)")
time.sleep(random.uniform(1, 3))

Pros: Often bypasses Imperva Cons: Slower, still may trigger CAPTCHA eventually


Strategy 3: Session Persistence

Keep a browser session alive and logged in:

  1. Manual login once - Complete any CAPTCHA manually
  2. Save cookies - Export session cookies
  3. Reuse session - Load cookies on each check
  4. Longer intervals - Check every 3-5 minutes instead of 60-90 seconds
# Save cookies after manual login
import pickle
pickle.dump(driver.get_cookies(), open("cookies.pkl", "wb"))

# Load cookies on subsequent runs
cookies = pickle.load(open("cookies.pkl", "rb"))
for cookie in cookies:
    driver.add_cookie(cookie)

Strategy 4: Residential Proxy Rotation

Use residential proxies that look like real home internet connections:

Providers

  • Bright Data (formerly Luminati) - Best but expensive
  • Oxylabs - Good quality
  • Smartproxy - Budget option
  • IPRoyal - Pay-per-GB

Cost

  • ~$10-15/GB for residential proxies
  • ~50-100 requests per MB depending on page size
proxies = {
    "http": "http://user:pass@proxy.provider.com:port",
    "https": "http://user:pass@proxy.provider.com:port"
}

Strategy 5: Human-Like Behavior Pattern

If continuing with browser automation:

import random
import time

def human_like_check():
    # Random delay between checks (2-5 minutes)
    delay = random.uniform(120, 300)

    # Random user agents
    user_agents = [
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64)...",
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)...",
    ]

    # Vary timing throughout the day
    # Check more during business hours, less at night
    hour = datetime.now().hour
    if 2 <= hour <= 6:  # 2am-6am
        delay *= 2  # Slower at night

    # Add random mouse movements
    # Add random scrolling
    # Occasionally visit other pages (home, about)

    return delay

Strategy 6: Webhook from Manual Monitoring

The most reliable approach for Pokemon Center:

  1. Open Pokemon Center in your actual browser
  2. Use Distill.io browser extension to monitor changes
  3. Set up webhook to forward alerts to your system

This way:

  • You're using a real browser with real cookies
  • Imperva sees normal human behavior
  • Change detection triggers your notification system

Given Pokemon Center's aggressive protection, I recommend a hybrid approach:

  1. Primary: Join existing stock alert Discord servers

    • Let others deal with the bot detection
    • Forward their alerts to your system
  2. Secondary: Use undetected-chromedriver with long intervals

    • Check every 3-5 minutes
    • Use session persistence
    • Add human-like behavior
  3. Backup: Manual Distill.io monitoring

    • For specific products you really care about
    • Most reliable but requires keeping browser open

Code Changes Needed

To implement these strategies, we would need to:

  1. Add undetected-chromedriver as an option
  2. Implement cookie/session persistence
  3. Add configurable random delays
  4. Add proxy support
  5. Consider adding Discord bot listener for third-party alerts

Would you like me to implement any of these strategies?