feat: Implement Walmart scraper and integrate with existing architecture

- Added Walmart scraper to scrape product data from Walmart.com, including category pages and product details.
- Introduced a stealth browser module to handle bot protection and improve scraping reliability.
- Created a SQLite database for tracking product history, price changes, stock events, and user favorites.
- Developed a Discord bot for user interaction, allowing location setting and stock checking at local stores.
- Implemented a favorites system to manage priority products and categories with custom notification settings.
- Added news aggregation module to fetch and analyze Pokemon TCG news from various sources.
- Created tools for API discovery and monitoring, including a backend monitor for detecting new products.
- Added unit tests for database operations, product filtering, and API endpoints to ensure functionality.
- Enhanced existing modules with improved error handling and logging for better maintainability.

.gitignore

@@ -6,11 +6,26 @@ __pycache__/
 venv/
 .venv/
 
-# Logs and data
+# Logs
 *.log
+
+# Data files (runtime/generated)
 products.json
+data/sessions/
+data/*.json
+data/*.pkl
+*.pkl
+
+# Debug artifacts
 debug_*.png
 
+# Large analysis files
+*.har
+*_analysis.json
+
+# Database
+*.db
+
 # IDE
 .vscode/
 .idea/

CLAUDE.md

@@ -0,0 +1,137 @@
+# Pokemon Stock Monitor - Project Guide
+
+## Overview
+A Pokemon TCG stock monitoring system that tracks retail sites (Target, PokemonCenter, GameStop, Best Buy, Walmart) for restocks and new drops, sending Discord notifications when products become available.
+
+## Architecture
+
+```
+User -> Chrome Extension (PokemonCenter API interception)
+     -> Python Monitor (main.py) -> Scrapers -> ProductTracker -> Discord Notifications
+     -> Flask Dashboard (localhost:5000) -> REST API
+```
+
+## Directory Structure
+
+### Core Application
+- `main.py` - Entry point, scheduling loop, orchestrates scraping
+- `config.py` - All configuration (intervals, URLs, Discord webhook, browser settings)
+
+### Source Code (`src/`)
+- `browser.py` - Playwright/Chrome browser management
+- `product_tracker.py` - Detects new products and restocks, manages state
+- `discord_notifier.py` - Sends Discord webhook notifications
+- `discord_bot.py` - Interactive Discord bot commands
+- `database.py` - SQLite database for historical tracking
+- `favorites.py` - User favorites/watchlist management
+- `scraper_state.py` - Persists scraper state between runs
+- `store_locator.py` - Target store location lookup
+
+### Scrapers (`scrapers/`)
+- `base.py` - `Product` dataclass, `BaseScraper` abstract class
+- `pokemoncenter.py` - PokemonCenter scraper (needs stealth browser)
+- `target.py` - Target.com scraper
+- `gamestop.py` - GameStop.com scraper (light bot protection)
+- `bestbuy.py` - BestBuy.com scraper (moderate bot protection)
+- `walmart.py` - Walmart.com scraper (aggressive bot protection, may need stealth)
+
+### Chrome Extension (`chrome-extension/`)
+- `manifest.json` - Extension config (MV3)
+- `background.js` - Service worker, manages alarms and notifications
+- `content.js` - Injected into PokemonCenter pages
+- `api-interceptor.js` - Intercepts PokemonCenter API responses
+- `popup.html/js` - Extension popup UI
+
+### Dashboard (`dashboard/`)
+- `app.py` - Flask app, routes
+- `api.py` - REST API endpoints (`/api/*`)
+- `templates/index.html` - Main template
+- `static/app.js` - Frontend JavaScript
+
+### Data (`data/`)
+- `products.json` - Tracked products state
+- `scraper_state.json` - Persisted scraper state
+- `known_skus.json` - Known product SKUs
+- `proxies.json` - Proxy configuration
+- `user_locations.json` - User store locations
+
+### Tools (`tools/`)
+Development and debugging utilities:
+- `api_monitor.py`, `backend_monitor.py` - API monitoring
+- `stealth_browser.py` - Anti-detection browser
+- `capture_api_calls.py` - HAR capture for debugging
+
+## Key Classes
+
+### `Product` (scrapers/base.py)
+```python
+@dataclass
+class Product:
+    name: str
+    url: str
+    price: Optional[str]
+    in_stock: bool
+    image_url: Optional[str]
+    site: str
+    product_id: Optional[str]
+```
+
+### `ProductTracker` (src/product_tracker.py)
+- `process_products(products)` -> `(new_products, restocked_products)`
+- `get_stats()` -> dashboard statistics
+
+### `BaseScraper` (scrapers/base.py)
+- `scrape_category_page(url)` -> `List[Product]`
+- `check_product_stock(url)` -> `(in_stock, price)`
+
+## Common Tasks
+
+### Add a new retailer scraper
+1. Create `scrapers/newsite.py` extending `BaseScraper`
+2. Implement `scrape_category_page()` and `check_product_stock()`
+3. Add to `scrapers/__init__.py`
+4. Add config in `config.py` (URLs, enable flag)
+5. Add check function in `main.py`
+
+### Modify Discord notifications
+- Webhook format: `src/discord_notifier.py`
+- Bot commands: `src/discord_bot.py`
+
+### Change check intervals or URLs
+- Edit `config.py`
+
+### Dashboard changes
+- Routes: `dashboard/app.py`
+- API: `dashboard/api.py`
+- Frontend: `dashboard/static/app.js`
+
+### Chrome extension changes
+- Popup UI: `chrome-extension/popup.html`, `popup.js`
+- API interception: `chrome-extension/api-interceptor.js`
+- Background logic: `chrome-extension/background.js`
+
+## Conventions
+- Python 3.13+
+- Logging via `logging` module (configured in main.py)
+- Dataclasses for data structures
+- Flask for web/API
+- Playwright for browser automation
+- SQLite for persistence (`stats.db`)
+
+## Running
+
+```bash
+# Main monitor
+python main.py
+
+# Dashboard only
+python dashboard/app.py
+
+# Discord bot
+python src/discord_bot.py
+```
+
+## Testing
+```bash
+python -m pytest tests/
+```

chrome-extension/api-interceptor.js

@@ -0,0 +1,206 @@
+// API Interceptor - Captures Pokemon Center API responses
+// Injected into the page context to intercept fetch/XHR
+
+(function() {
+  'use strict';
+
+  // Avoid double injection
+  if (window.__pokemonMonitorInjected) return;
+  window.__pokemonMonitorInjected = true;
+
+  const API_PATTERNS = [
+    '/tpci-ecommweb-api/product/',
+    '/site/resourceapi/category/',
+    '/tpci-ecommweb-api/review/',
+    '/graphql',
+  ];
+
+  function isApiUrl(url) {
+    return API_PATTERNS.some(pattern => url.includes(pattern));
+  }
+
+  function extractSkusFromData(data, skus = new Set()) {
+    if (!data) return skus;
+
+    if (typeof data === 'object') {
+      // Look for SKU-like fields
+      for (const key of ['sku', 'skuCode', 'productId', 'id', 'code']) {
+        if (data[key] && typeof data[key] === 'string') {
+          const value = data[key];
+          // Pokemon Center SKUs: 699-17113, 191-85953, etc.
+          if (/^\d{1,3}-\d{4,6}$/.test(value) || /^\d{5,}$/.test(value)) {
+            skus.add(value);
+          }
+        }
+      }
+
+      // Recurse into nested objects/arrays
+      for (const value of Object.values(data)) {
+        if (typeof value === 'object') {
+          extractSkusFromData(value, skus);
+        }
+      }
+    }
+
+    if (Array.isArray(data)) {
+      for (const item of data) {
+        extractSkusFromData(item, skus);
+      }
+    }
+
+    return skus;
+  }
+
+  function sendToExtension(type, data) {
+    window.postMessage({
+      source: 'pokemon-api-interceptor',
+      type: type,
+      data: data
+    }, '*');
+  }
+
+  // Intercept fetch
+  const originalFetch = window.fetch;
+  window.fetch = async function(...args) {
+    const response = await originalFetch.apply(this, args);
+
+    try {
+      const url = typeof args[0] === 'string' ? args[0] : args[0]?.url || '';
+
+      if (isApiUrl(url) && response.ok) {
+        // Clone response so we can read it without consuming
+        const clone = response.clone();
+        const contentType = clone.headers.get('content-type') || '';
+
+        if (contentType.includes('application/json')) {
+          clone.json().then(data => {
+            const skus = extractSkusFromData(data);
+
+            sendToExtension('api-response', {
+              url: url,
+              timestamp: Date.now(),
+              skuCount: skus.size,
+              skus: Array.from(skus),
+              preview: JSON.stringify(data).slice(0, 500)
+            });
+          }).catch(() => {});
+        }
+      }
+    } catch (e) {
+      // Silently ignore errors
+    }
+
+    return response;
+  };
+
+  // Intercept XMLHttpRequest
+  const originalXhrOpen = XMLHttpRequest.prototype.open;
+  const originalXhrSend = XMLHttpRequest.prototype.send;
+
+  XMLHttpRequest.prototype.open = function(method, url, ...rest) {
+    this._monitorUrl = url;
+    return originalXhrOpen.apply(this, [method, url, ...rest]);
+  };
+
+  XMLHttpRequest.prototype.send = function(...args) {
+    const xhr = this;
+    const url = xhr._monitorUrl || '';
+
+    if (isApiUrl(url)) {
+      xhr.addEventListener('load', function() {
+        try {
+          if (xhr.status === 200) {
+            const contentType = xhr.getResponseHeader('content-type') || '';
+            if (contentType.includes('application/json')) {
+              const data = JSON.parse(xhr.responseText);
+              const skus = extractSkusFromData(data);
+
+              sendToExtension('api-response', {
+                url: url,
+                timestamp: Date.now(),
+                skuCount: skus.size,
+                skus: Array.from(skus),
+                preview: JSON.stringify(data).slice(0, 500)
+              });
+            }
+          }
+        } catch (e) {
+          // Silently ignore
+        }
+      });
+    }
+
+    return originalXhrSend.apply(this, args);
+  };
+
+  console.log('[Pokemon Monitor] API interceptor active');
+
+  // Also scan embedded page data for SKUs (handles server-side rendered pages)
+  function scanPageForSkus() {
+    const skus = new Set();
+
+    // Method 1: Extract from product URLs in the page
+    const productLinks = document.querySelectorAll('a[href*="/product/"]');
+    productLinks.forEach(link => {
+      const match = link.href.match(/\/product\/(\d{1,3}-\d{4,6})/);
+      if (match) {
+        skus.add(match[1]);
+      }
+    });
+
+    // Method 2: Look for __NEXT_DATA__ (Next.js embedded data)
+    const nextDataScript = document.getElementById('__NEXT_DATA__');
+    if (nextDataScript) {
+      try {
+        const data = JSON.parse(nextDataScript.textContent);
+        extractSkusFromData(data, skus);
+      } catch (e) {}
+    }
+
+    // Method 3: Look for any script tags with JSON containing product data
+    document.querySelectorAll('script[type="application/json"], script[type="application/ld+json"]').forEach(script => {
+      try {
+        const data = JSON.parse(script.textContent);
+        extractSkusFromData(data, skus);
+      } catch (e) {}
+    });
+
+    // Method 4: Look for data attributes on product elements
+    document.querySelectorAll('[data-sku], [data-product-id], [data-product-sku]').forEach(el => {
+      const sku = el.dataset.sku || el.dataset.productId || el.dataset.productSku;
+      if (sku && /^\d{1,3}-\d{4,6}$/.test(sku)) {
+        skus.add(sku);
+      }
+    });
+
+    if (skus.size > 0) {
+      console.log(`[Pokemon Monitor] Found ${skus.size} SKUs in page`);
+      sendToExtension('api-response', {
+        url: window.location.href,
+        timestamp: Date.now(),
+        skuCount: skus.size,
+        skus: Array.from(skus),
+        source: 'page-scan'
+      });
+    }
+  }
+
+  // Scan page after it loads
+  if (document.readyState === 'complete') {
+    setTimeout(scanPageForSkus, 1000);
+  } else {
+    window.addEventListener('load', () => setTimeout(scanPageForSkus, 1000));
+  }
+
+  // Re-scan when page content changes (for infinite scroll, etc.)
+  let scanTimeout;
+  const observer = new MutationObserver(() => {
+    clearTimeout(scanTimeout);
+    scanTimeout = setTimeout(scanPageForSkus, 2000);
+  });
+
+  // Start observing after initial load
+  setTimeout(() => {
+    observer.observe(document.body, { childList: true, subtree: true });
+  }, 3000);
+})();

chrome-extension/background.js

@@ -9,7 +9,9 @@ const DEFAULT_CONFIG = {
   ],
   keywords: [], // Empty = all products, or ["chaos rising", "booster"] etc
   notifyNewProducts: true,
-  notifyRestocks: true
+  notifyRestocks: true,
+  dashboardUrl: "http://localhost:5000",  // Local dashboard URL
+  syncToDashboard: true  // Enable syncing to local dashboard
 };
 
 // Store known products
@@ -20,11 +22,21 @@ let checkLoopRunning = false; // Prevent multiple loops
 let isChecking = false; // Prevent overlapping checks
 let lastNotificationTime = 0; // Rate limiting for Discord
 
+// API monitoring - track SKUs seen from backend API calls
+let knownSkus = new Set();
+let apiStats = {
+  lastApiResponse: null,
+  totalSkusTracked: 0,
+  newSkusDetected: 0,
+  apiResponseCount: 0
+};
+
 // Initialize
 chrome.runtime.onInstalled.addListener(() => {
   console.log("Pokemon Stock Monitor installed");
   loadConfig().then(() => {
     loadProducts();
+    loadKnownSkus();
     startCheckLoop();
   });
 });
@@ -33,6 +45,7 @@ chrome.runtime.onInstalled.addListener(() => {
 chrome.runtime.onStartup.addListener(() => {
   loadConfig().then(() => {
     loadProducts();
+    loadKnownSkus();
     startCheckLoop();
   });
 });
@@ -108,6 +121,97 @@ async function saveProducts() {
   await chrome.storage.local.set({ knownProducts });
 }
 
+// Load known SKUs from storage (for API monitoring)
+async function loadKnownSkus() {
+  const stored = await chrome.storage.local.get("knownSkus");
+  if (stored.knownSkus) {
+    knownSkus = new Set(stored.knownSkus);
+  }
+  console.log(`[API Monitor] Loaded ${knownSkus.size} known SKUs`);
+}
+
+// Save known SKUs to storage
+async function saveKnownSkus() {
+  await chrome.storage.local.set({ knownSkus: Array.from(knownSkus) });
+}
+
+// Handle API data from content script interceptor
+async function handleApiData(data) {
+  apiStats.apiResponseCount++;
+  apiStats.lastApiResponse = new Date().toISOString();
+
+  if (!data.skus || data.skus.length === 0) return;
+
+  const newSkus = [];
+
+  for (const sku of data.skus) {
+    if (!knownSkus.has(sku)) {
+      knownSkus.add(sku);
+      newSkus.push(sku);
+      apiStats.newSkusDetected++;
+    }
+  }
+
+  apiStats.totalSkusTracked = knownSkus.size;
+
+  if (newSkus.length > 0) {
+    console.log(`[API Monitor] NEW SKUs detected: ${newSkus.join(', ')}`);
+
+    // Send Discord alert for new SKUs
+    if (config.discordWebhook && config.notifyNewProducts) {
+      for (const sku of newSkus.slice(0, 3)) { // Limit to 3 to avoid spam
+        await sendSkuAlert(sku, data.url);
+        await new Promise(r => setTimeout(r, 1000));
+      }
+    }
+
+    await saveKnownSkus();
+  }
+}
+
+// Send alert for new SKU detected via API
+async function sendSkuAlert(sku, sourceUrl) {
+  const productUrl = `https://www.pokemoncenter.com/product/${sku}`;
+
+  const embed = {
+    title: "NEW SKU DETECTED (API)",
+    description: `**SKU: ${sku}**\n\nDetected in backend API before public listing!`,
+    url: productUrl,
+    color: 0xFF00FF, // Magenta for API detections
+    fields: [
+      { name: "SKU", value: sku, inline: true },
+      { name: "Source", value: "API Intercept", inline: true },
+      { name: "Link", value: `[VIEW PRODUCT](${productUrl})`, inline: false }
+    ],
+    footer: { text: "Pokemon Monitor - API Detection" },
+    timestamp: new Date().toISOString()
+  };
+
+  try {
+    const response = await fetch(config.discordWebhook, {
+      method: "POST",
+      headers: { "Content-Type": "application/json" },
+      body: JSON.stringify({
+        content: "@everyone NEW SKU FROM API!",
+        embeds: [embed]
+      })
+    });
+
+    if (response.ok) {
+      console.log(`[API Monitor] Discord alert sent for SKU: ${sku}`);
+
+      chrome.notifications.create({
+        type: "basic",
+        iconUrl: "icon128.png",
+        title: "NEW SKU DETECTED!",
+        message: `SKU: ${sku} - Check Discord!`
+      });
+    }
+  } catch (error) {
+    console.error("[API Monitor] Error sending alert:", error);
+  }
+}
+
 // Main stock check function
 async function runStockCheck() {
   if (!config.enabled) {
@@ -287,11 +391,55 @@ function processProducts(products) {
         lastSeen: now,
         lastInStock: product.inStock ? now : null
       };
+
+      // Queue new_drop event for dashboard sync
+      pendingEvents.push({
+        type: 'new_drop',
+        url: product.url,
+        name: product.name,
+        price: product.price,
+        inStock: product.inStock,
+        timestamp: now
+      });
     } else {
       // Existing product - check for restock
       if (product.inStock && !existing.inStock) {
         restockedProducts.push(product);
         existing.lastInStock = now;
+
+        // Queue restock event for dashboard sync
+        pendingEvents.push({
+          type: 'restock',
+          url: product.url,
+          name: product.name,
+          price: product.price,
+          previousPrice: existing.price,
+          timestamp: now
+        });
+      }
+
+      // Track when items go OUT of stock (for selling rate calculation)
+      if (!product.inStock && existing.inStock) {
+        pendingEvents.push({
+          type: 'out_of_stock',
+          url: product.url,
+          name: product.name,
+          price: product.price,
+          lastInStock: existing.lastInStock,  // When it came in stock
+          timestamp: now  // When it sold out
+        });
+      }
+
+      // Track price changes
+      if (product.price && existing.price && product.price !== existing.price) {
+        pendingEvents.push({
+          type: 'price_change',
+          url: product.url,
+          name: product.name,
+          oldPrice: existing.price,
+          newPrice: product.price,
+          timestamp: now
+        });
       }
 
       // Update
@@ -393,7 +541,69 @@ async function sendDiscordNotification(product, alertType) {
   }
 }
 
-// Listen for messages from popup
+// Track pending events to sync (restocks, new drops)
+let pendingEvents = [];
+
+// Sync data to local dashboard
+async function syncToDashboard() {
+  if (!config.syncToDashboard || !config.dashboardUrl) {
+    return { success: false, reason: 'sync disabled' };
+  }
+
+  // Build products array with all tracking data
+  const productsToSync = Object.values(knownProducts).map(p => ({
+    ...p,
+    site: 'pokemoncenter',
+    sku: p.productId || extractSkuFromUrl(p.url)
+  }));
+
+  const syncData = {
+    skus: Array.from(knownSkus),
+    products: productsToSync,
+    apiStats: apiStats,
+    events: pendingEvents  // Include pending events (restocks, new drops)
+  };
+
+  try {
+    const response = await fetch(`${config.dashboardUrl}/api/extension/sync`, {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify(syncData)
+    });
+
+    if (response.ok) {
+      const result = await response.json();
+      console.log(`[Dashboard Sync] Success - ${result.total_skus} SKUs, ${result.total_products} products`);
+      // Clear pending events after successful sync
+      pendingEvents = [];
+      return { success: true, ...result };
+    } else {
+      console.warn(`[Dashboard Sync] Failed: ${response.status}`);
+      return { success: false, status: response.status };
+    }
+  } catch (error) {
+    // Dashboard might not be running - silently fail
+    console.debug(`[Dashboard Sync] Dashboard not available: ${error.message}`);
+    return { success: false, error: error.message };
+  }
+}
+
+// Helper to extract SKU from Pokemon Center URL
+function extractSkuFromUrl(url) {
+  if (!url) return null;
+  const match = url.match(/\/product\/([^\/\?]+)/);
+  return match ? match[1] : null;
+}
+
+// Auto-sync to dashboard periodically (every 5 minutes)
+setInterval(() => {
+  if (config.syncToDashboard) {
+    syncToDashboard();
+  }
+}, 5 * 60 * 1000);
+
+
+// Listen for messages from popup and content scripts
 chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
   if (message.type === "getConfig") {
     sendResponse(config);
@@ -407,12 +617,27 @@ chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
   } else if (message.type === "getStats") {
     sendResponse({
       totalProducts: Object.keys(knownProducts).length,
-      enabled: config.enabled
+      enabled: config.enabled,
+      apiStats: apiStats,
+      knownSkuCount: knownSkus.size
     });
   } else if (message.type === "clearProducts") {
     knownProducts = {};
     saveProducts();
     sendResponse({ success: true });
+  } else if (message.type === "clearSkus") {
+    knownSkus = new Set();
+    apiStats.totalSkusTracked = 0;
+    saveKnownSkus();
+    sendResponse({ success: true });
+  } else if (message.type === "apiData") {
+    // Handle API data from content script interceptor
+    handleApiData(message.data);
+    sendResponse({ success: true });
+  } else if (message.type === "syncToDashboard") {
+    // Manual sync to local dashboard
+    syncToDashboard().then(result => sendResponse(result));
+    return true; // Keep channel open for async response
   }
   return true;
 });
@@ -420,7 +645,9 @@ chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
 // Run initial check after a short delay
 setTimeout(() => {
   loadConfig().then(() => {
-    loadProducts().then(() => {
+    loadProducts();
+    loadKnownSkus().then(() => {
+      console.log("[API Monitor] Ready - will intercept API calls when you browse Pokemon Center");
       if (config.enabled) {
         console.log(`Check interval: ${config.checkIntervalSeconds} seconds`);
         runStockCheck();

chrome-extension/content.js

@@ -1,9 +1,32 @@
 // Pokemon Stock Monitor - Content Script
-// Runs on PokemonCenter pages to extract product data
+// Runs on PokemonCenter pages to extract product data and intercept APIs
 
 (function() {
   console.log("[Pokemon Monitor] Content script loaded on:", window.location.href);
 
+  // Inject the API interceptor into the page context
+  function injectApiInterceptor() {
+    const script = document.createElement('script');
+    script.src = chrome.runtime.getURL('api-interceptor.js');
+    script.onload = () => script.remove();
+    (document.head || document.documentElement).appendChild(script);
+  }
+
+  // Listen for messages from the injected interceptor
+  window.addEventListener('message', (event) => {
+    if (event.source !== window) return;
+    if (event.data?.source !== 'pokemon-api-interceptor') return;
+
+    // Forward API data to background script
+    chrome.runtime.sendMessage({
+      type: 'apiData',
+      data: event.data.data
+    }).catch(() => {});
+  });
+
+  // Inject the interceptor
+  injectApiInterceptor();
+
   // Listen for messages from background script
   chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
     if (message.type === "extractProducts") {

chrome-extension/manifest.json

@@ -1,8 +1,8 @@
 {
   "manifest_version": 3,
   "name": "Pokemon Stock Monitor",
-  "version": "1.0.1",
+  "version": "1.1.0",
-  "description": "Monitors PokemonCenter for restocks and new drops, sends Discord notifications",
+  "description": "Monitors PokemonCenter for restocks and new drops via API interception, sends Discord notifications",
   "permissions": [
     "alarms",
     "storage",
@@ -24,7 +24,13 @@
     {
       "matches": ["https://www.pokemoncenter.com/*"],
       "js": ["content.js"],
-      "run_at": "document_idle"
+      "run_at": "document_start"
+    }
+  ],
+  "web_accessible_resources": [
+    {
+      "resources": ["api-interceptor.js"],
+      "matches": ["https://www.pokemoncenter.com/*"]
     }
   ]
 }

chrome-extension/popup.html

@@ -242,10 +242,24 @@
         <span class="slider"></span>
       </label>
     </div>
+    <div class="toggle-row">
+      <span>Sync to Dashboard</span>
+      <label class="toggle" for="syncDashboard">
+        <input type="checkbox" id="syncDashboard" checked>
+        <span class="slider"></span>
+      </label>
+    </div>
+  </div>
+
+  <div class="section">
+    <label>Dashboard URL</label>
+    <input type="text" id="dashboardUrl" placeholder="http://localhost:5000">
+    <small>Local dashboard for viewing data</small>
   </div>
 
   <button class="btn-primary" id="saveBtn">Save Settings</button>
   <button class="btn-secondary" id="checkNowBtn">Check Now</button>
+  <button class="btn-secondary" id="syncBtn">Sync to Dashboard</button>
   <button class="btn-danger" id="clearBtn">Clear Product History</button>
 
   <div class="saved-msg" id="savedMsg">Settings saved!</div>

chrome-extension/popup.js

@@ -13,6 +13,8 @@ document.addEventListener("DOMContentLoaded", async () => {
   document.getElementById("enabled").checked = config.enabled !== false;
   document.getElementById("notifyNew").checked = config.notifyNewProducts !== false;
   document.getElementById("notifyRestock").checked = config.notifyRestocks !== false;
+  document.getElementById("syncDashboard").checked = config.syncToDashboard !== false;
+  document.getElementById("dashboardUrl").value = config.dashboardUrl || "http://localhost:5000";
 
   // Update stats
   document.getElementById("productCount").textContent = stats.totalProducts || 0;
@@ -30,7 +32,9 @@ document.addEventListener("DOMContentLoaded", async () => {
       checkIntervalSeconds: parseInt(document.getElementById("interval").value) || 15,
       enabled: document.getElementById("enabled").checked,
       notifyNewProducts: document.getElementById("notifyNew").checked,
-      notifyRestocks: document.getElementById("notifyRestock").checked
+      notifyRestocks: document.getElementById("notifyRestock").checked,
+      syncToDashboard: document.getElementById("syncDashboard").checked,
+      dashboardUrl: document.getElementById("dashboardUrl").value.trim() || "http://localhost:5000"
     };
 
     await chrome.runtime.sendMessage({ type: "saveConfig", config: newConfig });
@@ -52,6 +56,16 @@ document.addEventListener("DOMContentLoaded", async () => {
       showSaved("History cleared!");
     }
   });
+
+  // Sync to Dashboard button
+  document.getElementById("syncBtn").addEventListener("click", async () => {
+    const result = await chrome.runtime.sendMessage({ type: "syncToDashboard" });
+    if (result.success) {
+      showSaved(`Synced! ${result.total_skus || 0} SKUs, ${result.total_products || 0} products`);
+    } else {
+      showSaved("Sync failed - is dashboard running?");
+    }
+  });
 });
 
 function updateStatus(enabled) {

config.py

@@ -5,14 +5,38 @@ Update DISCORD_WEBHOOK_URL with your actual webhook URL
 
 # Discord webhook URL - GET THIS FROM YOUR DISCORD SERVER
 # Server Settings -> Integrations -> Webhooks -> New Webhook -> Copy Webhook URL
-DISCORD_WEBHOOK_URL = "YOUR_WEBHOOK_URL_HERE"
+DISCORD_WEBHOOK_URL = "https://discord.com/api/webhooks/1471927173353963570/8Xh4Y_D8zRDi5MAP6ZlX0rf2Fc5wEtRKCup74kBZltk2qaatxArFp-yrk7ZWh5EOZbHp"
+
+# Enable/disable Discord notifications
+NOTIFICATIONS_ENABLED = True  # Set to False to disable all Discord alerts
+NEW_DROP_NOTIFICATIONS = True  # Set to False to silence new drop notifications
+SKIP_DUPLICATE_SKUS = True  # Skip notifications for products we've already notified about
+
+# Discord Bot Token (for interactive bot commands)
+# Create a bot at https://discord.com/developers/applications
+# Required for !setlocation, !stores, !stock commands
+DISCORD_BOT_TOKEN = "MTQ4NjQ2ODM2NDkyMTczNzM3OQ.Gxl8pq.wIvK2eVbp1G0SJhW4XTYD3zopR1gohaNv5sxlQ"
 
 # How often to check each site (in seconds)
 CHECK_INTERVAL_SECONDS = 60
 
+# Pagination settings - how many pages to scrape per site
+# Set to 1 for fastest checks, higher for more thorough scraping
+MAX_PAGES = {
+    "target": 1,      # Each page has ~24 products
+    "bestbuy": 1,
+    "gamestop": 1,
+    "walmart": 1,
+    "pokemoncenter": 1,
+}
+
+# Sort by newest - ensures new drops appear first
+SORT_BY_NEWEST = True
+
 # Which sites to monitor
 # Note: PokemonCenter has strong bot protection (Imperva) - may need manual workarounds
-SITES_ENABLED = ["target"]  # Options: "pokemoncenter", "target", "walmart", "bestbuy"
+# Note: Walmart has aggressive bot protection (PerimeterX) - may need stealth mode
+SITES_ENABLED = ["target", "bestbuy"]  # Options: "pokemoncenter", "target", "gamestop", "bestbuy" (disabled - needs rework), "walmart"
 
 # PokemonCenter URLs to monitor
 POKEMON_CENTER_URLS = [
@@ -24,6 +48,21 @@ TARGET_URLS = [
     "https://www.target.com/s?searchTerm=pokemon+tcg",
 ]
 
+# GameStop URLs to monitor
+GAMESTOP_URLS = [
+    "https://www.gamestop.com/search/?q=pokemon+tcg&lang=en_US",
+]
+
+# Best Buy URLs to monitor
+BESTBUY_URLS = [
+    "https://www.bestbuy.com/site/searchpage.jsp?st=pokemon+tcg",
+]
+
+# Walmart URLs to monitor
+WALMART_URLS = [
+    "https://www.walmart.com/search?q=pokemon+tcg",
+]
+
 # Keyword filtering (disabled by default)
 KEYWORD_FILTER_ENABLED = False
 KEYWORDS = [
@@ -46,10 +85,77 @@ SLOW_MO = 0  # Milliseconds to slow down browser actions (for debugging)
 # Leave as None to use a fresh browser profile
 # Set to None to use a fresh profile, or a path to use an existing profile
 # Note: Chrome must be closed when using an existing profile
-CHROME_USER_DATA_DIR = None
+CHROME_USER_DATA_DIR = None  # Disabled - causes issues with Playwright persistent context
 
 # Use real Chrome instead of Playwright's Chromium (helps with bot detection)
 USE_REAL_CHROME = True
 
+# =============================================================================
+# STEALTH MODE SETTINGS (for Pokemon Center and other protected sites)
+# =============================================================================
+
+# Use undetected-chromedriver instead of Playwright (better for Imperva/Cloudflare)
+USE_STEALTH_BROWSER = True
+
+# Stealth browser settings
+STEALTH_HEADLESS = False  # Keep False - headless is more detectable
+STEALTH_SESSION_NAME = "pokemoncenter"  # Name for cookie/session persistence
+
+# Human-like behavior settings
+MIN_CHECK_INTERVAL = 180  # Minimum seconds between checks for protected sites
+MAX_CHECK_INTERVAL = 300  # Maximum seconds (will randomize between min/max)
+HUMAN_DELAY_MIN = 2.0  # Minimum delay between actions (seconds)
+HUMAN_DELAY_MAX = 5.0  # Maximum delay between actions (seconds)
+
+# Proxy settings (optional but recommended for heavy use)
+USE_PROXIES = False  # Set to True to enable proxy rotation
+# Configure proxies in proxies.json file
+
+# Auto-solve CAPTCHA settings
+CAPTCHA_WAIT_TIMEOUT = 120  # Seconds to wait for manual CAPTCHA solve
+PAUSE_ON_CAPTCHA = True  # Pause monitoring when CAPTCHA detected (requires manual solve)
+
 # Logging
 LOG_LEVEL = "INFO"
+
+# Dashboard settings
+DASHBOARD_HOST = "0.0.0.0"  # Listen on all interfaces
+DASHBOARD_PORT = 5000
+DASHBOARD_DEBUG = False  # Set to True for development
+
+# Database settings
+DATABASE_PATH = None  # None = use default (stats.db in project root)
+
+# =============================================================================
+# NEWS AGGREGATION SETTINGS
+# =============================================================================
+
+# News sources configuration
+NEWS_CONFIG = {
+    'twitter': {
+        'enabled': False,  # Enable when you have API access
+        'bearer_token': 'YOUR_TWITTER_BEARER_TOKEN',  # Get from https://developer.twitter.com
+        'accounts': [
+            'pokepullzhq',       # Poke Pullz
+            'PokemonRestocks',   # Pokemon TCG Restocks & News
+            'PokemonDealsTCG',   # Pokemon Deals, Alerts & News!
+            'PokeNotifyX'        # PokeNotify
+        ],
+        'fetch_interval': 300  # 5 minutes (free tier: 1500 tweets/month)
+    },
+    'manual_sources': {
+        'enabled': True,
+        'sources': ['Poke Pullz Discord', 'PokePings Discord', 'Other']
+    },
+    'pokemon_official': {
+        'enabled': True,
+        'url': 'https://www.pokemon.com/us/pokemon-tcg-news/',
+        'fetch_interval': 3600  # 1 hour (official news updates less frequently)
+    }
+}
+
+# Auto-fetch news on dashboard start
+NEWS_AUTO_FETCH = True
+
+# Clean up old news articles after this many days
+NEWS_RETENTION_DAYS = 30

dashboard/__init__.py

@@ -0,0 +1,4 @@
+# Dashboard module
+from .app import run_dashboard
+
+__all__ = ['run_dashboard']

dashboard/app.py

@@ -0,0 +1,77 @@
+"""
+Flask dashboard for Pokemon Stock Monitor.
+Provides web UI for stats visualization and favorites management.
+"""
+
+import os
+import sys
+from pathlib import Path
+
+# Add parent directory to path for imports
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from flask import Flask, render_template, jsonify, request
+from flask_cors import CORS
+
+from src.database import get_database
+from src.favorites import get_favorites_manager
+from .api import api_bp
+
+# Create Flask app
+app = Flask(__name__)
+CORS(app)
+
+# Register API blueprint
+app.register_blueprint(api_bp, url_prefix='/api')
+
+
+@app.route('/')
+def dashboard():
+    """Main dashboard page"""
+    return render_template('index.html')
+
+
+@app.route('/products')
+def products_page():
+    """Products listing page"""
+    return render_template('index.html', page='products')
+
+
+@app.route('/news')
+def news_page():
+    """News aggregation page"""
+    return render_template('index.html', page='news')
+
+
+@app.route('/analytics')
+def analytics_page():
+    """Analytics page"""
+    return render_template('index.html', page='analytics')
+
+
+@app.route('/favorites')
+def favorites_page():
+    """Favorites management page"""
+    return render_template('index.html', page='favorites')
+
+
+@app.route('/settings')
+def settings_page():
+    """Settings page"""
+    return render_template('index.html', page='settings')
+
+
+@app.route('/users')
+def users_page():
+    """Users management page"""
+    return render_template('index.html', page='users')
+
+
+def run_dashboard(host: str = '0.0.0.0', port: int = 5000, debug: bool = False):
+    """Run the dashboard server"""
+    print(f"Starting Pokemon Stock Monitor Dashboard on http://{host}:{port}")
+    app.run(host=host, port=port, debug=debug, threaded=True)
+
+
+if __name__ == '__main__':
+    run_dashboard(debug=True)

dashboard/templates/index.html

@@ -0,0 +1,371 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>Pokemon Stock Monitor</title>
+  <link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
+  <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
+</head>
+<body>
+  <div class="app">
+    <!-- Sidebar Navigation -->
+    <nav class="sidebar">
+      <div class="logo">
+        <span class="logo-icon">&#9889;</span>
+        <span class="logo-text">Pokemon Monitor</span>
+      </div>
+      <ul class="nav-links">
+        <li><a href="#" data-page="dashboard" class="active">Dashboard</a></li>
+        <li><a href="#" data-page="products">Products</a></li>
+        <li><a href="#" data-page="news">News</a></li>
+        <li><a href="#" data-page="analytics">Analytics</a></li>
+        <li><a href="#" data-page="favorites">Favorites</a></li>
+        <li><a href="#" data-page="control">Control Panel</a></li>
+        <li><a href="#" data-page="users">Users</a></li>
+        <li><a href="#" data-page="settings">Settings</a></li>
+      </ul>
+      <div class="sidebar-footer">
+        <div class="status-indicator">
+          <span class="status-dot" id="monitorStatus"></span>
+          <span id="statusText">Loading...</span>
+        </div>
+      </div>
+    </nav>
+
+    <!-- Main Content -->
+    <main class="content">
+      <!-- Dashboard Page -->
+      <section id="page-dashboard" class="page active">
+        <h1>Dashboard</h1>
+
+        <!-- Store Cards -->
+        <div class="store-cards" id="storeCards">
+          <!-- Populated by JavaScript -->
+        </div>
+
+        <div class="stats-grid">
+          <div class="stat-card clickable" onclick="goToFilteredProducts('new_drop', 'today')">
+            <div class="stat-value" id="newDropsToday">-</div>
+            <div class="stat-label">New Drops Today</div>
+          </div>
+          <div class="stat-card clickable" onclick="goToFilteredProducts('restock', 'today')">
+            <div class="stat-value" id="restocksToday">-</div>
+            <div class="stat-label">Restocks Today</div>
+          </div>
+          <div class="stat-card clickable" onclick="goToFilteredProducts('new_drop', 'week')">
+            <div class="stat-value" id="newDropsWeek">-</div>
+            <div class="stat-label">New Drops (7 days)</div>
+          </div>
+          <div class="stat-card clickable" onclick="goToFilteredProducts('restock', 'week')">
+            <div class="stat-value" id="restocksWeek">-</div>
+            <div class="stat-label">Restocks (7 days)</div>
+          </div>
+        </div>
+
+        <div class="section">
+          <h2>Recent Activity</h2>
+          <div class="activity-feed" id="activityFeed">
+            <div class="loading">Loading...</div>
+          </div>
+        </div>
+      </section>
+
+      <!-- Products Page -->
+      <section id="page-products" class="page">
+        <h1>Products</h1>
+
+        <div id="eventFilterBadge" class="event-filter-badge" style="display: none;"></div>
+
+        <div class="filters">
+          <select id="filterSite">
+            <option value="">All Sites</option>
+            <option value="pokemoncenter">PokemonCenter</option>
+            <option value="target">Target</option>
+          </select>
+          <select id="filterCategory">
+            <option value="">All Categories</option>
+            <option value="ETB">ETB</option>
+            <option value="Booster Bundle">Booster Bundle</option>
+            <option value="Booster Box">Booster Box</option>
+            <option value="Booster Pack">Booster Pack</option>
+            <option value="Collection Box">Collection Box</option>
+            <option value="Tin">Tin</option>
+          </select>
+          <select id="filterStock">
+            <option value="">All Stock Status</option>
+            <option value="true">In Stock</option>
+            <option value="false">Out of Stock</option>
+          </select>
+          <label class="checkbox-label">
+            <input type="checkbox" id="filterFavorites">
+            Favorites Only
+          </label>
+        </div>
+
+        <div class="products-grid" id="productsGrid">
+          <div class="loading">Loading products...</div>
+        </div>
+      </section>
+
+      <!-- Analytics Page -->
+      <section id="page-analytics" class="page">
+        <h1>Analytics</h1>
+
+        <div class="charts-grid">
+          <div class="chart-card">
+            <h3>Drop Timing (by hour)</h3>
+            <canvas id="dropTimingChart"></canvas>
+          </div>
+          <div class="chart-card">
+            <h3>Stock Duration by Category</h3>
+            <canvas id="stockDurationChart"></canvas>
+          </div>
+          <div class="chart-card">
+            <h3>Products by Site</h3>
+            <canvas id="siteChart"></canvas>
+          </div>
+        </div>
+      </section>
+
+      <!-- News Page -->
+      <section id="page-news" class="page">
+        <h1>News</h1>
+
+        <!-- News Stats Overview -->
+        <div class="news-stats-bar" id="newsStatsBar">
+          <div class="sentiment-overview">
+            <span class="sentiment-label">Sentiment:</span>
+            <span class="sentiment-bar">
+              <span class="sentiment-positive" id="sentimentPositiveBar"></span>
+              <span class="sentiment-neutral" id="sentimentNeutralBar"></span>
+              <span class="sentiment-negative" id="sentimentNegativeBar"></span>
+            </span>
+            <span class="sentiment-text" id="sentimentText">Loading...</span>
+          </div>
+          <div class="news-count">
+            <span id="todayArticles">0</span> articles today
+          </div>
+        </div>
+
+        <!-- News Filters -->
+        <div class="filters news-filters">
+          <select id="filterNewsSource">
+            <option value="">All Sources</option>
+            <option value="twitter">Twitter</option>
+            <option value="pokemon_official">Pokemon.com</option>
+            <option value="discord_manual">Discord (Manual)</option>
+          </select>
+          <select id="filterNewsSentiment">
+            <option value="">All Sentiment</option>
+            <option value="positive">Positive</option>
+            <option value="neutral">Neutral</option>
+            <option value="negative">Negative</option>
+          </select>
+          <label class="checkbox-label">
+            <input type="checkbox" id="filterDropRelated">
+            Drop/Restock Related Only
+          </label>
+          <button id="refreshNewsBtn" class="btn-secondary">
+            <span class="btn-icon">&#8635;</span> Refresh News
+          </button>
+        </div>
+
+        <!-- Manual News Input -->
+        <div class="manual-news-section">
+          <details>
+            <summary>Add Manual News (Discord messages, etc.)</summary>
+            <div class="manual-news-form">
+              <div class="form-row">
+                <select id="manualNewsSource">
+                  <option value="Poke Pullz Discord">Poke Pullz Discord</option>
+                  <option value="PokePings Discord">PokePings Discord</option>
+                  <option value="Other">Other</option>
+                </select>
+                <input type="text" id="manualNewsTitle" placeholder="Title (optional)">
+              </div>
+              <textarea id="manualNewsContent" placeholder="Paste the message content here..." rows="3"></textarea>
+              <div class="form-row">
+                <input type="text" id="manualNewsUrl" placeholder="URL (optional)">
+                <button id="addManualNewsBtn" class="btn-primary">Add News</button>
+              </div>
+            </div>
+          </details>
+        </div>
+
+        <!-- News Feed -->
+        <div class="news-feed" id="newsFeed">
+          <div class="loading">Loading news...</div>
+        </div>
+      </section>
+
+      <!-- Favorites Page -->
+      <section id="page-favorites" class="page">
+        <h1>Favorites</h1>
+
+        <div class="favorites-section">
+          <div class="add-favorite-form">
+            <h3>Add New Favorite</h3>
+            <div class="form-row">
+              <select id="favType">
+                <option value="category">Category/Set</option>
+                <option value="product">Product URL</option>
+              </select>
+              <input type="text" id="favValue" placeholder="e.g., ETB, Chaos Rising, or product URL">
+              <select id="favPriority">
+                <option value="high">High Priority</option>
+                <option value="medium">Medium Priority</option>
+                <option value="low">Low Priority</option>
+              </select>
+              <button id="addFavoriteBtn" class="btn-primary">Add Favorite</button>
+            </div>
+            <div class="suggestions" id="suggestions"></div>
+          </div>
+
+          <h3>Your Favorites</h3>
+          <div class="favorites-list" id="favoritesList">
+            <div class="loading">Loading favorites...</div>
+          </div>
+        </div>
+      </section>
+
+      <!-- Control Panel Page -->
+      <section id="page-control" class="page">
+        <h1>Control Panel</h1>
+
+        <div class="control-panel">
+          <!-- Monitor Status Card -->
+          <div class="control-card monitor-status-card">
+            <div class="control-card-header">
+              <h3>Monitor Status</h3>
+              <span class="monitor-badge" id="monitorBadge">Stopped</span>
+            </div>
+            <div class="control-card-body">
+              <p id="monitorStatusText">Monitor is not running</p>
+              <p class="last-check-text">Last check: <span id="controlLastCheck">Never</span></p>
+            </div>
+            <div class="control-card-actions">
+              <button id="startMonitorBtn" class="btn-primary btn-large">
+                <span class="btn-icon">&#9654;</span> Start Monitor
+              </button>
+              <button id="stopMonitorBtn" class="btn-danger btn-large" disabled>
+                <span class="btn-icon">&#9632;</span> Stop Monitor
+              </button>
+            </div>
+          </div>
+
+          <!-- Check Interval -->
+          <div class="control-card">
+            <div class="control-card-header">
+              <h3>Check Interval</h3>
+            </div>
+            <div class="control-card-body">
+              <div class="interval-input">
+                <input type="number" id="checkIntervalInput" min="10" max="600" value="60">
+                <span>seconds</span>
+                <button id="saveIntervalBtn" class="btn-secondary">Save</button>
+              </div>
+              <p class="help-text">How often to check for new products (minimum 10 seconds)</p>
+            </div>
+          </div>
+
+          <!-- Scraper Controls -->
+          <div class="control-card scrapers-card">
+            <div class="control-card-header">
+              <h3>Scrapers</h3>
+            </div>
+            <div class="control-card-body">
+              <div class="scrapers-grid" id="scrapersGrid">
+                <!-- Populated by JavaScript -->
+              </div>
+            </div>
+          </div>
+
+          <!-- Quick Actions -->
+          <div class="control-card">
+            <div class="control-card-header">
+              <h3>Quick Actions</h3>
+            </div>
+            <div class="control-card-body">
+              <button id="runCheckNowBtn" class="btn-secondary">
+                <span class="btn-icon">&#8635;</span> Run Check Now
+              </button>
+              <button id="viewLogsBtn" class="btn-secondary">
+                <span class="btn-icon">&#128196;</span> View Logs
+              </button>
+            </div>
+          </div>
+        </div>
+      </section>
+
+      <!-- Users Page -->
+      <section id="page-users" class="page">
+        <h1>Users</h1>
+        <p class="help-text">Manage users and their store locations for local stock checking</p>
+
+        <div class="users-section">
+          <div class="add-user-form">
+            <h3>Add New User</h3>
+            <div class="form-group">
+              <label for="newUserName">Name</label>
+              <input type="text" id="newUserName" placeholder="e.g., John">
+            </div>
+            <div class="form-group">
+              <label for="newUserZip">Zip Code</label>
+              <input type="text" id="newUserZip" placeholder="e.g., 90210" maxlength="10">
+            </div>
+            <div class="form-group">
+              <label for="newUserRadius">Search Radius (miles)</label>
+              <input type="number" id="newUserRadius" value="25" min="5" max="100">
+            </div>
+            <button id="addUserBtn" class="btn-primary">Add User</button>
+          </div>
+
+          <h3>Current Users</h3>
+          <div id="usersList" class="users-list">
+            <!-- Populated by JavaScript -->
+            <p class="loading">Loading users...</p>
+          </div>
+        </div>
+      </section>
+
+      <!-- Settings Page -->
+      <section id="page-settings" class="page">
+        <h1>Settings</h1>
+
+        <div class="settings-section">
+          <h3>Discord Webhook</h3>
+          <p class="help-text">Configure your Discord webhook URL in config.py</p>
+
+          <h3>Monitor Status</h3>
+          <div id="monitorInfo">
+            <p>Last check: <span id="lastCheckTime">-</span></p>
+            <p>Next check: <span id="nextCheckTime">-</span></p>
+          </div>
+
+          <h3>Data Management</h3>
+          <button id="migrateBtn" class="btn-secondary">Migrate products.json to Database</button>
+          <button id="exportBtn" class="btn-secondary">Export Data</button>
+
+          <h3>Pokemon Center Cleanup</h3>
+          <p class="help-text">Remove Pokemon Center products with broken URLs (SKUs without product slugs)</p>
+          <button id="checkBrokenUrlsBtn" class="btn-secondary">Check Broken URLs</button>
+          <button id="cleanupBrokenUrlsBtn" class="btn-danger">Cleanup Broken URLs</button>
+          <p id="brokenUrlsResult" class="help-text"></p>
+        </div>
+      </section>
+    </main>
+  </div>
+
+  <!-- Add Favorite Modal -->
+  <div class="modal" id="productModal">
+    <div class="modal-content">
+      <span class="modal-close">&times;</span>
+      <h2 id="modalProductName">Product Details</h2>
+      <div id="modalProductDetails"></div>
+    </div>
+  </div>
+
+  <script src="{{ url_for('static', filename='app.js') }}"></script>
+</body>
+</html>

data/__init__.py

@@ -0,0 +1 @@
+# Data directory - contains runtime data files

discord_notifier.py

@@ -1,183 +0,0 @@
-"""
-Discord webhook notifications for stock alerts
-Sends rich embeds with product info and direct links
-"""
-
-import logging
-import requests
-from datetime import datetime
-from typing import Optional
-from config import DISCORD_WEBHOOK_URL
-
-logger = logging.getLogger(__name__)
-
-# Colors for different notification types
-COLOR_RESTOCK = 0x00FF00  # Green - item back in stock
-COLOR_NEW_DROP = 0x0099FF  # Blue - new product listing
-COLOR_PREORDER = 0xFFAA00  # Orange - pre-order available
-COLOR_ERROR = 0xFF0000  # Red - error notification
-
-# Site icons/emojis
-SITE_EMOJIS = {
-    "pokemoncenter": "\U0001F7E1",  # Yellow circle
-    "target": "\U0001F534",  # Red circle
-    "walmart": "\U0001F535",  # Blue circle
-    "bestbuy": "\U0001F7E1",  # Yellow circle
-}
-
-
-def send_stock_alert(
-    product_name: str,
-    product_url: str,
-    price: str,
-    site: str,
-    alert_type: str = "restock",
-    image_url: Optional[str] = None,
-):
-    """
-    Send a Discord notification for a stock alert
-
-    Args:
-        product_name: Name of the product
-        product_url: Direct link to the product
-        price: Price string (e.g., "$49.99")
-        site: Site name (pokemoncenter, target, etc.)
-        alert_type: "restock", "new_drop", or "preorder"
-        image_url: Optional product image URL
-    """
-    if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
-        logger.error("Discord webhook URL not configured! Update config.py")
-        return False
-
-    # Choose color based on alert type
-    if alert_type == "restock":
-        color = COLOR_RESTOCK
-        title = f"\U0001F6A8 RESTOCK ALERT"
-    elif alert_type == "new_drop":
-        color = COLOR_NEW_DROP
-        title = f"\U0001F195 NEW DROP"
-    elif alert_type == "preorder":
-        color = COLOR_PREORDER
-        title = f"\u23F0 PRE-ORDER AVAILABLE"
-    else:
-        color = COLOR_RESTOCK
-        title = f"\U0001F514 STOCK ALERT"
-
-    site_emoji = SITE_EMOJIS.get(site.lower(), "\U0001F6D2")
-    site_display = site.replace("pokemoncenter", "Pokemon Center").title()
-
-    # Build the embed
-    embed = {
-        "title": title,
-        "description": f"**{product_name}**",
-        "url": product_url,
-        "color": color,
-        "fields": [
-            {"name": "Price", "value": price or "See link", "inline": True},
-            {"name": "Store", "value": f"{site_emoji} {site_display}", "inline": True},
-            {"name": "Link", "value": f"[\U0001F6D2 BUY NOW]({product_url})", "inline": False},
-        ],
-        "footer": {"text": "Pokemon Stock Monitor"},
-        "timestamp": datetime.utcnow().isoformat(),
-    }
-
-    if image_url:
-        embed["thumbnail"] = {"url": image_url}
-
-    payload = {
-        "content": "@everyone",  # Ping everyone
-        "embeds": [embed],
-    }
-
-    try:
-        response = requests.post(
-            DISCORD_WEBHOOK_URL,
-            json=payload,
-            timeout=10,
-        )
-        response.raise_for_status()
-        logger.info(f"Discord notification sent for: {product_name}")
-        return True
-    except requests.exceptions.RequestException as e:
-        logger.error(f"Failed to send Discord notification: {e}")
-        return False
-
-
-def send_error_notification(error_message: str, site: str = "Unknown"):
-    """Send an error notification to Discord"""
-    if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
-        return False
-
-    embed = {
-        "title": "\u26A0\uFE0F Monitor Error",
-        "description": error_message,
-        "color": COLOR_ERROR,
-        "fields": [{"name": "Site", "value": site, "inline": True}],
-        "footer": {"text": "Pokemon Stock Monitor"},
-        "timestamp": datetime.utcnow().isoformat(),
-    }
-
-    payload = {"embeds": [embed]}
-
-    try:
-        response = requests.post(DISCORD_WEBHOOK_URL, json=payload, timeout=10)
-        response.raise_for_status()
-        return True
-    except requests.exceptions.RequestException as e:
-        logger.error(f"Failed to send error notification: {e}")
-        return False
-
-
-def send_startup_notification():
-    """Send a notification that the monitor has started"""
-    if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
-        return False
-
-    embed = {
-        "title": "\u2705 Monitor Started",
-        "description": "Pokemon Stock Monitor is now running and watching for restocks!",
-        "color": 0x00FF00,
-        "footer": {"text": "Pokemon Stock Monitor"},
-        "timestamp": datetime.utcnow().isoformat(),
-    }
-
-    payload = {"embeds": [embed]}
-
-    try:
-        response = requests.post(DISCORD_WEBHOOK_URL, json=payload, timeout=10)
-        response.raise_for_status()
-        return True
-    except requests.exceptions.RequestException as e:
-        logger.error(f"Failed to send startup notification: {e}")
-        return False
-
-
-def test_webhook():
-    """Test the Discord webhook connection"""
-    print("Testing Discord webhook...")
-
-    if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
-        print("ERROR: Webhook URL not configured!")
-        print("Edit config.py and set DISCORD_WEBHOOK_URL")
-        return False
-
-    # Send a test notification
-    success = send_stock_alert(
-        product_name="Test Product - Pokemon TCG Booster",
-        product_url="https://www.pokemoncenter.com/test",
-        price="$4.99",
-        site="pokemoncenter",
-        alert_type="restock",
-    )
-
-    if success:
-        print("SUCCESS! Check your Discord channel for the test message.")
-    else:
-        print("FAILED! Check the webhook URL and try again.")
-
-    return success
-
-
-if __name__ == "__main__":
-    # Run webhook test
-    test_webhook()

docs/BOT_DETECTION_STRATEGIES.md

@@ -0,0 +1,189 @@
+# Pokemon Center Bot Detection - Strategies & Solutions
+
+Pokemon Center uses **Imperva (Incapsula)** bot protection, which is one of the most aggressive anti-bot systems. Here are practical approaches to work around it.
+
+## Current Problem
+- Direct API calls: **403 Forbidden**
+- Sitemap.xml: **Blocked by Imperva**
+- Browser automation: **CAPTCHA after a few requests**
+
+---
+
+## Strategy 1: Third-Party Stock Trackers (Easiest)
+
+Instead of scraping Pokemon Center directly, integrate with existing stock tracking services:
+
+### Discord Bots/Webhooks
+Several Discord servers track Pokemon Center in real-time:
+- **Pokemon TCG Drops** - Dedicated Pokemon TCG stock alerts
+- **Stock Informer** - General retail stock tracking
+- Search Discord for "Pokemon Center stock alerts"
+
+You can join these servers and set up webhook forwarding to your own Discord.
+
+### Stock Alert Services
+- **NowInStock.net** - Has Pokemon Center tracking
+- **Distill.io** - Browser extension for change detection
+- **Visualping** - Monitors page changes
+
+### Implementation
+```python
+# Forward alerts from a public tracker to your system
+# Set up a Discord bot to listen to stock alert channels
+# Then forward to your own notification system
+```
+
+---
+
+## Strategy 2: Undetected Chrome Driver
+
+Use `undetected-chromedriver` which patches Chrome to avoid detection:
+
+```bash
+pip install undetected-chromedriver
+```
+
+```python
+import undetected_chromedriver as uc
+
+driver = uc.Chrome(headless=False)  # Headless often gets detected
+driver.get("https://www.pokemoncenter.com/category/tcg-cards")
+
+# Add human-like delays
+import time
+import random
+time.sleep(random.uniform(3, 7))
+
+# Scroll like a human
+driver.execute_script("window.scrollBy(0, 500)")
+time.sleep(random.uniform(1, 3))
+```
+
+**Pros:** Often bypasses Imperva
+**Cons:** Slower, still may trigger CAPTCHA eventually
+
+---
+
+## Strategy 3: Session Persistence
+
+Keep a browser session alive and logged in:
+
+1. **Manual login once** - Complete any CAPTCHA manually
+2. **Save cookies** - Export session cookies
+3. **Reuse session** - Load cookies on each check
+4. **Longer intervals** - Check every 3-5 minutes instead of 60-90 seconds
+
+```python
+# Save cookies after manual login
+import pickle
+pickle.dump(driver.get_cookies(), open("cookies.pkl", "wb"))
+
+# Load cookies on subsequent runs
+cookies = pickle.load(open("cookies.pkl", "rb"))
+for cookie in cookies:
+    driver.add_cookie(cookie)
+```
+
+---
+
+## Strategy 4: Residential Proxy Rotation
+
+Use residential proxies that look like real home internet connections:
+
+### Providers
+- **Bright Data** (formerly Luminati) - Best but expensive
+- **Oxylabs** - Good quality
+- **Smartproxy** - Budget option
+- **IPRoyal** - Pay-per-GB
+
+### Cost
+- ~$10-15/GB for residential proxies
+- ~50-100 requests per MB depending on page size
+
+```python
+proxies = {
+    "http": "http://user:pass@proxy.provider.com:port",
+    "https": "http://user:pass@proxy.provider.com:port"
+}
+```
+
+---
+
+## Strategy 5: Human-Like Behavior Pattern
+
+If continuing with browser automation:
+
+```python
+import random
+import time
+
+def human_like_check():
+    # Random delay between checks (2-5 minutes)
+    delay = random.uniform(120, 300)
+
+    # Random user agents
+    user_agents = [
+        "Mozilla/5.0 (Windows NT 10.0; Win64; x64)...",
+        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)...",
+    ]
+
+    # Vary timing throughout the day
+    # Check more during business hours, less at night
+    hour = datetime.now().hour
+    if 2 <= hour <= 6:  # 2am-6am
+        delay *= 2  # Slower at night
+
+    # Add random mouse movements
+    # Add random scrolling
+    # Occasionally visit other pages (home, about)
+
+    return delay
+```
+
+---
+
+## Strategy 6: Webhook from Manual Monitoring
+
+The most reliable approach for Pokemon Center:
+
+1. **Open Pokemon Center in your actual browser**
+2. **Use Distill.io browser extension** to monitor changes
+3. **Set up webhook** to forward alerts to your system
+
+This way:
+- You're using a real browser with real cookies
+- Imperva sees normal human behavior
+- Change detection triggers your notification system
+
+---
+
+## Recommended Approach
+
+Given Pokemon Center's aggressive protection, I recommend a **hybrid approach**:
+
+1. **Primary: Join existing stock alert Discord servers**
+   - Let others deal with the bot detection
+   - Forward their alerts to your system
+
+2. **Secondary: Use undetected-chromedriver with long intervals**
+   - Check every 3-5 minutes
+   - Use session persistence
+   - Add human-like behavior
+
+3. **Backup: Manual Distill.io monitoring**
+   - For specific products you really care about
+   - Most reliable but requires keeping browser open
+
+---
+
+## Code Changes Needed
+
+To implement these strategies, we would need to:
+
+1. Add `undetected-chromedriver` as an option
+2. Implement cookie/session persistence
+3. Add configurable random delays
+4. Add proxy support
+5. Consider adding Discord bot listener for third-party alerts
+
+Would you like me to implement any of these strategies?

docs/NOTES.md

@@ -0,0 +1,13 @@
+# Pokemon Stock Monitor - Notes
+
+## Bot Detection / CAPTCHA Log
+
+| Date | Time | Site | Notes |
+|------|------|------|-------|
+| 2026-03-25 | ~10:30 AM | PokemonCenter | CAPTCHAs appeared, bot protection triggered |
+
+## Tips
+- Pokemon Center uses Imperva bot protection
+- CAPTCHAs more likely during high traffic (product drops)
+- Using Chrome extension with real browser avoids most detection
+- If blocked, clear cookies and wait before retrying

docs/POKEMON_CENTER_DROPS_NOTES.md

@@ -0,0 +1,46 @@
+# Pokemon Center Drop Strategy Notes
+
+Source: https://www.pokepullz.ca/blog/pokemon-center-drops-guide.html
+
+## Queue System
+- Pokemon Center uses a **randomized virtual queue** that activates before major drops
+- Position is NOT based on when you arrived - it's assigned randomly once queue activates
+- This prevents site crashes and distributes access fairly
+
+## Timing Signals
+- Security changes on the Pokemon Center site serve as early warning signals that drops are imminent
+- Products sometimes load in waves rather than all at once
+
+## Multi-Device Strategy
+1. Join queue on primary device (desktop on WiFi) first
+2. Wait a few minutes, then join on second device (phone on mobile data)
+3. This gives you two separate queue positions and accounts for staggered product loading
+
+## Technical Tips
+
+### The Cart Trick
+When Add to Cart button isn't displaying during glitchy drops:
+1. Click the cart icon to go to your cart page
+2. Hit browser back button to return to product page
+3. This forces a page reload that may show the button
+
+## Order Flagging Risks (Avoid These!)
+- Typos in billing/shipping information
+- Mismatched address data
+- VPNs or browser extensions
+- Multiple rapid orders
+- Overloaded carts
+- Queue-checking tools or auto-refreshers
+
+## Pre-Drop Prep Checklist
+- [ ] Log into account days ahead
+- [ ] Have devices ready
+- [ ] Monitor Discord alerts
+- [ ] Disable browser extensions
+- [ ] Turn off VPN
+
+## Implications for Our Monitor
+1. **Don't use VPN/proxies during actual purchases** - only for monitoring
+2. **Queue position is random** - early detection still helps you GET in queue early
+3. **Watch for security changes** - could indicate imminent drop
+4. **Multi-wave loading** - may need multiple checks to catch all products

docs/SMART_MONITOR_DESIGN.md

@@ -0,0 +1,199 @@
+# Smart Monitor Design Document
+
+## Overview
+Design for an adaptive monitoring system that mimics human behavior and automatically switches modes based on detected activity.
+
+---
+
+## Monitoring Modes
+
+### 1. STEALTH MODE (Default)
+- **Check interval**: 45-90 seconds (randomized around 1 minute)
+- **Behavior**: Human-like patterns, decoy requests
+- **Purpose**: Avoid detection during normal monitoring
+
+### 2. ALERT MODE (Triggered by new SKU)
+- **Check interval**: 15-30 seconds (faster, still randomized)
+- **Duration**: 30-60 minutes after detection
+- **Purpose**: Catch the drop going live quickly
+
+### 3. COOLDOWN MODE (After alert period)
+- **Gradually slow back down** to stealth mode
+- **Prevents sudden behavior change** which could trigger detection
+
+### 4. SLEEP MODE (Optional - night hours)
+- **Check interval**: 5-10 minutes
+- **Active hours**: e.g., 2am-6am local time
+- **Purpose**: Real humans sleep, bots don't
+
+---
+
+## Human-Like Obfuscation Techniques
+
+### Timing Randomization
+```python
+# Instead of exact intervals:
+time.sleep(60)  # BAD - robotic
+
+# Use gaussian distribution around target:
+import random
+base_interval = 60
+jitter = random.gauss(0, 10)  # +/- 10 seconds standard deviation
+time.sleep(max(30, base_interval + jitter))  # GOOD - human-like
+```
+
+### Decoy Requests
+Mix in non-API requests to look like a real browser:
+- Occasionally fetch an image from the site
+- Load the homepage or a random category page
+- Request favicon, robots.txt, etc.
+
+```python
+DECOY_URLS = [
+    "/favicon.ico",
+    "/",
+    "/category/plush",
+    "/category/accessories",
+]
+
+def make_decoy_request():
+    """Occasionally make a non-API request"""
+    if random.random() < 0.2:  # 20% of the time
+        url = random.choice(DECOY_URLS)
+        session.get(API_BASE + url)
+```
+
+### Referrer Rotation
+Change the Referer header to look like you're browsing:
+```python
+REFERRERS = [
+    "https://www.pokemoncenter.com/",
+    "https://www.pokemoncenter.com/category/tcg-cards",
+    "https://www.pokemoncenter.com/category/new-releases",
+    "https://www.google.com/",
+]
+
+headers["Referer"] = random.choice(REFERRERS)
+```
+
+### User-Agent Variation
+Slightly vary the User-Agent (within reason):
+```python
+# Base UA with minor variations
+chrome_versions = ["146.0.0.0", "145.0.0.0", "146.0.7680.154"]
+version = random.choice(chrome_versions)
+headers["User-Agent"] = f"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/{version} Safari/537.36"
+```
+
+### Session Breaks
+Occasionally "take a break" like a human would:
+```python
+def maybe_take_break():
+    """Occasionally pause for a longer period"""
+    if random.random() < 0.05:  # 5% chance
+        break_duration = random.randint(120, 300)  # 2-5 minutes
+        logger.info(f"Taking a break for {break_duration}s...")
+        time.sleep(break_duration)
+```
+
+### Time-of-Day Awareness
+Adjust behavior based on time:
+```python
+from datetime import datetime
+
+def get_interval_for_time():
+    hour = datetime.now().hour
+
+    if 2 <= hour < 6:
+        # Late night - slower checks
+        return random.randint(300, 600)  # 5-10 min
+    elif 9 <= hour < 17:
+        # Business hours - prime drop time
+        return random.randint(45, 90)  # ~1 min
+    else:
+        # Evening/early morning
+        return random.randint(60, 120)  # 1-2 min
+```
+
+---
+
+## Mode Transition Logic
+
+```
+                    ┌─────────────────┐
+                    │  STEALTH MODE   │
+                    │  (45-90 sec)    │
+                    └────────┬────────┘
+                             │
+                    New SKU detected?
+                             │
+              YES ───────────┴─────────── NO
+               │                          │
+               ▼                          │
+    ┌─────────────────┐                   │
+    │   ALERT MODE    │                   │
+    │  (15-30 sec)    │◄──────────────────┘
+    └────────┬────────┘
+             │
+      30-60 min elapsed?
+             │
+          YES │
+             ▼
+    ┌─────────────────┐
+    │  COOLDOWN MODE  │
+    │  (gradually     │
+    │   slow down)    │
+    └────────┬────────┘
+             │
+             ▼
+    Back to STEALTH MODE
+```
+
+---
+
+## Detection Tracking
+
+When a new product is detected:
+
+1. **Log detection** with timestamp
+2. **Fetch full product details**
+3. **Send Discord alert**
+4. **Enter ALERT MODE**
+5. **Track if product becomes "announced"**:
+   - Visible on homepage
+   - Linked from category pages
+   - Social media announcement
+
+When product is announced, return to STEALTH MODE.
+
+---
+
+## Implementation Priority
+
+1. [x] Basic backend monitor (done)
+2. [ ] Add timing randomization
+3. [ ] Implement mode switching
+4. [ ] Add decoy requests
+5. [ ] Add time-of-day awareness
+6. [ ] Add session breaks
+7. [ ] Integrate with Discord notifier
+
+---
+
+## Testing Plan
+
+1. **Test with cookies**: Run warmup, verify API access works
+2. **Test SKU detection**: Manually add a fake SKU to known list, verify detection
+3. **Test mode switching**: Simulate detection, verify faster checks activate
+4. **Monitor for blocks**: Run for extended period, note when cookies expire
+
+---
+
+## Risk Mitigation
+
+| Risk | Mitigation |
+|------|------------|
+| Cookies expire | Auto-detect 403, prompt for re-warmup |
+| IP blocked | Support proxy rotation (already built) |
+| Pattern detected | Randomization + decoys + breaks |
+| Rate limited | Back off on 429 responses |

docs/backend-api-monitor.md

@@ -0,0 +1,285 @@
+# PokemonCenter Backend API Monitor
+
+## Overview
+
+This document outlines how to build a backend API monitor that detects new products before they appear on the website - similar to what accounts like @pokepullzhq do.
+
+## Why Backend Monitoring?
+
+- **Faster detection**: Products are loaded into the backend/API before the frontend displays them
+- **Less aggressive**: API calls are lighter than full page loads
+- **Avoids bot detection**: Direct API calls look different than browser automation
+- **More reliable**: JSON responses are easier to parse than HTML
+
+---
+
+## Step 1: Find the API Endpoints
+
+### How to Investigate
+
+1. Open https://www.pokemoncenter.com/category/tcg-cards in Chrome
+2. Open DevTools (F12)
+3. Go to **Network** tab
+4. Check **Preserve log**
+5. Filter by **Fetch/XHR**
+6. Refresh the page
+7. Look for API calls that return product data
+
+### What to Look For
+
+**Common patterns:**
+- GraphQL endpoints: `/graphql` or `/api/graphql`
+- REST APIs: `/api/products`, `/api/catalog`, `/api/search`
+- Third-party services:
+  - **Algolia** (search): `*.algolia.net` or `*.algolianet.com`
+  - **Contentful** (CMS): `cdn.contentful.com`
+  - **Commercetools** (e-commerce): `*.commercetools.com`
+  - **Salesforce Commerce**: `*.demandware.net`
+
+**Signs you found the right endpoint:**
+- Response contains product names, prices, SKUs
+- Response has `inStock`, `availability`, or similar fields
+- Response includes product URLs or IDs
+
+### Example Findings to Document
+
+For each endpoint found, note:
+```
+URL: https://api.pokemoncenter.com/products?category=tcg
+Method: GET
+Headers:
+  - Authorization: Bearer xxx (if any)
+  - x-api-key: xxx (if any)
+Response format: JSON
+Contains: productId, name, price, availability, url
+```
+
+---
+
+## Step 2: API Response Analysis
+
+Once you find the product API, analyze the response structure:
+
+### Key Fields to Track
+
+```json
+{
+  "products": [
+    {
+      "id": "12345",           // Unique product ID
+      "sku": "PKM-CR-ETB",     // SKU code
+      "name": "Chaos Rising ETB",
+      "url": "/product/chaos-rising-etb",
+      "price": 54.99,
+      "availability": {
+        "inStock": false,       // Current stock status
+        "preorder": true,       // Pre-order available?
+        "releaseDate": "2026-04-15"
+      },
+      "status": "ACTIVE",       // May be "HIDDEN", "DRAFT" before launch
+      "publishedAt": null       // null = not yet visible on site
+    }
+  ]
+}
+```
+
+### Detection Strategies
+
+1. **New Product Detection**
+   - Compare product IDs against known list
+   - New ID = new product added to backend
+
+2. **Pre-Launch Detection**
+   - Product exists but `publishedAt` is null
+   - Product has `status: "DRAFT"` or similar
+   - Product `availability.inStock` changes from false to true
+
+3. **Stock Change Detection**
+   - Track `inStock` or `availability` field changes
+
+---
+
+## Step 3: Implementation Plan
+
+### Option A: Add to Chrome Extension
+
+```javascript
+// In background.js - add API monitoring alongside page monitoring
+
+async function checkBackendAPI() {
+  const API_URL = "https://api.pokemoncenter.com/products?category=tcg";
+
+  try {
+    const response = await fetch(API_URL, {
+      headers: {
+        // Add any required headers discovered during investigation
+        'Accept': 'application/json',
+      }
+    });
+
+    const data = await response.json();
+    const products = data.products || [];
+
+    // Compare against known products
+    for (const product of products) {
+      if (!knownBackendProducts[product.id]) {
+        // NEW PRODUCT DETECTED!
+        await sendDiscordNotification({
+          name: product.name,
+          url: `https://www.pokemoncenter.com${product.url}`,
+          price: `$${product.price}`,
+          inStock: product.availability?.inStock
+        }, "backend_detect");
+
+        knownBackendProducts[product.id] = product;
+      }
+    }
+  } catch (error) {
+    console.error("Backend API check failed:", error);
+  }
+}
+```
+
+### Option B: Standalone Python Script
+
+```python
+# backend_monitor.py
+import requests
+import time
+import json
+from discord_webhook import DiscordWebhook, DiscordEmbed
+
+API_URL = "https://api.pokemoncenter.com/products"
+DISCORD_WEBHOOK = "your-webhook-url"
+KNOWN_PRODUCTS_FILE = "known_backend_products.json"
+CHECK_INTERVAL = 30  # seconds - can be faster for API
+
+def load_known_products():
+    try:
+        with open(KNOWN_PRODUCTS_FILE, 'r') as f:
+            return json.load(f)
+    except FileNotFoundError:
+        return {}
+
+def save_known_products(products):
+    with open(KNOWN_PRODUCTS_FILE, 'w') as f:
+        json.dump(products, f)
+
+def check_api():
+    headers = {
+        'User-Agent': 'Mozilla/5.0...',
+        # Add discovered headers
+    }
+
+    response = requests.get(API_URL, headers=headers)
+    return response.json()
+
+def send_alert(product, alert_type):
+    webhook = DiscordWebhook(url=DISCORD_WEBHOOK, content="@everyone")
+
+    embed = DiscordEmbed(
+        title=f"🚨 {alert_type.upper()}",
+        description=f"**{product['name']}**",
+        color=0xFF0000 if alert_type == "BACKEND DETECT" else 0x00FF00
+    )
+    embed.add_embed_field(name="Price", value=f"${product.get('price', 'TBD')}")
+    embed.add_embed_field(name="Status", value=product.get('status', 'Unknown'))
+    embed.add_embed_field(name="Link", value=f"[VIEW]({product['url']})", inline=False)
+
+    webhook.add_embed(embed)
+    webhook.execute()
+
+def main():
+    known = load_known_products()
+    print(f"Loaded {len(known)} known products")
+
+    while True:
+        try:
+            data = check_api()
+            products = data.get('products', [])
+
+            for product in products:
+                pid = product['id']
+
+                if pid not in known:
+                    print(f"NEW BACKEND PRODUCT: {product['name']}")
+                    send_alert(product, "BACKEND DETECT")
+                    known[pid] = product
+                    save_known_products(known)
+
+                # Check for status changes
+                elif known[pid].get('status') != product.get('status'):
+                    print(f"STATUS CHANGE: {product['name']}")
+                    send_alert(product, "STATUS CHANGE")
+                    known[pid] = product
+                    save_known_products(known)
+
+            print(f"Checked {len(products)} products")
+
+        except Exception as e:
+            print(f"Error: {e}")
+
+        time.sleep(CHECK_INTERVAL)
+
+if __name__ == "__main__":
+    main()
+```
+
+---
+
+## Step 4: Rate Limiting Considerations
+
+### API vs Page Scraping
+
+| Approach | Safe Interval | Notes |
+|----------|---------------|-------|
+| Full page load | 60-90 sec | Heavy, triggers bot detection |
+| API call | 15-30 sec | Lighter, more tolerant |
+| GraphQL query | 15-30 sec | Depends on complexity |
+
+### Best Practices
+
+1. **Use proper headers**: Include realistic User-Agent, Accept, etc.
+2. **Don't hammer**: Even APIs have rate limits
+3. **Cache responses**: Don't re-process unchanged data
+4. **Handle 429s gracefully**: Back off when rate limited
+
+---
+
+## Step 5: Advanced - Multiple Detection Layers
+
+For maximum coverage, run both:
+
+1. **Backend API Monitor** (every 30 sec)
+   - Fast detection of new products in system
+   - Lighter on resources
+
+2. **Frontend Page Monitor** (every 90 sec)
+   - Confirms products are live on website
+   - Catches anything API might miss
+
+### Alert Priority
+
+```
+BACKEND DETECT (API) = "Product loaded, drop imminent!"
+FRONTEND DETECT (Page) = "Product is LIVE, buy now!"
+RESTOCK = "Back in stock!"
+```
+
+---
+
+## Next Steps
+
+1. **Wait for rate limit to clear** (~30 min)
+2. **Investigate the API** using DevTools Network tab
+3. **Document the endpoints** you find
+4. **Share the findings** so we can build the monitor
+
+### Questions to Answer During Investigation
+
+- [ ] What URL serves product data?
+- [ ] What headers are required?
+- [ ] Is authentication needed?
+- [ ] What does the response structure look like?
+- [ ] Are there pagination parameters?
+- [ ] Is there a "hidden" or "draft" status visible in API?

main.py

@@ -16,14 +16,25 @@ from config import (
     SITES_ENABLED,
     POKEMON_CENTER_URLS,
     TARGET_URLS,
+    GAMESTOP_URLS,
+    BESTBUY_URLS,
+    WALMART_URLS,
     KEYWORD_FILTER_ENABLED,
     KEYWORDS,
     LOG_LEVEL,
+    NEW_DROP_NOTIFICATIONS,
+    MAX_PAGES,
+)
+from src.browser import get_browser, shutdown_browser
+from src.product_tracker import ProductTracker
+from src.discord_notifier import send_stock_alert, send_startup_notification, send_error_notification
+from scrapers import (
+    PokemonCenterScraper,
+    TargetScraper,
+    GameStopScraper,
+    BestBuyScraper,
+    WalmartScraper,
 )
-from browser import get_browser, shutdown_browser
-from product_tracker import ProductTracker
-from discord_notifier import send_stock_alert, send_startup_notification, send_error_notification
-from scrapers import PokemonCenterScraper, TargetScraper
 
 # Setup logging
 logging.basicConfig(
@@ -43,6 +54,9 @@ tracker = ProductTracker()
 scrapers = {
     "pokemoncenter": PokemonCenterScraper(),
     "target": TargetScraper(),
+    "gamestop": GameStopScraper(),
+    "bestbuy": BestBuyScraper(),
+    "walmart": WalmartScraper(),
 }
 
 
@@ -54,7 +68,7 @@ def check_pokemoncenter():
     try:
         for url in POKEMON_CENTER_URLS:
             logger.info(f"Scraping: {url}")
-            products = scraper.scrape_category_page(url)
+            products = scraper.scrape_category_page(url, max_pages=MAX_PAGES.get("pokemoncenter", 1))
 
             # Apply keyword filter if enabled
             if KEYWORD_FILTER_ENABLED and KEYWORDS:
@@ -64,7 +78,8 @@ def check_pokemoncenter():
             # Process and detect changes
             new_products, restocked_products = tracker.process_products(products)
 
-            # Send notifications for new products
+            # Send notifications for new products (if enabled)
+            if NEW_DROP_NOTIFICATIONS:
                 for product in new_products:
                     if product.in_stock:
                         send_stock_alert(
@@ -110,7 +125,10 @@ def check_target():
     try:
         for url in TARGET_URLS:
             logger.info(f"Scraping: {url}")
-            products = scraper.scrape_category_page(url)
+            products = scraper.scrape_category_page(url, max_pages=MAX_PAGES.get("target", 1))
+
+            # Always filter to Pokemon products only
+            products = scraper.filter_pokemon_products(products)
 
             # Apply keyword filter if enabled
             if KEYWORD_FILTER_ENABLED and KEYWORDS:
@@ -120,7 +138,8 @@ def check_target():
             # Process and detect changes
             new_products, restocked_products = tracker.process_products(products)
 
-            # Send notifications for new products
+            # Send notifications for new products (if enabled)
+            if NEW_DROP_NOTIFICATIONS:
                 for product in new_products:
                     if product.in_stock:
                         send_stock_alert(
@@ -158,6 +177,186 @@ def check_target():
         send_error_notification(f"Error checking Target: {str(e)}", "Target")
 
 
+def check_gamestop():
+    """Check GameStop for restocks and new drops"""
+    logger.info("Checking GameStop...")
+    scraper = scrapers["gamestop"]
+
+    try:
+        for url in GAMESTOP_URLS:
+            logger.info(f"Scraping: {url}")
+            products = scraper.scrape_category_page(url, max_pages=MAX_PAGES.get("gamestop", 1))
+
+            # Always filter to Pokemon products only
+            products = scraper.filter_pokemon_products(products)
+
+            # Apply keyword filter if enabled
+            if KEYWORD_FILTER_ENABLED and KEYWORDS:
+                products = scraper.filter_by_keywords(products, KEYWORDS)
+                logger.info(f"After keyword filter: {len(products)} products")
+
+            # Process and detect changes
+            new_products, restocked_products = tracker.process_products(products)
+
+            # Send notifications for new products (if enabled)
+            if NEW_DROP_NOTIFICATIONS:
+                for product in new_products:
+                    if product.in_stock:
+                        send_stock_alert(
+                            product_name=product.name,
+                            product_url=product.url,
+                            price=product.price or "See link",
+                            site="gamestop",
+                            alert_type="new_drop",
+                            image_url=product.image_url,
+                        )
+                        logger.info(f"Sent notification for new product: {product.name}")
+
+            # Send notifications for restocks
+            for product in restocked_products:
+                send_stock_alert(
+                    product_name=product.name,
+                    product_url=product.url,
+                    price=product.price or "See link",
+                    site="gamestop",
+                    alert_type="restock",
+                    image_url=product.image_url,
+                )
+                logger.info(f"Sent notification for restock: {product.name}")
+
+            # Log stats
+            stats = tracker.get_stats()
+            logger.info(
+                f"Stats: {stats['total_products']} total, "
+                f"{stats['in_stock']} in stock, "
+                f"{stats['out_of_stock']} out of stock"
+            )
+
+    except Exception as e:
+        logger.error(f"Error checking GameStop: {e}")
+        send_error_notification(f"Error checking GameStop: {str(e)}", "GameStop")
+
+
+def check_bestbuy():
+    """Check Best Buy for restocks and new drops"""
+    logger.info("Checking Best Buy...")
+    scraper = scrapers["bestbuy"]
+
+    try:
+        for url in BESTBUY_URLS:
+            logger.info(f"Scraping: {url}")
+            products = scraper.scrape_category_page(url, max_pages=MAX_PAGES.get("bestbuy", 1))
+
+            # Always filter to Pokemon products only
+            products = scraper.filter_pokemon_products(products)
+
+            # Apply keyword filter if enabled
+            if KEYWORD_FILTER_ENABLED and KEYWORDS:
+                products = scraper.filter_by_keywords(products, KEYWORDS)
+                logger.info(f"After keyword filter: {len(products)} products")
+
+            # Process and detect changes
+            new_products, restocked_products = tracker.process_products(products)
+
+            # Send notifications for new products (if enabled)
+            if NEW_DROP_NOTIFICATIONS:
+                for product in new_products:
+                    if product.in_stock:
+                        send_stock_alert(
+                            product_name=product.name,
+                            product_url=product.url,
+                            price=product.price or "See link",
+                            site="bestbuy",
+                            alert_type="new_drop",
+                            image_url=product.image_url,
+                        )
+                        logger.info(f"Sent notification for new product: {product.name}")
+
+            # Send notifications for restocks
+            for product in restocked_products:
+                send_stock_alert(
+                    product_name=product.name,
+                    product_url=product.url,
+                    price=product.price or "See link",
+                    site="bestbuy",
+                    alert_type="restock",
+                    image_url=product.image_url,
+                )
+                logger.info(f"Sent notification for restock: {product.name}")
+
+            # Log stats
+            stats = tracker.get_stats()
+            logger.info(
+                f"Stats: {stats['total_products']} total, "
+                f"{stats['in_stock']} in stock, "
+                f"{stats['out_of_stock']} out of stock"
+            )
+
+    except Exception as e:
+        logger.error(f"Error checking Best Buy: {e}")
+        send_error_notification(f"Error checking Best Buy: {str(e)}", "Best Buy")
+
+
+def check_walmart():
+    """Check Walmart for restocks and new drops"""
+    logger.info("Checking Walmart...")
+    scraper = scrapers["walmart"]
+
+    try:
+        for url in WALMART_URLS:
+            logger.info(f"Scraping: {url}")
+            products = scraper.scrape_category_page(url, max_pages=MAX_PAGES.get("walmart", 1))
+
+            # Always filter to Pokemon products only
+            products = scraper.filter_pokemon_products(products)
+
+            # Apply keyword filter if enabled
+            if KEYWORD_FILTER_ENABLED and KEYWORDS:
+                products = scraper.filter_by_keywords(products, KEYWORDS)
+                logger.info(f"After keyword filter: {len(products)} products")
+
+            # Process and detect changes
+            new_products, restocked_products = tracker.process_products(products)
+
+            # Send notifications for new products (if enabled)
+            if NEW_DROP_NOTIFICATIONS:
+                for product in new_products:
+                    if product.in_stock:
+                        send_stock_alert(
+                            product_name=product.name,
+                            product_url=product.url,
+                            price=product.price or "See link",
+                            site="walmart",
+                            alert_type="new_drop",
+                            image_url=product.image_url,
+                        )
+                        logger.info(f"Sent notification for new product: {product.name}")
+
+            # Send notifications for restocks
+            for product in restocked_products:
+                send_stock_alert(
+                    product_name=product.name,
+                    product_url=product.url,
+                    price=product.price or "See link",
+                    site="walmart",
+                    alert_type="restock",
+                    image_url=product.image_url,
+                )
+                logger.info(f"Sent notification for restock: {product.name}")
+
+            # Log stats
+            stats = tracker.get_stats()
+            logger.info(
+                f"Stats: {stats['total_products']} total, "
+                f"{stats['in_stock']} in stock, "
+                f"{stats['out_of_stock']} out of stock"
+            )
+
+    except Exception as e:
+        logger.error(f"Error checking Walmart: {e}")
+        send_error_notification(f"Error checking Walmart: {str(e)}", "Walmart")
+
+
 def run_checks():
     """Run all enabled site checks"""
     logger.info(f"Running checks at {datetime.now().strftime('%H:%M:%S')}")
@@ -168,6 +367,15 @@ def run_checks():
     if "target" in SITES_ENABLED:
         check_target()
 
+    if "gamestop" in SITES_ENABLED:
+        check_gamestop()
+
+    if "bestbuy" in SITES_ENABLED:
+        check_bestbuy()
+
+    if "walmart" in SITES_ENABLED:
+        check_walmart()
+
     logger.info("Check cycle complete")
 
 

product_tracker.py

@@ -1,136 +0,0 @@
-"""
-Product tracker - keeps track of known products to detect new drops and restocks
-"""
-
-import json
-import logging
-from pathlib import Path
-from typing import Dict, List, Set, Optional
-from dataclasses import asdict
-from datetime import datetime
-
-from scrapers.base import Product
-
-logger = logging.getLogger(__name__)
-
-# File to store known products
-PRODUCTS_FILE = Path(__file__).parent / "products.json"
-
-
-class ProductTracker:
-    """Tracks known products to detect new listings and stock changes"""
-
-    def __init__(self, products_file: Path = PRODUCTS_FILE):
-        self.products_file = products_file
-        self.products: Dict[str, dict] = {}  # URL -> product data
-        self.load()
-
-    def load(self):
-        """Load products from file"""
-        if self.products_file.exists():
-            try:
-                with open(self.products_file, "r", encoding="utf-8") as f:
-                    self.products = json.load(f)
-                logger.info(f"Loaded {len(self.products)} tracked products")
-            except (json.JSONDecodeError, IOError) as e:
-                logger.error(f"Error loading products file: {e}")
-                self.products = {}
-        else:
-            self.products = {}
-            logger.info("No existing products file, starting fresh")
-
-    def save(self):
-        """Save products to file"""
-        try:
-            with open(self.products_file, "w", encoding="utf-8") as f:
-                json.dump(self.products, f, indent=2, ensure_ascii=False)
-            logger.debug(f"Saved {len(self.products)} products")
-        except IOError as e:
-            logger.error(f"Error saving products file: {e}")
-
-    def process_products(self, products: List[Product]) -> tuple[List[Product], List[Product]]:
-        """
-        Process a list of scraped products and detect changes
-
-        Args:
-            products: List of products from scraper
-
-        Returns:
-            Tuple of (new_products, restocked_products)
-        """
-        new_products = []
-        restocked_products = []
-
-        for product in products:
-            url = product.url
-
-            if url not in self.products:
-                # New product!
-                new_products.append(product)
-                self.products[url] = {
-                    "name": product.name,
-                    "url": url,
-                    "price": product.price,
-                    "in_stock": product.in_stock,
-                    "image_url": product.image_url,
-                    "site": product.site,
-                    "product_id": product.product_id,
-                    "first_seen": datetime.now().isoformat(),
-                    "last_seen": datetime.now().isoformat(),
-                    "last_in_stock": datetime.now().isoformat() if product.in_stock else None,
-                }
-                logger.info(f"NEW PRODUCT: {product.name}")
-
-            else:
-                # Existing product - check for restock
-                existing = self.products[url]
-                was_in_stock = existing.get("in_stock", False)
-
-                # Update last seen
-                existing["last_seen"] = datetime.now().isoformat()
-                existing["price"] = product.price or existing.get("price")
-                existing["image_url"] = product.image_url or existing.get("image_url")
-
-                if product.in_stock and not was_in_stock:
-                    # RESTOCK!
-                    restocked_products.append(product)
-                    existing["last_in_stock"] = datetime.now().isoformat()
-                    logger.info(f"RESTOCK: {product.name}")
-
-                existing["in_stock"] = product.in_stock
-                self.products[url] = existing
-
-        self.save()
-        return new_products, restocked_products
-
-    def get_known_urls(self) -> Set[str]:
-        """Get all known product URLs"""
-        return set(self.products.keys())
-
-    def get_product(self, url: str) -> Optional[dict]:
-        """Get a specific product by URL"""
-        return self.products.get(url)
-
-    def mark_out_of_stock(self, url: str):
-        """Mark a product as out of stock"""
-        if url in self.products:
-            self.products[url]["in_stock"] = False
-            self.save()
-
-    def clear(self):
-        """Clear all tracked products"""
-        self.products = {}
-        self.save()
-        logger.info("Cleared all tracked products")
-
-    def get_stats(self) -> dict:
-        """Get tracking statistics"""
-        total = len(self.products)
-        in_stock = sum(1 for p in self.products.values() if p.get("in_stock", False))
-        out_of_stock = total - in_stock
-
-        return {
-            "total_products": total,
-            "in_stock": in_stock,
-            "out_of_stock": out_of_stock,
-        }

pytest.ini

@@ -0,0 +1,9 @@
+[pytest]
+testpaths = tests
+python_files = test_*.py
+python_classes = Test*
+python_functions = test_*
+addopts = -v --tb=short
+filterwarnings =
+    ignore::DeprecationWarning
+    ignore::PendingDeprecationWarning

requirements.txt

@@ -3,3 +3,13 @@ playwright-stealth>=1.0.6
 requests>=2.31.0
 schedule>=1.2.1
 beautifulsoup4>=4.12.0
+flask>=3.0.0
+flask-cors>=4.0.0
+psutil>=5.9.0
+undetected-chromedriver>=3.5.0
+selenium>=4.15.0
+setuptools>=70.0.0  # Required for Python 3.13+ (distutils compatibility)
+discord.py>=2.3.0  # Discord bot for location-based store search
+nltk>=3.8.0  # Sentiment analysis using VADER
+pytest>=8.0.0  # Testing framework
+pytest-cov>=4.0.0  # Coverage reporting

scrapers/__init__.py

@@ -1,5 +1,15 @@
 # Scrapers package
 from .pokemoncenter import PokemonCenterScraper
 from .target import TargetScraper
+from .gamestop import GameStopScraper, warmup_gamestop
+from .bestbuy import BestBuyScraper
+from .walmart import WalmartScraper
 
-__all__ = ["PokemonCenterScraper", "TargetScraper"]
+__all__ = [
+    "PokemonCenterScraper",
+    "TargetScraper",
+    "GameStopScraper",
+    "BestBuyScraper",
+    "WalmartScraper",
+    "warmup_gamestop",
+]

scrapers/base.py

@@ -35,6 +35,50 @@ class BaseScraper(ABC):
 
     site_name: str = "unknown"
 
+    # Terms that indicate a Pokemon product
+    POKEMON_TERMS = [
+        "pokemon", "pokémon", "poke", "tcg",
+        "pikachu", "charizard", "mewtwo", "eevee", "snorlax",
+        "booster", "elite trainer", "etb",
+        "scarlet", "violet", "prismatic", "evolutions",
+    ]
+
+    # Terms that indicate NOT a Pokemon product (false positives from search)
+    EXCLUDE_TERMS = [
+        "ice cube", "oven", "barbie", "hot wheels", "lego",
+        "furniture", "appliance", "kitchen", "bedding",
+        "glitter girls", "masters of the universe", "transformers",
+        "room essentials", "threshold",
+    ]
+
+    def is_pokemon_product(self, product: Product) -> bool:
+        """
+        Check if a product is actually a Pokemon product.
+        Filters out false positives from search results.
+        """
+        name_lower = product.name.lower()
+
+        # Check for exclusion terms first
+        for term in self.EXCLUDE_TERMS:
+            if term in name_lower:
+                return False
+
+        # Check for Pokemon terms
+        for term in self.POKEMON_TERMS:
+            if term in name_lower:
+                return True
+
+        # If no Pokemon terms found, reject it
+        return False
+
+    def filter_pokemon_products(self, products: List[Product]) -> List[Product]:
+        """Filter to only include valid Pokemon products"""
+        filtered = [p for p in products if self.is_pokemon_product(p)]
+        rejected = len(products) - len(filtered)
+        if rejected > 0:
+            logger.info(f"Filtered out {rejected} non-Pokemon products")
+        return filtered
+
     @abstractmethod
     def scrape_category_page(self, url: str) -> List[Product]:
         """

scrapers/bestbuy.py

@@ -0,0 +1,505 @@
+"""
+Best Buy scraper - Uses undetected-chromedriver to bypass bot protection
+"""
+
+import re
+import json
+import logging
+import time
+import sys
+import os
+from typing import List, Optional
+from bs4 import BeautifulSoup
+
+from .base import BaseScraper, Product
+
+# Add parent directory to path for tools import
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+logger = logging.getLogger(__name__)
+
+
+class BestBuyScraper(BaseScraper):
+    """Scraper for BestBuy.com - Uses undetected-chromedriver"""
+
+    site_name = "bestbuy"
+    base_url = "https://www.bestbuy.com"
+
+    def __init__(self):
+        self._stealth_browser = None
+
+    def _get_stealth_browser(self):
+        """Get or create stealth browser for Best Buy"""
+        if self._stealth_browser is None:
+            from tools.stealth_browser import StealthBrowser
+            self._stealth_browser = StealthBrowser(headless=False, session_name="bestbuy")
+            self._stealth_browser.start()
+        return self._stealth_browser
+
+    def scrape_category_page(self, url: str, max_pages: int = 1) -> List[Product]:
+        """
+        Scrape a Best Buy search/category page for all products
+        Uses undetected-chromedriver to bypass bot protection
+
+        Args:
+            url: Search/category page URL
+            max_pages: Maximum number of pages to scrape (default 1)
+
+        Returns:
+            List of Product objects
+        """
+        all_products = []
+
+        # Add sort by newest if not already in URL
+        if "sort=" not in url:
+            separator = "&" if "?" in url else "?"
+            url = f"{url}{separator}sp=-releasedate"
+
+        # Use stealth browser
+        try:
+            browser = self._get_stealth_browser()
+        except Exception as e:
+            logger.error(f"Failed to start stealth browser: {e}")
+            return all_products
+
+        for page_num in range(1, max_pages + 1):
+            products = []
+            page_url = url if page_num == 1 else f"{url}&cp={page_num}"
+
+            try:
+                logger.info(f"Navigating to: {page_url}")
+                browser.driver.get(page_url)
+                time.sleep(8)  # Wait longer for initial page load
+
+                # Scroll for lazy loading - wait longer between scrolls
+                browser.driver.execute_script("window.scrollTo(0, 500)")
+                time.sleep(2)
+                browser.driver.execute_script("window.scrollTo(0, 1500)")
+                time.sleep(2)
+                browser.driver.execute_script("window.scrollTo(0, 3000)")
+                time.sleep(2)
+                browser.driver.execute_script("window.scrollTo(0, 0)")  # Scroll back to top
+                time.sleep(3)
+
+                # Wait for products to load (check for content)
+                for _ in range(10):
+                    html = browser.driver.page_source
+                    if "sku-item" in html or "sku-title" in html or "priceView" in html:
+                        logger.info("Product content detected")
+                        break
+                    time.sleep(1)
+
+                # Get HTML after waiting
+                html = browser.driver.page_source
+
+                # Save debug screenshot
+                try:
+                    browser.driver.save_screenshot("debug_bestbuy.png")
+                    logger.info("Saved debug screenshot to debug_bestbuy.png")
+                except:
+                    pass
+
+                soup = BeautifulSoup(html, "html.parser")
+
+                # Try to find product data in JSON scripts
+                scripts = soup.find_all("script", type="application/json")
+                for script in scripts:
+                    try:
+                        data = json.loads(script.string)
+                        products.extend(self._extract_products_from_json(data))
+                    except (json.JSONDecodeError, TypeError):
+                        continue
+
+                # Try multiple selectors for product cards - Best Buy updates these frequently
+                product_cards = (
+                    soup.select("li.sku-item")
+                    or soup.select("[data-sku-id]")
+                    or soup.select(".sku-item")
+                    or soup.select("[class*='sku-item']")
+                    or soup.select("div.shop-sku-list-item")
+                    or soup.select("[class*='productCard']")
+                    or soup.select("[class*='product-card']")
+                    or soup.select(".list-item")
+                )
+
+                logger.info(f"Found {len(product_cards)} product cards on page {page_num}")
+
+                for card in product_cards:
+                    product = self._parse_product_card(card)
+                    if product:
+                        products.append(product)
+
+                # Fallback: parse product links
+                if not product_cards:
+                    product_links = soup.select("a[href*='/site/'][href*='.p']")
+                    logger.info(f"Fallback: Found {len(product_links)} product links")
+
+                    href_to_links = {}
+                    for link in product_links:
+                        href = link.get("href", "")
+                        if not href or ".p" not in href:
+                            continue
+                        if href not in href_to_links:
+                            href_to_links[href] = []
+                        href_to_links[href].append(link)
+
+                    for href, links in href_to_links.items():
+                        best_link = links[0]
+                        for link in links:
+                            if link.get("aria-label") or len(link.get_text(strip=True)) > 10:
+                                best_link = link
+                                break
+
+                        product = self._parse_product_link(best_link, soup)
+                        if product:
+                            products.append(product)
+
+                    # Also try parsing from data attributes and script tags
+                    if not products:
+                        products.extend(self._extract_from_page_data(soup, browser))
+
+                # Don't close - reuse browser for next page
+
+            except Exception as e:
+                logger.error(f"Error scraping Best Buy page {page_num}: {e}")
+                break  # Don't raise - just return what we have
+
+            all_products.extend(products)
+
+            # Stop if no products found on this page (no more pages)
+            if not products:
+                break
+
+        # Remove duplicates
+        seen_urls = set()
+        unique_products = []
+        for p in all_products:
+            if p.url not in seen_urls:
+                seen_urls.add(p.url)
+                unique_products.append(p)
+
+        logger.info(f"Scraped {len(unique_products)} unique products from Best Buy")
+        return unique_products
+
+    def _extract_from_page_data(self, soup, browser) -> List[Product]:
+        """Extract products from page data attributes and evaluate JS if needed"""
+        products = []
+
+        # Try to get product data from data attributes
+        items_with_data = soup.select("[data-testid][data-sku-id]")
+        for item in items_with_data:
+            sku_id = item.get("data-sku-id", "")
+            if sku_id:
+                # Find name and price within this element
+                name_elem = item.select_one("h4") or item.select_one("[class*='title']") or item.select_one("a")
+                name = name_elem.get_text(strip=True) if name_elem else ""
+
+                if name and len(name) > 5:
+                    price_text = item.get_text()
+                    price_match = re.search(r"\$[\d,]+\.?\d*", price_text)
+                    price = price_match.group() if price_match else None
+
+                    products.append(Product(
+                        name=name,
+                        url=f"{self.base_url}/site/{sku_id}.p",
+                        price=price,
+                        in_stock=True,
+                        image_url=None,
+                        site=self.site_name,
+                        product_id=sku_id,
+                    ))
+
+        # Try to extract from window.__INITIAL_STATE__ or similar JS objects
+        try:
+            initial_state = browser.driver.execute_script("""
+                if (window.__INITIAL_STATE__) return JSON.stringify(window.__INITIAL_STATE__);
+                if (window.__NEXT_DATA__) return JSON.stringify(window.__NEXT_DATA__);
+                return null;
+            """)
+            if initial_state:
+                data = json.loads(initial_state)
+                products.extend(self._extract_products_from_json(data))
+        except Exception as e:
+            logger.debug(f"Could not extract from JS state: {e}")
+
+        return products
+
+    def _extract_products_from_json(self, data, depth=0) -> List[Product]:
+        """Recursively search JSON for product data"""
+        products = []
+        if depth > 10:
+            return products
+
+        if isinstance(data, dict):
+            # Check if this looks like a Best Buy product
+            if "skuId" in data or ("name" in data and "regularPrice" in data):
+                product = self._parse_product_json(data)
+                if product:
+                    products.append(product)
+
+            for value in data.values():
+                products.extend(self._extract_products_from_json(value, depth + 1))
+
+        elif isinstance(data, list):
+            for item in data:
+                products.extend(self._extract_products_from_json(item, depth + 1))
+
+        return products
+
+    def _parse_product_json(self, data: dict) -> Optional[Product]:
+        """Parse a product from Best Buy's JSON data"""
+        try:
+            name = data.get("name") or data.get("displayName", "")
+            if not name:
+                return None
+
+            sku_id = data.get("skuId") or data.get("sku", "")
+            url_slug = data.get("url") or ""
+
+            if url_slug:
+                url = url_slug if url_slug.startswith("http") else f"{self.base_url}{url_slug}"
+            elif sku_id:
+                url = f"{self.base_url}/site/{sku_id}.p"
+            else:
+                return None
+
+            # Get price
+            price = None
+            if "regularPrice" in data:
+                price = f"${data['regularPrice']:.2f}"
+            elif "salePrice" in data:
+                price = f"${data['salePrice']:.2f}"
+
+            # Check availability
+            in_stock = True
+            availability = data.get("availability", {})
+            if isinstance(availability, dict):
+                in_stock = availability.get("isAvailable", True)
+            elif data.get("orderable") is False:
+                in_stock = False
+
+            # Get image
+            image_url = data.get("image") or data.get("thumbnailImage")
+
+            return Product(
+                name=name,
+                url=url,
+                price=price,
+                in_stock=in_stock,
+                image_url=image_url,
+                site=self.site_name,
+                product_id=str(sku_id),
+            )
+
+        except Exception as e:
+            logger.debug(f"Error parsing Best Buy product JSON: {e}")
+            return None
+
+    def _parse_product_card(self, card) -> Optional[Product]:
+        """Parse a product card element"""
+        try:
+            # Find link
+            link = card.select_one("a[href*='/site/'][href*='.p']") or card.select_one("a.image-link")
+            if not link:
+                link = card.select_one("a")
+            if not link:
+                return None
+
+            href = link.get("href", "")
+            if not href:
+                return None
+            url = href if href.startswith("http") else f"{self.base_url}{href}"
+
+            # Get name
+            name_elem = (
+                card.select_one(".sku-title a")
+                or card.select_one("h4.sku-header a")
+                or card.select_one("[data-testid='product-title']")
+                or card.select_one(".sku-title")
+                or link
+            )
+            name = name_elem.get_text(strip=True) if name_elem else "Unknown"
+
+            # Skip if name too short
+            if len(name) < 5:
+                return None
+
+            # Get price
+            price_elem = (
+                card.select_one(".priceView-customer-price span")
+                or card.select_one("[data-testid='customer-price']")
+                or card.select_one(".pricing-price__regular-price")
+                or card.select_one("[class*='price']")
+            )
+            price = None
+            if price_elem:
+                price_match = re.search(r"\$[\d,]+\.?\d*", price_elem.get_text())
+                if price_match:
+                    price = price_match.group()
+
+            # Check stock
+            in_stock = self._check_card_stock_status(card)
+
+            # Get image
+            img = card.select_one("img")
+            image_url = None
+            if img:
+                image_url = img.get("src") or img.get("data-src")
+                if image_url and not image_url.startswith("http"):
+                    image_url = f"https:{image_url}" if image_url.startswith("//") else f"{self.base_url}{image_url}"
+
+            # Extract SKU ID from URL
+            product_id = ""
+            match = re.search(r"/(\d+)\.p", url)
+            if match:
+                product_id = match.group(1)
+
+            return Product(
+                name=name,
+                url=url,
+                price=price,
+                in_stock=in_stock,
+                image_url=image_url,
+                site=self.site_name,
+                product_id=product_id,
+            )
+
+        except Exception as e:
+            logger.debug(f"Error parsing Best Buy product card: {e}")
+            return None
+
+    def _parse_product_link(self, link, soup) -> Optional[Product]:
+        """Parse a product from a product link element"""
+        try:
+            href = link.get("href", "")
+            if not href:
+                return None
+
+            url = href if href.startswith("http") else f"{self.base_url}{href}"
+
+            name = link.get("aria-label", "") or link.get_text(strip=True)
+
+            if not name or len(name) < 5:
+                return None
+
+            # Extract SKU ID
+            product_id = ""
+            match = re.search(r"/(\d+)\.p", url)
+            if match:
+                product_id = match.group(1)
+
+            # Find price near link
+            parent = link.find_parent()
+            price = None
+            for _ in range(5):
+                if parent:
+                    price_text = parent.get_text()
+                    price_match = re.search(r"\$[\d,]+\.?\d*", price_text)
+                    if price_match:
+                        price = price_match.group()
+                        break
+                    parent = parent.find_parent()
+
+            # Find image
+            image_url = None
+            parent = link.find_parent()
+            for _ in range(5):
+                if parent:
+                    img = parent.select_one("img")
+                    if img:
+                        image_url = img.get("src") or img.get("data-src")
+                        break
+                    parent = parent.find_parent()
+
+            # Check stock
+            in_stock = True
+            parent = link.find_parent()
+            for _ in range(5):
+                if parent:
+                    text = parent.get_text().lower()
+                    if any(phrase in text for phrase in ["sold out", "out of stock", "unavailable"]):
+                        in_stock = False
+                        break
+                    parent = parent.find_parent()
+
+            return Product(
+                name=name,
+                url=url,
+                price=price,
+                in_stock=in_stock,
+                image_url=image_url,
+                site=self.site_name,
+                product_id=product_id,
+            )
+
+        except Exception as e:
+            logger.debug(f"Error parsing Best Buy product link: {e}")
+            return None
+
+    def _check_card_stock_status(self, card) -> bool:
+        """Check if a product card indicates in-stock status"""
+        card_text = card.get_text().lower() if hasattr(card, "get_text") else str(card).lower()
+
+        # Check for disabled add to cart button
+        add_btn = card.select_one(".add-to-cart-button")
+        if add_btn and "btn-disabled" in add_btn.get("class", []):
+            return False
+
+        out_of_stock_phrases = [
+            "sold out",
+            "out of stock",
+            "unavailable",
+            "coming soon",
+            "not available",
+        ]
+
+        for phrase in out_of_stock_phrases:
+            if phrase in card_text:
+                return False
+
+        in_stock_phrases = [
+            "add to cart",
+            "add to bag",
+            "available",
+            "in stock",
+        ]
+
+        for phrase in in_stock_phrases:
+            if phrase in card_text:
+                return True
+
+        return True
+
+    def check_product_stock(self, product_url: str) -> tuple[bool, Optional[str]]:
+        """Check if a specific product is in stock"""
+        try:
+            browser = self._get_stealth_browser()
+            browser.driver.get(product_url)
+            time.sleep(3)
+            html = browser.driver.page_source
+            soup = BeautifulSoup(html, "html.parser")
+            page_text = soup.get_text().lower()
+
+            out_of_stock = any(
+                phrase in page_text
+                for phrase in ["sold out", "out of stock", "unavailable", "coming soon"]
+            )
+
+            in_stock = not out_of_stock
+
+            # Get price
+            price = None
+            price_elem = (
+                soup.select_one(".priceView-customer-price span")
+                or soup.select_one("[data-testid='customer-price']")
+            )
+            if price_elem:
+                price_match = re.search(r"\$[\d,]+\.?\d*", price_elem.get_text())
+                if price_match:
+                    price = price_match.group()
+
+            return in_stock, price
+
+        except Exception as e:
+            logger.error(f"Error checking Best Buy product stock: {e}")
+            return False, None

scrapers/gamestop.py

@@ -0,0 +1,478 @@
+"""
+GameStop.com scraper
+"""
+
+import re
+import logging
+import time
+import sys
+import os
+from typing import List, Optional
+from bs4 import BeautifulSoup
+
+from .base import BaseScraper, Product
+from src.browser import get_browser
+from config import CAPTCHA_WAIT_TIMEOUT
+
+# Add parent directory to path for tools import
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+logger = logging.getLogger(__name__)
+
+
+def warmup_gamestop():
+    """
+    Warmup function to solve Cloudflare CAPTCHA manually.
+    Uses undetected-chromedriver to bypass bot detection.
+    """
+    from tools.stealth_browser import StealthBrowser
+
+    print("\n" + "=" * 60)
+    print("GAMESTOP WARMUP - Using Undetected Chrome")
+    print("=" * 60)
+    print("A browser window should open.")
+    print("If you see a Cloudflare challenge, solve it manually.")
+    print(f"Waiting up to {CAPTCHA_WAIT_TIMEOUT} seconds...")
+    print("=" * 60 + "\n")
+
+    logger.info("Starting GameStop warmup with undetected-chromedriver...")
+
+    # Use stealth browser instead of Playwright
+    browser = StealthBrowser(headless=False, session_name="gamestop")
+
+    try:
+        browser.start()
+        logger.info("Stealth browser started")
+
+        # Navigate to GameStop
+        browser.driver.get("https://www.gamestop.com")
+        time.sleep(3)
+
+        # Check for Cloudflare challenge
+        start_time = time.time()
+        challenge_detected = False
+
+        while time.time() - start_time < CAPTCHA_WAIT_TIMEOUT:
+            try:
+                html = browser.driver.page_source
+                html_lower = html.lower()
+                title = browser.driver.title.lower()
+
+                # Cloudflare challenge indicators
+                is_cloudflare_challenge = (
+                    "just a moment" in title or
+                    "checking your browser" in html_lower or
+                    "cf-challenge" in html_lower or
+                    "turnstile" in html_lower or
+                    (len(html) < 5000 and "challenge" in html_lower)
+                )
+
+                if is_cloudflare_challenge:
+                    if not challenge_detected:
+                        challenge_detected = True
+                        print(">>> Cloudflare challenge detected! Please solve it in the browser window.")
+                    logger.info("Cloudflare challenge detected - waiting for manual solve...")
+                    time.sleep(5)
+                else:
+                    # Check if we're on actual GameStop content
+                    if len(html) > 10000 and "gamestop" in html_lower:
+                        print(">>> Challenge solved! GameStop page loaded successfully.")
+                        logger.info("GameStop warmup complete!")
+                        browser.stop()
+                        return True
+                    time.sleep(2)
+            except Exception as e:
+                logger.debug(f"Error checking page: {e}")
+                time.sleep(2)
+
+        logger.warning("GameStop warmup timed out - CAPTCHA may not be solved")
+        print(">>> Warmup timed out. You may need to try again.")
+        browser.stop()
+        return False
+
+    except Exception as e:
+        logger.error(f"Error during warmup: {e}")
+        try:
+            browser.stop()
+        except:
+            pass
+        return False
+
+
+class GameStopScraper(BaseScraper):
+    """Scraper for GameStop.com - Uses undetected-chromedriver to bypass Cloudflare"""
+
+    site_name = "gamestop"
+    base_url = "https://www.gamestop.com"
+
+    def __init__(self):
+        self._stealth_browser = None
+
+    def _get_stealth_browser(self):
+        """Get or create stealth browser for GameStop"""
+        if self._stealth_browser is None:
+            from tools.stealth_browser import StealthBrowser
+            self._stealth_browser = StealthBrowser(headless=False, session_name="gamestop")
+            self._stealth_browser.start()
+        return self._stealth_browser
+
+    def scrape_category_page(self, url: str, max_pages: int = 1) -> List[Product]:
+        """
+        Scrape a GameStop search/category page for all products
+        Uses undetected-chromedriver to bypass Cloudflare
+
+        Args:
+            url: Search/category page URL
+            max_pages: Maximum number of pages to scrape (default 1)
+
+        Returns:
+            List of Product objects
+        """
+        all_products = []
+
+        # Add sort by newest if not already in URL
+        if "sort=" not in url.lower():
+            separator = "&" if "?" in url else "?"
+            url = f"{url}{separator}sort=date-desc"
+
+        # Use stealth browser for GameStop
+        try:
+            browser = self._get_stealth_browser()
+        except Exception as e:
+            logger.error(f"Failed to start stealth browser: {e}")
+            return all_products
+
+        for page_num in range(1, max_pages + 1):
+            products = []
+            page_url = url if page_num == 1 else f"{url}&start={24 * (page_num - 1)}"
+
+            try:
+                logger.info(f"Navigating to: {page_url}")
+                browser.driver.get(page_url)
+                time.sleep(5)  # Wait for page load
+
+                html = browser.driver.page_source
+
+                # Check for Cloudflare challenge
+                html_lower = html.lower()
+                title = browser.driver.title.lower()
+
+                is_cloudflare = (
+                    "just a moment" in title or
+                    "checking your browser" in html_lower or
+                    "cf-challenge" in html_lower
+                )
+
+                if is_cloudflare:
+                    logger.warning("Cloudflare challenge detected! Waiting for manual solve...")
+                    start_time = time.time()
+                    while time.time() - start_time < CAPTCHA_WAIT_TIMEOUT:
+                        time.sleep(5)
+                        html = browser.driver.page_source
+                        html_lower = html.lower()
+                        title = browser.driver.title.lower()
+                        if "just a moment" not in title and "cf-challenge" not in html_lower:
+                            logger.info("Cloudflare challenge solved!")
+                            break
+                    else:
+                        logger.error("Cloudflare challenge not solved in time")
+                        return all_products
+
+                # Save debug screenshot
+                try:
+                    browser.driver.save_screenshot("debug_gamestop.png")
+                    logger.info("Saved debug screenshot to debug_gamestop.png")
+                except:
+                    pass
+
+                soup = BeautifulSoup(html, "html.parser")
+
+                # Try multiple selectors for product cards
+                product_cards = (
+                    soup.select(".product-tile")
+                    or soup.select("[data-testid='product-tile']")
+                    or soup.select(".grid-tile")
+                    or soup.select(".product-grid-tile")
+                    or soup.select("[class*='ProductTile']")
+                )
+
+                logger.info(f"Found {len(product_cards)} product cards")
+
+                # If no cards found, try link-based extraction
+                if not product_cards:
+                    product_links = soup.select("a[href*='/products/']")
+                    logger.info(f"Fallback: Found {len(product_links)} product links")
+
+                    # Group by href
+                    href_to_links = {}
+                    for link in product_links:
+                        href = link.get("href", "")
+                        if not href or "/products/" not in href:
+                            continue
+                        if href not in href_to_links:
+                            href_to_links[href] = []
+                        href_to_links[href].append(link)
+
+                    for href, links in href_to_links.items():
+                        # Pick best link
+                        best_link = links[0]
+                        for link in links:
+                            if link.get("aria-label") or len(link.get_text(strip=True)) > 10:
+                                best_link = link
+                                break
+
+                        product = self._parse_product_link(best_link, soup)
+                        if product:
+                            products.append(product)
+                else:
+                    for card in product_cards:
+                        product = self._parse_product_card(card)
+                        if product:
+                            products.append(product)
+
+                # Don't close - reuse browser for next page
+
+            except Exception as e:
+                logger.error(f"Error scraping GameStop page {page_num}: {e}")
+                break  # Don't raise - just return what we have
+
+            all_products.extend(products)
+
+            # Stop if no products found on this page
+            if not products:
+                break
+
+        # Remove duplicates
+        seen_urls = set()
+        unique_products = []
+        for p in all_products:
+            if p.url not in seen_urls:
+                seen_urls.add(p.url)
+                unique_products.append(p)
+
+        logger.info(f"Scraped {len(unique_products)} unique products from GameStop")
+        return unique_products
+
+    def _parse_product_card(self, card) -> Optional[Product]:
+        """Parse a product card element"""
+        try:
+            # Find link - try multiple patterns
+            link = (
+                card.select_one("a[href*='/products/']") or
+                card.select_one("a[href*='/video-games/']") or
+                card.select_one("a[href*='/collectibles/']") or
+                card.select_one("a.product-tile-link") or
+                card.select_one("a")
+            )
+            if not link:
+                logger.debug("No link found in card")
+                return None
+
+            href = link.get("href", "")
+            if not href:
+                logger.debug("Empty href")
+                return None
+            url = href if href.startswith("http") else f"{self.base_url}{href}"
+
+            # Get name - try multiple patterns
+            name_elem = (
+                card.select_one(".product-tile-title")
+                or card.select_one(".product-name a")
+                or card.select_one(".product-name")
+                or card.select_one(".product-title")
+                or card.select_one("[data-testid='product-name']")
+                or card.select_one("a[aria-label]")
+                or link
+            )
+
+            # Try to get name from aria-label first
+            name = ""
+            if name_elem:
+                name = name_elem.get("aria-label", "") or name_elem.get_text(strip=True)
+
+            # Skip if name too short
+            if len(name) < 5:
+                logger.debug(f"Name too short: '{name}' from {url[:50]}")
+                return None
+
+            # Get price
+            price_elem = (
+                card.select_one(".price-sales")
+                or card.select_one(".product-price")
+                or card.select_one("[data-testid='price']")
+                or card.select_one("[class*='price']")
+            )
+            price = None
+            if price_elem:
+                price_match = re.search(r"\$[\d,]+\.?\d*", price_elem.get_text())
+                if price_match:
+                    price = price_match.group()
+
+            # Check stock
+            in_stock = self._check_card_stock_status(card)
+
+            # Get image
+            img = card.select_one("img")
+            image_url = None
+            if img:
+                image_url = img.get("src") or img.get("data-src") or img.get("data-lazy")
+                if image_url and not image_url.startswith("http"):
+                    image_url = f"{self.base_url}{image_url}"
+
+            # Extract product ID from URL
+            product_id = ""
+            match = re.search(r"/products/[^/]+/(\d+)", url)
+            if match:
+                product_id = match.group(1)
+
+            return Product(
+                name=name,
+                url=url,
+                price=price,
+                in_stock=in_stock,
+                image_url=image_url,
+                site=self.site_name,
+                product_id=product_id,
+            )
+
+        except Exception as e:
+            logger.debug(f"Error parsing GameStop product card: {e}")
+            return None
+
+    def _parse_product_link(self, link, soup) -> Optional[Product]:
+        """Parse a product from a product link element"""
+        try:
+            href = link.get("href", "")
+            if not href:
+                return None
+
+            url = href if href.startswith("http") else f"{self.base_url}{href}"
+
+            # Get name
+            name = link.get("aria-label", "") or link.get_text(strip=True)
+
+            # Skip if name too short
+            if not name or len(name) < 5:
+                return None
+
+            # Extract product ID from URL
+            product_id = ""
+            match = re.search(r"/products/[^/]+/(\d+)", url)
+            if match:
+                product_id = match.group(1)
+
+            # Try to find price near this link
+            parent = link.find_parent()
+            price = None
+            for _ in range(5):
+                if parent:
+                    price_text = parent.get_text()
+                    price_match = re.search(r"\$[\d,]+\.?\d*", price_text)
+                    if price_match:
+                        price = price_match.group()
+                        break
+                    parent = parent.find_parent()
+
+            # Try to find image
+            image_url = None
+            parent = link.find_parent()
+            for _ in range(5):
+                if parent:
+                    img = parent.select_one("img")
+                    if img:
+                        image_url = img.get("src") or img.get("data-src")
+                        if image_url and not image_url.startswith("http"):
+                            image_url = f"{self.base_url}{image_url}"
+                        break
+                    parent = parent.find_parent()
+
+            # Check stock
+            in_stock = True
+            parent = link.find_parent()
+            for _ in range(5):
+                if parent:
+                    text = parent.get_text().lower()
+                    if any(phrase in text for phrase in ["sold out", "out of stock", "not available"]):
+                        in_stock = False
+                        break
+                    parent = parent.find_parent()
+
+            return Product(
+                name=name,
+                url=url,
+                price=price,
+                in_stock=in_stock,
+                image_url=image_url,
+                site=self.site_name,
+                product_id=product_id,
+            )
+
+        except Exception as e:
+            logger.debug(f"Error parsing GameStop product link: {e}")
+            return None
+
+    def _check_card_stock_status(self, card) -> bool:
+        """Check if a product card indicates in-stock status"""
+        card_text = card.get_text().lower() if hasattr(card, "get_text") else str(card).lower()
+
+        out_of_stock_phrases = [
+            "sold out",
+            "out of stock",
+            "not available",
+            "unavailable",
+            "currently unavailable",
+        ]
+
+        for phrase in out_of_stock_phrases:
+            if phrase in card_text:
+                return False
+
+        in_stock_phrases = [
+            "add to cart",
+            "add to bag",
+            "available",
+            "buy now",
+            "in stock",
+        ]
+
+        for phrase in in_stock_phrases:
+            if phrase in card_text:
+                return True
+
+        # Default: assume in stock if listed
+        return True
+
+    def check_product_stock(self, product_url: str) -> tuple[bool, Optional[str]]:
+        """Check if a specific product is in stock"""
+        try:
+            browser = self._get_stealth_browser()
+            browser.driver.get(product_url)
+            time.sleep(3)
+            html = browser.driver.page_source
+            soup = BeautifulSoup(html, "html.parser")
+            page_text = soup.get_text().lower()
+
+            out_of_stock = any(
+                phrase in page_text
+                for phrase in ["sold out", "out of stock", "not available", "unavailable"]
+            )
+
+            in_stock = not out_of_stock
+
+            # Get price
+            price = None
+            price_elem = (
+                soup.select_one(".product-price")
+                or soup.select_one(".price-sales")
+                or soup.select_one("[data-testid='price']")
+            )
+            if price_elem:
+                price_match = re.search(r"\$[\d,]+\.?\d*", price_elem.get_text())
+                if price_match:
+                    price = price_match.group()
+
+            return in_stock, price
+
+        except Exception as e:
+            logger.error(f"Error checking GameStop product stock: {e}")
+            return False, None

scrapers/pokemoncenter.py

@@ -1,6 +1,6 @@
 """
 PokemonCenter.com scraper
-Handles bot protection with Playwright stealth
+Handles bot protection with undetected-chromedriver stealth browser
 """
 
 import re
@@ -9,32 +9,71 @@ from typing import List, Optional
 from bs4 import BeautifulSoup
 
 from .base import BaseScraper, Product
-from browser import get_browser
+import config
 
 logger = logging.getLogger(__name__)
 
 
+def get_stealth_or_regular_browser():
+    """Get appropriate browser based on config"""
+    if config.USE_STEALTH_BROWSER:
+        from tools.stealth_browser import get_stealth_browser
+        return get_stealth_browser(
+            headless=config.STEALTH_HEADLESS,
+            session_name=config.STEALTH_SESSION_NAME
+        ), True
+    else:
+        from src.browser import get_browser
+        return get_browser(), False
+
+
 class PokemonCenterScraper(BaseScraper):
     """Scraper for PokemonCenter.com"""
 
     site_name = "pokemoncenter"
     base_url = "https://www.pokemoncenter.com"
 
-    def scrape_category_page(self, url: str) -> List[Product]:
+    def scrape_category_page(self, url: str, max_pages: int = 1) -> List[Product]:
         """
         Scrape a PokemonCenter category page for all products
 
         Args:
             url: Category page URL
+            max_pages: Maximum pages to scrape (not yet implemented for PokemonCenter)
 
         Returns:
             List of Product objects
         """
-        browser = get_browser()
+        # TODO: Implement pagination for PokemonCenter when needed
+        browser, is_stealth = get_stealth_or_regular_browser()
         products = []
 
         try:
-            # Navigate and wait for page to load (don't wait for specific selector)
+            if is_stealth:
+                # Use stealth browser (undetected-chromedriver)
+                html = browser.get_page(url, wait_time=config.HUMAN_DELAY_MAX)
+
+                # Check for CAPTCHA
+                if browser.check_for_captcha():
+                    logger.warning("CAPTCHA detected on PokemonCenter!")
+                    if config.PAUSE_ON_CAPTCHA and not config.STEALTH_HEADLESS:
+                        browser.wait_for_captcha_solve(config.CAPTCHA_WAIT_TIMEOUT)
+                        # Re-fetch page after CAPTCHA solve
+                        html = browser.get_page(url, wait_time=config.HUMAN_DELAY_MAX)
+                    else:
+                        logger.error("Cannot solve CAPTCHA in headless mode")
+                        return []
+
+                # Save screenshot for debugging
+                try:
+                    browser.screenshot("debug_screenshot.png")
+                    logger.info("Saved debug screenshot to debug_screenshot.png")
+                except:
+                    pass
+
+                page = None  # Stealth browser doesn't return page object
+            else:
+                # Use regular Playwright browser
                 page, html = browser.get_page_content(
                     url,
                     wait_for_selector=None,  # Let it use networkidle instead
@@ -96,6 +135,8 @@ class PokemonCenterScraper(BaseScraper):
                     if product:
                         products.append(product)
 
+            # Close page only for Playwright browser
+            if page is not None:
                 page.close()
 
         except Exception as e:
@@ -224,9 +265,25 @@ class PokemonCenterScraper(BaseScraper):
         Returns:
             Tuple of (is_in_stock, price)
         """
-        browser = get_browser()
+        browser, is_stealth = get_stealth_or_regular_browser()
 
         try:
+            if is_stealth:
+                # Use stealth browser
+                html = browser.get_page(product_url, wait_time=config.HUMAN_DELAY_MAX)
+
+                # Check for CAPTCHA
+                if browser.check_for_captcha():
+                    logger.warning("CAPTCHA detected on product page!")
+                    if config.PAUSE_ON_CAPTCHA and not config.STEALTH_HEADLESS:
+                        browser.wait_for_captcha_solve(config.CAPTCHA_WAIT_TIMEOUT)
+                        html = browser.get_page(product_url, wait_time=config.HUMAN_DELAY_MAX)
+                    else:
+                        return False, None
+
+                page = None
+            else:
+                # Use regular Playwright browser
                 page, html = browser.get_page_content(
                     product_url,
                     wait_for_selector="button, [data-testid]",
@@ -257,6 +314,7 @@ class PokemonCenterScraper(BaseScraper):
                 if price_match:
                     price = price_match.group()
 
+            if page is not None:
                 page.close()
             return in_stock, price
 

scrapers/target.py

@@ -9,7 +9,7 @@ from typing import List, Optional
 from bs4 import BeautifulSoup
 
 from .base import BaseScraper, Product
-from browser import get_browser
+from src.browser import get_browser
 
 logger = logging.getLogger(__name__)
 
@@ -20,22 +20,32 @@ class TargetScraper(BaseScraper):
     site_name = "target"
     base_url = "https://www.target.com"
 
-    def scrape_category_page(self, url: str) -> List[Product]:
+    def scrape_category_page(self, url: str, max_pages: int = 1) -> List[Product]:
         """
         Scrape a Target search/category page for all products
 
         Args:
             url: Search/category page URL
+            max_pages: Maximum number of pages to scrape (default 1)
 
         Returns:
             List of Product objects
         """
         browser = get_browser()
+        all_products = []
+
+        # Add sort by newest if not already in URL
+        if "sortBy=" not in url:
+            separator = "&" if "?" in url else "?"
+            url = f"{url}{separator}sortBy=newest"
+
+        for page_num in range(1, max_pages + 1):
             products = []
+            page_url = url if page_num == 1 else f"{url}&Nao={24 * (page_num - 1)}"
 
             try:
                 page, html = browser.get_page_content(
-                url,
+                    page_url,
                     wait_for_selector=None,
                     timeout=60000,
                 )
@@ -94,13 +104,21 @@ class TargetScraper(BaseScraper):
                 page.close()
 
             except Exception as e:
-            logger.error(f"Error scraping Target category page: {e}")
+                logger.error(f"Error scraping Target page {page_num}: {e}")
+                if page_num == 1:
                     raise
+                break
+
+            all_products.extend(products)
+
+            # Stop if no products found on this page
+            if not products:
+                break
 
         # Remove duplicates
         seen_urls = set()
         unique_products = []
-        for p in products:
+        for p in all_products:
             if p.url not in seen_urls:
                 seen_urls.add(p.url)
                 unique_products.append(p)

scrapers/walmart.py

@@ -0,0 +1,444 @@
+"""
+Walmart.com scraper
+
+Note: Walmart has aggressive bot protection (PerimeterX).
+May need stealth browser mode for sustained use.
+"""
+
+import re
+import json
+import logging
+from typing import List, Optional
+from bs4 import BeautifulSoup
+
+from .base import BaseScraper, Product
+from src.browser import get_browser
+
+logger = logging.getLogger(__name__)
+
+
+class WalmartScraper(BaseScraper):
+    """Scraper for Walmart.com"""
+
+    site_name = "walmart"
+    base_url = "https://www.walmart.com"
+
+    def scrape_category_page(self, url: str, max_pages: int = 1) -> List[Product]:
+        """
+        Scrape a Walmart search/category page for all products
+
+        Args:
+            url: Search/category page URL
+            max_pages: Maximum pages to scrape (not yet implemented for Walmart)
+
+        Returns:
+            List of Product objects
+        """
+        # TODO: Implement pagination for Walmart when needed
+        browser = get_browser()
+        products = []
+
+        try:
+            page, html = browser.get_page_content(
+                url,
+                wait_for_selector=None,
+                timeout=60000,
+            )
+
+            # Save debug screenshot
+            try:
+                page.screenshot(path="debug_walmart.png")
+                logger.info("Saved debug screenshot to debug_walmart.png")
+            except:
+                pass
+
+            soup = BeautifulSoup(html, "html.parser")
+
+            # Try to extract from __NEXT_DATA__ JSON (Walmart uses Next.js)
+            next_data = soup.select_one("script#__NEXT_DATA__")
+            if next_data:
+                try:
+                    data = json.loads(next_data.string)
+                    products.extend(self._extract_products_from_next_data(data))
+                except (json.JSONDecodeError, TypeError) as e:
+                    logger.debug(f"Error parsing __NEXT_DATA__: {e}")
+
+            # Also try application/json scripts
+            scripts = soup.find_all("script", type="application/json")
+            for script in scripts:
+                try:
+                    data = json.loads(script.string)
+                    products.extend(self._extract_products_from_json(data))
+                except (json.JSONDecodeError, TypeError):
+                    continue
+
+            # Try multiple selectors for product cards
+            product_cards = (
+                soup.select("[data-item-id]")
+                or soup.select(".search-result-gridview-item")
+                or soup.select("[data-testid='list-view']")
+                or soup.select("[class*='product-item']")
+                or soup.select("[class*='ProductCard']")
+            )
+
+            logger.info(f"Found {len(product_cards)} product cards")
+
+            for card in product_cards:
+                product = self._parse_product_card(card)
+                if product:
+                    products.append(product)
+
+            # Fallback: parse product links
+            if not product_cards and not products:
+                product_links = soup.select("a[href*='/ip/']")
+                logger.info(f"Fallback: Found {len(product_links)} product links")
+
+                href_to_links = {}
+                for link in product_links:
+                    href = link.get("href", "")
+                    if not href or "/ip/" not in href:
+                        continue
+                    if href not in href_to_links:
+                        href_to_links[href] = []
+                    href_to_links[href].append(link)
+
+                for href, links in href_to_links.items():
+                    best_link = links[0]
+                    for link in links:
+                        if link.get("aria-label") or len(link.get_text(strip=True)) > 10:
+                            best_link = link
+                            break
+
+                    product = self._parse_product_link(best_link, soup)
+                    if product:
+                        products.append(product)
+
+            page.close()
+
+        except Exception as e:
+            logger.error(f"Error scraping Walmart category page: {e}")
+            raise
+
+        # Remove duplicates
+        seen_urls = set()
+        unique_products = []
+        for p in products:
+            if p.url not in seen_urls:
+                seen_urls.add(p.url)
+                unique_products.append(p)
+
+        logger.info(f"Scraped {len(unique_products)} unique products from Walmart")
+        return unique_products
+
+    def _extract_products_from_next_data(self, data: dict) -> List[Product]:
+        """Extract products from Next.js __NEXT_DATA__ JSON"""
+        products = []
+
+        try:
+            # Navigate to search results in Next.js data structure
+            props = data.get("props", {})
+            page_props = props.get("pageProps", {})
+            initial_data = page_props.get("initialData", {})
+            search_result = initial_data.get("searchResult", {})
+            item_stacks = search_result.get("itemStacks", [])
+
+            for stack in item_stacks:
+                items = stack.get("items", [])
+                for item in items:
+                    product = self._parse_walmart_item(item)
+                    if product:
+                        products.append(product)
+
+        except Exception as e:
+            logger.debug(f"Error extracting from __NEXT_DATA__: {e}")
+
+        return products
+
+    def _extract_products_from_json(self, data, depth=0) -> List[Product]:
+        """Recursively search JSON for product data"""
+        products = []
+        if depth > 10:
+            return products
+
+        if isinstance(data, dict):
+            # Check if this looks like a Walmart product
+            if "usItemId" in data or ("name" in data and "priceInfo" in data):
+                product = self._parse_walmart_item(data)
+                if product:
+                    products.append(product)
+
+            for value in data.values():
+                products.extend(self._extract_products_from_json(value, depth + 1))
+
+        elif isinstance(data, list):
+            for item in data:
+                products.extend(self._extract_products_from_json(item, depth + 1))
+
+        return products
+
+    def _parse_walmart_item(self, data: dict) -> Optional[Product]:
+        """Parse a product from Walmart's JSON data"""
+        try:
+            name = data.get("name") or data.get("title", "")
+            if not name:
+                return None
+
+            item_id = data.get("usItemId") or data.get("id", "")
+            canonical_url = data.get("canonicalUrl") or data.get("productPageUrl", "")
+
+            if canonical_url:
+                url = canonical_url if canonical_url.startswith("http") else f"{self.base_url}{canonical_url}"
+            elif item_id:
+                url = f"{self.base_url}/ip/{item_id}"
+            else:
+                return None
+
+            # Get price
+            price = None
+            price_info = data.get("priceInfo", {})
+            if isinstance(price_info, dict):
+                current_price = price_info.get("currentPrice", {})
+                if isinstance(current_price, dict):
+                    price = current_price.get("priceString")
+                elif price_info.get("priceString"):
+                    price = price_info.get("priceString")
+
+            if not price and "price" in data:
+                price_val = data.get("price")
+                if isinstance(price_val, (int, float)):
+                    price = f"${price_val:.2f}"
+
+            # Check availability
+            in_stock = True
+            availability = data.get("availabilityStatusV2", {})
+            if isinstance(availability, dict):
+                status = availability.get("value", "").upper()
+                in_stock = status not in ["OUT_OF_STOCK", "NOT_AVAILABLE"]
+            elif data.get("availabilityStatus"):
+                in_stock = data.get("availabilityStatus") != "OUT_OF_STOCK"
+
+            # Get image
+            image_url = None
+            image_info = data.get("imageInfo", {})
+            if isinstance(image_info, dict):
+                image_url = image_info.get("thumbnailUrl") or image_info.get("url")
+            elif data.get("image"):
+                image_url = data.get("image")
+
+            return Product(
+                name=name,
+                url=url,
+                price=price,
+                in_stock=in_stock,
+                image_url=image_url,
+                site=self.site_name,
+                product_id=str(item_id),
+            )
+
+        except Exception as e:
+            logger.debug(f"Error parsing Walmart product JSON: {e}")
+            return None
+
+    def _parse_product_card(self, card) -> Optional[Product]:
+        """Parse a product card element"""
+        try:
+            # Find link
+            link = card.select_one("a[href*='/ip/']") or card.select_one("a")
+            if not link:
+                return None
+
+            href = link.get("href", "")
+            if not href:
+                return None
+            url = href if href.startswith("http") else f"{self.base_url}{href}"
+
+            # Get name
+            name_elem = (
+                card.select_one("[data-automation-id='product-title']")
+                or card.select_one(".product-title-link span")
+                or card.select_one("[class*='ProductTitle']")
+                or card.select_one("[class*='product-title']")
+                or link
+            )
+            name = name_elem.get_text(strip=True) if name_elem else "Unknown"
+
+            # Skip if name too short
+            if len(name) < 5:
+                return None
+
+            # Get price
+            price_elem = (
+                card.select_one("[data-automation-id='product-price']")
+                or card.select_one(".price-current")
+                or card.select_one("[class*='ProductPrice']")
+                or card.select_one("[class*='price']")
+            )
+            price = None
+            if price_elem:
+                price_match = re.search(r"\$[\d,]+\.?\d*", price_elem.get_text())
+                if price_match:
+                    price = price_match.group()
+
+            # Check stock
+            in_stock = self._check_card_stock_status(card)
+
+            # Get image
+            img = card.select_one("img")
+            image_url = None
+            if img:
+                image_url = img.get("src") or img.get("data-src")
+                if image_url and not image_url.startswith("http"):
+                    image_url = f"https:{image_url}" if image_url.startswith("//") else f"{self.base_url}{image_url}"
+
+            # Extract item ID from URL
+            product_id = ""
+            match = re.search(r"/ip/[^/]+/(\d+)", url) or re.search(r"/ip/(\d+)", url)
+            if match:
+                product_id = match.group(1)
+
+            return Product(
+                name=name,
+                url=url,
+                price=price,
+                in_stock=in_stock,
+                image_url=image_url,
+                site=self.site_name,
+                product_id=product_id,
+            )
+
+        except Exception as e:
+            logger.debug(f"Error parsing Walmart product card: {e}")
+            return None
+
+    def _parse_product_link(self, link, soup) -> Optional[Product]:
+        """Parse a product from a product link element"""
+        try:
+            href = link.get("href", "")
+            if not href:
+                return None
+
+            url = href if href.startswith("http") else f"{self.base_url}{href}"
+
+            name = link.get("aria-label", "") or link.get_text(strip=True)
+
+            if not name or len(name) < 5:
+                return None
+
+            # Extract item ID
+            product_id = ""
+            match = re.search(r"/ip/[^/]+/(\d+)", url) or re.search(r"/ip/(\d+)", url)
+            if match:
+                product_id = match.group(1)
+
+            # Find price near link
+            parent = link.find_parent()
+            price = None
+            for _ in range(5):
+                if parent:
+                    price_text = parent.get_text()
+                    price_match = re.search(r"\$[\d,]+\.?\d*", price_text)
+                    if price_match:
+                        price = price_match.group()
+                        break
+                    parent = parent.find_parent()
+
+            # Find image
+            image_url = None
+            parent = link.find_parent()
+            for _ in range(5):
+                if parent:
+                    img = parent.select_one("img")
+                    if img:
+                        image_url = img.get("src") or img.get("data-src")
+                        break
+                    parent = parent.find_parent()
+
+            # Check stock
+            in_stock = True
+            parent = link.find_parent()
+            for _ in range(5):
+                if parent:
+                    text = parent.get_text().lower()
+                    if any(phrase in text for phrase in ["out of stock", "unavailable", "not available"]):
+                        in_stock = False
+                        break
+                    parent = parent.find_parent()
+
+            return Product(
+                name=name,
+                url=url,
+                price=price,
+                in_stock=in_stock,
+                image_url=image_url,
+                site=self.site_name,
+                product_id=product_id,
+            )
+
+        except Exception as e:
+            logger.debug(f"Error parsing Walmart product link: {e}")
+            return None
+
+    def _check_card_stock_status(self, card) -> bool:
+        """Check if a product card indicates in-stock status"""
+        card_text = card.get_text().lower() if hasattr(card, "get_text") else str(card).lower()
+
+        out_of_stock_phrases = [
+            "out of stock",
+            "unavailable",
+            "not available",
+            "sold out",
+            "pickup not available",
+        ]
+
+        for phrase in out_of_stock_phrases:
+            if phrase in card_text:
+                return False
+
+        in_stock_phrases = [
+            "add to cart",
+            "available",
+            "in stock",
+            "pickup available",
+            "delivery available",
+        ]
+
+        for phrase in in_stock_phrases:
+            if phrase in card_text:
+                return True
+
+        return True
+
+    def check_product_stock(self, product_url: str) -> tuple[bool, Optional[str]]:
+        """Check if a specific product is in stock"""
+        browser = get_browser()
+
+        try:
+            page, html = browser.get_page_content(product_url, timeout=30000)
+            soup = BeautifulSoup(html, "html.parser")
+            page_text = soup.get_text().lower()
+
+            out_of_stock = any(
+                phrase in page_text
+                for phrase in ["out of stock", "unavailable", "not available", "sold out"]
+            )
+
+            in_stock = not out_of_stock
+
+            # Get price
+            price = None
+            price_elem = (
+                soup.select_one("[data-automation-id='product-price']")
+                or soup.select_one("[itemprop='price']")
+                or soup.select_one("[class*='price']")
+            )
+            if price_elem:
+                price_match = re.search(r"\$[\d,]+\.?\d*", price_elem.get_text())
+                if price_match:
+                    price = price_match.group()
+
+            page.close()
+            return in_stock, price
+
+        except Exception as e:
+            logger.error(f"Error checking Walmart product stock: {e}")
+            return False, None

src/__init__.py

@@ -0,0 +1 @@
+# Core application modules

src/browser.py

@@ -71,6 +71,8 @@ class StealthBrowser:
         """Start using an existing Chrome profile (better for bot detection)"""
         self.using_persistent = True
 
+        logger.info(f"Using profile path: {CHROME_USER_DATA_DIR}")
+
         # Launch persistent context with user's REAL Chrome (not Playwright's Chromium)
         # This is needed because Chrome encrypts credentials with its own binary
         self.context = self.playwright.chromium.launch_persistent_context(
@@ -84,9 +86,19 @@ class StealthBrowser:
             args=[
                 "--disable-blink-features=AutomationControlled",
                 "--disable-dev-shm-usage",
+                "--no-first-run",
+                "--no-default-browser-check",
             ]
         )
 
+        # Close any default blank pages that were opened
+        for page in self.context.pages:
+            if page.url == "about:blank":
+                try:
+                    page.close()
+                except:
+                    pass
+
         # Apply stealth
         stealth.apply_stealth_sync(self.context)
 

src/database.py

@@ -0,0 +1,929 @@
+"""
+SQLite database for Pokemon Stock Monitor stats tracking.
+Stores product history, price changes, stock events, and favorites.
+"""
+
+import sqlite3
+import json
+import logging
+from datetime import datetime, timedelta
+from pathlib import Path
+from typing import Optional, List, Dict, Any
+from contextlib import contextmanager
+
+logger = logging.getLogger(__name__)
+
+# Database file path
+DB_PATH = Path(__file__).parent.parent / "data" / "stats.db"
+
+
+class Database:
+    """SQLite database wrapper for stats tracking"""
+
+    def __init__(self, db_path: str = None):
+        self.db_path = db_path or str(DB_PATH)
+        self._init_schema()
+
+    @contextmanager
+    def get_connection(self):
+        """Context manager for database connections"""
+        conn = sqlite3.connect(self.db_path)
+        conn.row_factory = sqlite3.Row  # Return rows as dictionaries
+        try:
+            yield conn
+            conn.commit()
+        except Exception as e:
+            conn.rollback()
+            raise e
+        finally:
+            conn.close()
+
+    def _init_schema(self):
+        """Initialize database schema"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            # Products table
+            cursor.execute("""
+                CREATE TABLE IF NOT EXISTS products (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    url TEXT UNIQUE NOT NULL,
+                    name TEXT NOT NULL,
+                    site TEXT NOT NULL,
+                    product_id TEXT,
+                    image_url TEXT,
+                    current_price TEXT,
+                    in_stock BOOLEAN DEFAULT 0,
+                    first_seen DATETIME DEFAULT CURRENT_TIMESTAMP,
+                    last_seen DATETIME,
+                    category TEXT
+                )
+            """)
+
+            # Price history table
+            cursor.execute("""
+                CREATE TABLE IF NOT EXISTS price_history (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    product_id INTEGER NOT NULL,
+                    price TEXT,
+                    recorded_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+                    FOREIGN KEY (product_id) REFERENCES products(id)
+                )
+            """)
+
+            # Stock events table
+            cursor.execute("""
+                CREATE TABLE IF NOT EXISTS stock_events (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    product_id INTEGER NOT NULL,
+                    event_type TEXT NOT NULL,
+                    recorded_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+                    FOREIGN KEY (product_id) REFERENCES products(id)
+                )
+            """)
+
+            # Favorites table
+            cursor.execute("""
+                CREATE TABLE IF NOT EXISTS favorites (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    type TEXT NOT NULL,
+                    value TEXT NOT NULL,
+                    display_name TEXT,
+                    priority TEXT DEFAULT 'high',
+                    notify_discord BOOLEAN DEFAULT 1,
+                    notify_sound BOOLEAN DEFAULT 0,
+                    custom_webhook TEXT,
+                    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+                    UNIQUE(type, value)
+                )
+            """)
+
+            # Check logs table
+            cursor.execute("""
+                CREATE TABLE IF NOT EXISTS check_logs (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    site TEXT,
+                    products_found INTEGER DEFAULT 0,
+                    new_products INTEGER DEFAULT 0,
+                    restocks INTEGER DEFAULT 0,
+                    duration_ms INTEGER,
+                    success BOOLEAN DEFAULT 1,
+                    error_message TEXT,
+                    checked_at DATETIME DEFAULT CURRENT_TIMESTAMP
+                )
+            """)
+
+            # News articles table
+            cursor.execute("""
+                CREATE TABLE IF NOT EXISTS news_articles (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    source TEXT NOT NULL,
+                    source_account TEXT,
+                    external_id TEXT UNIQUE,
+                    title TEXT,
+                    content TEXT NOT NULL,
+                    url TEXT,
+                    author TEXT,
+                    image_url TEXT,
+                    published_at DATETIME,
+                    fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+                    sentiment_score REAL,
+                    sentiment_label TEXT,
+                    keywords TEXT,
+                    related_product_ids TEXT,
+                    is_drop_related BOOLEAN DEFAULT 0,
+                    is_restock_related BOOLEAN DEFAULT 0
+                )
+            """)
+
+            # Users table for multi-user support
+            cursor.execute("""
+                CREATE TABLE IF NOT EXISTS users (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    name TEXT NOT NULL UNIQUE,
+                    zip_code TEXT,
+                    radius_miles INTEGER DEFAULT 25,
+                    discord_webhook TEXT,
+                    notify_enabled BOOLEAN DEFAULT 1,
+                    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+                    last_active DATETIME DEFAULT CURRENT_TIMESTAMP
+                )
+            """)
+
+            # Create indexes for common queries
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_products_site ON products(site)")
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_products_category ON products(category)")
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_stock_events_product ON stock_events(product_id)")
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_stock_events_type ON stock_events(event_type)")
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_price_history_product ON price_history(product_id)")
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_check_logs_site ON check_logs(site)")
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_news_source ON news_articles(source)")
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_news_published ON news_articles(published_at)")
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_news_sentiment ON news_articles(sentiment_label)")
+            cursor.execute("CREATE INDEX IF NOT EXISTS idx_users_name ON users(name)")
+
+            logger.info("Database schema initialized")
+
+    # ==================== Product Methods ====================
+
+    def get_or_create_product(self, url: str, name: str, site: str,
+                               product_id: str = None, image_url: str = None,
+                               price: str = None, in_stock: bool = False) -> int:
+        """Get existing product or create new one. Returns product ID."""
+        category = self._detect_category(name)
+
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            # Try to get existing product
+            cursor.execute("SELECT id FROM products WHERE url = ?", (url,))
+            row = cursor.fetchone()
+
+            if row:
+                # Update existing product
+                cursor.execute("""
+                    UPDATE products
+                    SET name = ?, current_price = ?, in_stock = ?,
+                        last_seen = CURRENT_TIMESTAMP, image_url = COALESCE(?, image_url),
+                        category = COALESCE(?, category)
+                    WHERE id = ?
+                """, (name, price, in_stock, image_url, category, row['id']))
+                return row['id']
+            else:
+                # Create new product
+                cursor.execute("""
+                    INSERT INTO products (url, name, site, product_id, image_url,
+                                         current_price, in_stock, last_seen, category)
+                    VALUES (?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP, ?)
+                """, (url, name, site, product_id, image_url, price, in_stock, category))
+                return cursor.lastrowid
+
+    def get_product(self, product_id: int) -> Optional[Dict]:
+        """Get product by ID"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("SELECT * FROM products WHERE id = ?", (product_id,))
+            row = cursor.fetchone()
+            return dict(row) if row else None
+
+    def get_product_by_url(self, url: str) -> Optional[Dict]:
+        """Get product by URL"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("SELECT * FROM products WHERE url = ?", (url,))
+            row = cursor.fetchone()
+            return dict(row) if row else None
+
+    def update_product_price(self, product_id: int, price: str):
+        """Update product price and record in history"""
+        if not price:
+            return
+
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                UPDATE products SET current_price = ?, last_seen = CURRENT_TIMESTAMP
+                WHERE id = ?
+            """, (price, product_id))
+
+        # Record in price history (only if changed)
+        self.record_price(product_id, price)
+
+    def update_product_stock(self, product_id: int, in_stock: bool):
+        """Update product stock status"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                UPDATE products SET in_stock = ?, last_seen = CURRENT_TIMESTAMP
+                WHERE id = ?
+            """, (in_stock, product_id))
+
+    def get_products(self, site: str = None, category: str = None,
+                     in_stock: bool = None, favorites_only: bool = False,
+                     event_type: str = None, period: str = None,
+                     limit: int = 100, offset: int = 0) -> List[Dict]:
+        """Get products with optional filters
+
+        Args:
+            event_type: Filter by recent event type ('new_drop' or 'restock')
+            period: Time period for event filter ('today' or 'week')
+        """
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            # If filtering by event type, use a join with stock_events
+            if event_type and period:
+                period_sql = "-1 day" if period == "today" else "-7 days"
+                query = """
+                    SELECT DISTINCT p.* FROM products p
+                    INNER JOIN stock_events e ON p.id = e.product_id
+                    WHERE e.event_type = ?
+                    AND e.recorded_at >= datetime('now', ?)
+                """
+                params = [event_type, period_sql]
+            else:
+                query = "SELECT * FROM products WHERE 1=1"
+                params = []
+
+            if site:
+                query += " AND p.site = ?" if event_type else " AND site = ?"
+                params.append(site)
+            if category:
+                query += " AND p.category = ?" if event_type else " AND category = ?"
+                params.append(category)
+            if in_stock is not None:
+                query += " AND p.in_stock = ?" if event_type else " AND in_stock = ?"
+                params.append(in_stock)
+            if favorites_only:
+                prefix = "p." if event_type else ""
+                query += f""" AND (
+                    {prefix}url IN (SELECT value FROM favorites WHERE type = 'product')
+                    OR {prefix}category IN (SELECT value FROM favorites WHERE type = 'category')
+                )"""
+
+            order_col = "p.last_seen" if event_type else "last_seen"
+            query += f" ORDER BY {order_col} DESC LIMIT ? OFFSET ?"
+            params.extend([limit, offset])
+
+            cursor.execute(query, params)
+            return [dict(row) for row in cursor.fetchall()]
+
+    def get_product_count(self, site: str = None, in_stock: bool = None) -> int:
+        """Get count of products matching filters"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            query = "SELECT COUNT(*) FROM products WHERE 1=1"
+            params = []
+
+            if site:
+                query += " AND site = ?"
+                params.append(site)
+            if in_stock is not None:
+                query += " AND in_stock = ?"
+                params.append(in_stock)
+
+            cursor.execute(query, params)
+            return cursor.fetchone()[0]
+
+    def _detect_category(self, name: str) -> str:
+        """Auto-detect product category from name"""
+        name_lower = name.lower()
+
+        if "elite trainer" in name_lower or "etb" in name_lower:
+            return "ETB"
+        elif "booster bundle" in name_lower:
+            return "Booster Bundle"
+        elif "booster box" in name_lower:
+            return "Booster Box"
+        elif "booster pack" in name_lower or "sleeved booster" in name_lower:
+            return "Booster Pack"
+        elif "collection" in name_lower:
+            return "Collection Box"
+        elif "tin" in name_lower:
+            return "Tin"
+        elif "blister" in name_lower:
+            return "Blister"
+        elif "binder" in name_lower or "album" in name_lower:
+            return "Accessories"
+        else:
+            return "Other"
+
+    # ==================== Price History Methods ====================
+
+    def record_price(self, product_id: int, price: str):
+        """Record a price point for a product"""
+        if not price:
+            return
+
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            # Check if price changed from last record
+            cursor.execute("""
+                SELECT price FROM price_history
+                WHERE product_id = ?
+                ORDER BY recorded_at DESC LIMIT 1
+            """, (product_id,))
+            row = cursor.fetchone()
+
+            # Only record if price changed or no history exists
+            if not row or row['price'] != price:
+                cursor.execute("""
+                    INSERT INTO price_history (product_id, price)
+                    VALUES (?, ?)
+                """, (product_id, price))
+                logger.debug(f"Recorded price change for product {product_id}: {price}")
+
+    def get_price_history(self, product_id: int, days: int = 30) -> List[Dict]:
+        """Get price history for a product"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                SELECT price, recorded_at FROM price_history
+                WHERE product_id = ? AND recorded_at >= datetime('now', ?)
+                ORDER BY recorded_at ASC
+            """, (product_id, f"-{days} days"))
+            return [dict(row) for row in cursor.fetchall()]
+
+    # ==================== Stock Event Methods ====================
+
+    def record_stock_event(self, product_id: int, event_type: str):
+        """Record a stock event (new_drop, restock, in_stock, out_of_stock)"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                INSERT INTO stock_events (product_id, event_type)
+                VALUES (?, ?)
+            """, (product_id, event_type))
+            logger.debug(f"Recorded {event_type} event for product {product_id}")
+
+    def get_recent_events(self, limit: int = 20, event_types: List[str] = None) -> List[Dict]:
+        """Get recent stock events with product info"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            query = """
+                SELECT e.*, p.name, p.url, p.site, p.image_url, p.current_price
+                FROM stock_events e
+                JOIN products p ON e.product_id = p.id
+            """
+            params = []
+
+            if event_types:
+                placeholders = ",".join("?" * len(event_types))
+                query += f" WHERE e.event_type IN ({placeholders})"
+                params.extend(event_types)
+
+            query += " ORDER BY e.recorded_at DESC LIMIT ?"
+            params.append(limit)
+
+            cursor.execute(query, params)
+            return [dict(row) for row in cursor.fetchall()]
+
+    def get_events_today(self, event_type: str = None) -> int:
+        """Count events from today"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            query = """
+                SELECT COUNT(*) FROM stock_events
+                WHERE DATE(recorded_at) = DATE('now')
+            """
+            params = []
+
+            if event_type:
+                query += " AND event_type = ?"
+                params.append(event_type)
+
+            cursor.execute(query, params)
+            return cursor.fetchone()[0]
+
+    def get_events_this_week(self, event_type: str = None) -> int:
+        """Count events from this week"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            query = """
+                SELECT COUNT(*) FROM stock_events
+                WHERE recorded_at >= datetime('now', '-7 days')
+            """
+            params = []
+
+            if event_type:
+                query += " AND event_type = ?"
+                params.append(event_type)
+
+            cursor.execute(query, params)
+            return cursor.fetchone()[0]
+
+    # ==================== Analytics Methods ====================
+
+    def get_drop_timing_stats(self, days: int = 30) -> List[Dict]:
+        """Get drop timing by hour of day"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                SELECT
+                    strftime('%H', recorded_at) as hour,
+                    COUNT(*) as count
+                FROM stock_events
+                WHERE event_type IN ('new_drop', 'restock')
+                AND recorded_at >= datetime('now', ?)
+                GROUP BY hour
+                ORDER BY hour
+            """, (f"-{days} days",))
+            return [dict(row) for row in cursor.fetchall()]
+
+    def get_stock_duration_stats(self) -> List[Dict]:
+        """Get average time items stay in stock"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            # Calculate duration between in_stock and out_of_stock events
+            cursor.execute("""
+                SELECT
+                    p.category,
+                    AVG(
+                        CAST((julianday(out_event.recorded_at) - julianday(in_event.recorded_at)) * 24 * 60 AS INTEGER)
+                    ) as avg_minutes_in_stock
+                FROM stock_events in_event
+                JOIN stock_events out_event ON in_event.product_id = out_event.product_id
+                    AND out_event.event_type = 'out_of_stock'
+                    AND out_event.recorded_at > in_event.recorded_at
+                JOIN products p ON in_event.product_id = p.id
+                WHERE in_event.event_type IN ('restock', 'new_drop')
+                GROUP BY p.category
+            """)
+            return [dict(row) for row in cursor.fetchall()]
+
+    def get_site_stats(self) -> List[Dict]:
+        """Get stats per site"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                SELECT
+                    site,
+                    COUNT(*) as total_products,
+                    SUM(CASE WHEN in_stock = 1 THEN 1 ELSE 0 END) as in_stock_count
+                FROM products
+                GROUP BY site
+            """)
+            return [dict(row) for row in cursor.fetchall()]
+
+    # ==================== Check Log Methods ====================
+
+    def log_check(self, site: str, products_found: int, new_products: int,
+                  restocks: int, duration_ms: int, success: bool = True,
+                  error_message: str = None):
+        """Log a monitoring check"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                INSERT INTO check_logs (site, products_found, new_products,
+                                       restocks, duration_ms, success, error_message)
+                VALUES (?, ?, ?, ?, ?, ?, ?)
+            """, (site, products_found, new_products, restocks, duration_ms,
+                  success, error_message))
+
+    def get_last_check(self, site: str = None) -> Optional[Dict]:
+        """Get the most recent check log"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            query = "SELECT * FROM check_logs"
+            params = []
+
+            if site:
+                query += " WHERE site = ?"
+                params.append(site)
+
+            query += " ORDER BY checked_at DESC LIMIT 1"
+            cursor.execute(query, params)
+            row = cursor.fetchone()
+            return dict(row) if row else None
+
+    def get_check_history(self, site: str = None, limit: int = 100) -> List[Dict]:
+        """Get check history"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            query = "SELECT * FROM check_logs"
+            params = []
+
+            if site:
+                query += " WHERE site = ?"
+                params.append(site)
+
+            query += " ORDER BY checked_at DESC LIMIT ?"
+            params.append(limit)
+
+            cursor.execute(query, params)
+            return [dict(row) for row in cursor.fetchall()]
+
+    # ==================== Favorites Methods ====================
+
+    def add_favorite(self, fav_type: str, value: str, display_name: str = None,
+                     priority: str = "high", notify_discord: bool = True,
+                     notify_sound: bool = False, custom_webhook: str = None) -> int:
+        """Add a favorite. Returns favorite ID."""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                INSERT OR REPLACE INTO favorites
+                (type, value, display_name, priority, notify_discord, notify_sound, custom_webhook)
+                VALUES (?, ?, ?, ?, ?, ?, ?)
+            """, (fav_type, value, display_name or value, priority,
+                  notify_discord, notify_sound, custom_webhook))
+            return cursor.lastrowid
+
+    def remove_favorite(self, favorite_id: int):
+        """Remove a favorite by ID"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("DELETE FROM favorites WHERE id = ?", (favorite_id,))
+
+    def get_favorites(self, fav_type: str = None) -> List[Dict]:
+        """Get all favorites, optionally filtered by type"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            if fav_type:
+                cursor.execute("SELECT * FROM favorites WHERE type = ? ORDER BY created_at DESC",
+                              (fav_type,))
+            else:
+                cursor.execute("SELECT * FROM favorites ORDER BY created_at DESC")
+
+            return [dict(row) for row in cursor.fetchall()]
+
+    def get_favorite(self, favorite_id: int) -> Optional[Dict]:
+        """Get a favorite by ID"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("SELECT * FROM favorites WHERE id = ?", (favorite_id,))
+            row = cursor.fetchone()
+            return dict(row) if row else None
+
+    def update_favorite(self, favorite_id: int, **kwargs):
+        """Update favorite settings"""
+        allowed_fields = ['display_name', 'priority', 'notify_discord',
+                         'notify_sound', 'custom_webhook']
+        updates = {k: v for k, v in kwargs.items() if k in allowed_fields}
+
+        if not updates:
+            return
+
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            set_clause = ", ".join(f"{k} = ?" for k in updates.keys())
+            cursor.execute(
+                f"UPDATE favorites SET {set_clause} WHERE id = ?",
+                list(updates.values()) + [favorite_id]
+            )
+
+    def check_is_favorite(self, url: str = None, category: str = None) -> Optional[Dict]:
+        """Check if a product URL or category is favorited"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            if url:
+                cursor.execute("""
+                    SELECT * FROM favorites
+                    WHERE type = 'product' AND value = ?
+                """, (url,))
+                row = cursor.fetchone()
+                if row:
+                    return dict(row)
+
+            if category:
+                cursor.execute("""
+                    SELECT * FROM favorites
+                    WHERE type = 'category' AND value = ?
+                """, (category,))
+                row = cursor.fetchone()
+                if row:
+                    return dict(row)
+
+            return None
+
+    # ==================== News Article Methods ====================
+
+    def add_news_article(self, source: str, content: str, source_account: str = None,
+                         external_id: str = None, title: str = None, url: str = None,
+                         author: str = None, image_url: str = None, published_at: str = None,
+                         sentiment_score: float = None, sentiment_label: str = None,
+                         keywords: List[str] = None, related_product_ids: List[int] = None,
+                         is_drop_related: bool = False, is_restock_related: bool = False) -> int:
+        """Add a news article. Returns article ID."""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            # Check if article already exists by external_id
+            if external_id:
+                cursor.execute("SELECT id FROM news_articles WHERE external_id = ?", (external_id,))
+                existing = cursor.fetchone()
+                if existing:
+                    return existing['id']
+
+            cursor.execute("""
+                INSERT INTO news_articles (source, source_account, external_id, title, content,
+                                          url, author, image_url, published_at, sentiment_score,
+                                          sentiment_label, keywords, related_product_ids,
+                                          is_drop_related, is_restock_related)
+                VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+            """, (
+                source, source_account, external_id, title, content,
+                url, author, image_url, published_at, sentiment_score,
+                sentiment_label,
+                json.dumps(keywords) if keywords else None,
+                json.dumps(related_product_ids) if related_product_ids else None,
+                is_drop_related, is_restock_related
+            ))
+            return cursor.lastrowid
+
+    def get_news_articles(self, source: str = None, sentiment: str = None,
+                          drop_related: bool = None, limit: int = 50,
+                          offset: int = 0) -> List[Dict]:
+        """Get news articles with optional filters"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            query = "SELECT * FROM news_articles WHERE 1=1"
+            params = []
+
+            if source:
+                query += " AND source = ?"
+                params.append(source)
+            if sentiment:
+                query += " AND sentiment_label = ?"
+                params.append(sentiment)
+            if drop_related is not None:
+                query += " AND (is_drop_related = ? OR is_restock_related = ?)"
+                params.extend([drop_related, drop_related])
+
+            query += " ORDER BY published_at DESC, fetched_at DESC LIMIT ? OFFSET ?"
+            params.extend([limit, offset])
+
+            cursor.execute(query, params)
+            articles = []
+            for row in cursor.fetchall():
+                article = dict(row)
+                # Parse JSON fields
+                if article.get('keywords'):
+                    article['keywords'] = json.loads(article['keywords'])
+                if article.get('related_product_ids'):
+                    article['related_product_ids'] = json.loads(article['related_product_ids'])
+                articles.append(article)
+            return articles
+
+    def get_news_article(self, article_id: int) -> Optional[Dict]:
+        """Get a news article by ID"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("SELECT * FROM news_articles WHERE id = ?", (article_id,))
+            row = cursor.fetchone()
+            if row:
+                article = dict(row)
+                if article.get('keywords'):
+                    article['keywords'] = json.loads(article['keywords'])
+                if article.get('related_product_ids'):
+                    article['related_product_ids'] = json.loads(article['related_product_ids'])
+                return article
+            return None
+
+    def get_news_stats(self) -> Dict:
+        """Get news statistics"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+
+            # Total articles
+            cursor.execute("SELECT COUNT(*) FROM news_articles")
+            total = cursor.fetchone()[0]
+
+            # By source
+            cursor.execute("""
+                SELECT source, COUNT(*) as count
+                FROM news_articles GROUP BY source
+            """)
+            by_source = {row['source']: row['count'] for row in cursor.fetchall()}
+
+            # By sentiment
+            cursor.execute("""
+                SELECT sentiment_label, COUNT(*) as count
+                FROM news_articles WHERE sentiment_label IS NOT NULL
+                GROUP BY sentiment_label
+            """)
+            by_sentiment = {row['sentiment_label']: row['count'] for row in cursor.fetchall()}
+
+            # Today's articles
+            cursor.execute("""
+                SELECT COUNT(*) FROM news_articles
+                WHERE DATE(fetched_at) = DATE('now')
+            """)
+            today = cursor.fetchone()[0]
+
+            # Drop related
+            cursor.execute("""
+                SELECT COUNT(*) FROM news_articles
+                WHERE is_drop_related = 1 OR is_restock_related = 1
+            """)
+            drop_related = cursor.fetchone()[0]
+
+            return {
+                'total': total,
+                'today': today,
+                'by_source': by_source,
+                'by_sentiment': by_sentiment,
+                'drop_related': drop_related
+            }
+
+    def update_news_correlation(self, article_id: int, related_product_ids: List[int],
+                                 is_drop_related: bool, is_restock_related: bool):
+        """Update news article with correlation data"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                UPDATE news_articles
+                SET related_product_ids = ?, is_drop_related = ?, is_restock_related = ?
+                WHERE id = ?
+            """, (json.dumps(related_product_ids), is_drop_related, is_restock_related, article_id))
+
+    def get_correlated_news(self, hours: int = 24) -> List[Dict]:
+        """Get news articles correlated with recent drops"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                SELECT * FROM news_articles
+                WHERE (is_drop_related = 1 OR is_restock_related = 1)
+                AND fetched_at >= datetime('now', ?)
+                ORDER BY published_at DESC
+            """, (f"-{hours} hours",))
+            articles = []
+            for row in cursor.fetchall():
+                article = dict(row)
+                if article.get('keywords'):
+                    article['keywords'] = json.loads(article['keywords'])
+                if article.get('related_product_ids'):
+                    article['related_product_ids'] = json.loads(article['related_product_ids'])
+                articles.append(article)
+            return articles
+
+    def delete_old_news(self, days: int = 30):
+        """Delete news articles older than specified days"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                DELETE FROM news_articles
+                WHERE fetched_at < datetime('now', ?)
+            """, (f"-{days} days",))
+            deleted = cursor.rowcount
+            logger.info(f"Deleted {deleted} old news articles")
+            return deleted
+
+    # ==================== Migration Methods ====================
+
+    def migrate_from_json(self, json_path: str):
+        """Migrate existing products.json to database"""
+        try:
+            with open(json_path, 'r') as f:
+                products = json.load(f)
+        except (FileNotFoundError, json.JSONDecodeError):
+            logger.warning(f"Could not load {json_path} for migration")
+            return
+
+        migrated = 0
+        for url, data in products.items():
+            product_id = self.get_or_create_product(
+                url=url,
+                name=data.get('name', 'Unknown'),
+                site=data.get('site', 'unknown'),
+                product_id=data.get('product_id'),
+                image_url=data.get('image_url'),
+                price=data.get('price'),
+                in_stock=data.get('in_stock', False)
+            )
+
+            # Record initial price if available
+            if data.get('price'):
+                self.record_price(product_id, data['price'])
+
+            # Record as existing product (not new_drop since it's historical)
+            if data.get('in_stock'):
+                self.record_stock_event(product_id, 'in_stock')
+
+            migrated += 1
+
+        logger.info(f"Migrated {migrated} products from {json_path}")
+
+    # ==================== User Methods ====================
+
+    def create_user(self, name: str, zip_code: str = None, radius_miles: int = 25,
+                    discord_webhook: str = None, notify_enabled: bool = True) -> int:
+        """Create a new user. Returns user ID."""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("""
+                INSERT INTO users (name, zip_code, radius_miles, discord_webhook, notify_enabled)
+                VALUES (?, ?, ?, ?, ?)
+            """, (name, zip_code, radius_miles, discord_webhook, notify_enabled))
+            logger.info(f"Created user: {name}")
+            return cursor.lastrowid
+
+    def get_user(self, user_id: int) -> Optional[Dict]:
+        """Get user by ID"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("SELECT * FROM users WHERE id = ?", (user_id,))
+            row = cursor.fetchone()
+            return dict(row) if row else None
+
+    def get_user_by_name(self, name: str) -> Optional[Dict]:
+        """Get user by name"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("SELECT * FROM users WHERE name = ?", (name,))
+            row = cursor.fetchone()
+            return dict(row) if row else None
+
+    def get_all_users(self) -> List[Dict]:
+        """Get all users"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("SELECT * FROM users ORDER BY name")
+            return [dict(row) for row in cursor.fetchall()]
+
+    def update_user(self, user_id: int, **kwargs) -> bool:
+        """Update user settings"""
+        allowed_fields = ['name', 'zip_code', 'radius_miles', 'discord_webhook', 'notify_enabled']
+        updates = {k: v for k, v in kwargs.items() if k in allowed_fields}
+
+        if not updates:
+            return False
+
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            set_clause = ", ".join(f"{k} = ?" for k in updates.keys())
+            cursor.execute(
+                f"UPDATE users SET {set_clause}, last_active = CURRENT_TIMESTAMP WHERE id = ?",
+                list(updates.values()) + [user_id]
+            )
+            logger.info(f"Updated user {user_id}: {updates}")
+            return cursor.rowcount > 0
+
+    def delete_user(self, user_id: int) -> bool:
+        """Delete a user"""
+        with self.get_connection() as conn:
+            cursor = conn.cursor()
+            cursor.execute("DELETE FROM users WHERE id = ?", (user_id,))
+            return cursor.rowcount > 0
+
+    def update_user_location(self, user_id: int, zip_code: str, radius_miles: int = 25) -> bool:
+        """Update user's location settings"""
+        return self.update_user(user_id, zip_code=zip_code, radius_miles=radius_miles)
+
+    # ==================== Stats Summary ====================
+
+    def get_dashboard_stats(self) -> Dict:
+        """Get summary stats for dashboard"""
+        return {
+            'total_products': self.get_product_count(),
+            'in_stock_count': self.get_product_count(in_stock=True),
+            'new_drops_today': self.get_events_today('new_drop'),
+            'new_drops_week': self.get_events_this_week('new_drop'),
+            'restocks_today': self.get_events_today('restock'),
+            'restocks_week': self.get_events_this_week('restock'),
+            'last_check': self.get_last_check(),
+            'sites': self.get_site_stats(),
+            'favorites_count': len(self.get_favorites())
+        }
+
+
+# Global database instance
+_db: Database = None
+
+
+def get_database() -> Database:
+    """Get the global database instance"""
+    global _db
+    if _db is None:
+        _db = Database()
+    return _db

src/discord_bot.py

@@ -0,0 +1,353 @@
+"""
+Pokemon Stock Monitor Discord Bot
+
+Interactive bot for setting location and checking local store stock.
+
+Commands:
+  !setlocation <zip_code>  - Set your location
+  !stores                   - Find nearby stores
+  !stock <retailer>         - Check Pokemon TCG stock at local stores
+  !help                     - Show commands
+
+Setup:
+  1. Create a bot at https://discord.com/developers/applications
+  2. Get your bot token
+  3. Add bot to your server with Message Content Intent enabled
+  4. Set DISCORD_BOT_TOKEN in config.py or environment variable
+"""
+
+import os
+import asyncio
+import logging
+from datetime import datetime
+from typing import Optional
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# Try to import discord.py
+try:
+    import discord
+    from discord.ext import commands
+    DISCORD_AVAILABLE = True
+except ImportError:
+    DISCORD_AVAILABLE = False
+    logger.warning("discord.py not installed. Run: pip install discord.py")
+
+from .store_locator import (
+    find_all_nearby_stores,
+    TargetLocator,
+    BestBuyLocator,
+    GameStopLocator,
+    WalmartLocator,
+    load_location_config,
+    save_location_config,
+    format_stores_for_discord
+)
+
+# User locations stored by Discord user ID
+user_locations = {}
+
+
+def load_user_locations():
+    """Load user locations from file"""
+    import json
+    from pathlib import Path
+
+    loc_file = Path(__file__).parent.parent / "data" / "user_locations.json"
+    if loc_file.exists():
+        with open(loc_file, 'r') as f:
+            return json.load(f)
+    return {}
+
+
+def save_user_locations():
+    """Save user locations to file"""
+    import json
+    from pathlib import Path
+
+    loc_file = Path(__file__).parent.parent / "data" / "user_locations.json"
+    with open(loc_file, 'w') as f:
+        json.dump(user_locations, f, indent=2)
+
+
+if DISCORD_AVAILABLE:
+
+    # Bot setup with intents
+    intents = discord.Intents.default()
+    intents.message_content = True
+
+    bot = commands.Bot(command_prefix='!', intents=intents, help_command=None)
+
+    @bot.event
+    async def on_ready():
+        global user_locations
+        user_locations = load_user_locations()
+        logger.info(f"Bot is ready! Logged in as {bot.user}")
+        logger.info(f"Loaded {len(user_locations)} user locations")
+
+    @bot.command(name='help')
+    async def help_command(ctx):
+        """Show help message"""
+        embed = discord.Embed(
+            title="Pokemon Stock Monitor Bot",
+            description="Find Pokemon TCG products at local stores!",
+            color=0xFFCC00
+        )
+
+        embed.add_field(
+            name="Commands",
+            value="""
+`!setlocation <zip_code>` - Set your location (e.g., `!setlocation 90210`)
+`!stores` - Find nearby Target, Best Buy, GameStop, Walmart
+`!stock` - Check Pokemon TCG stock at your local stores
+`!stock target` - Check Target specifically
+`!mylocation` - Show your saved location
+            """,
+            inline=False
+        )
+
+        embed.add_field(
+            name="Supported Retailers",
+            value="Target, Best Buy, GameStop, Walmart",
+            inline=False
+        )
+
+        embed.set_footer(text="Pokemon Stock Monitor")
+        await ctx.send(embed=embed)
+
+    @bot.command(name='setlocation')
+    async def set_location(ctx, zip_code: str, radius: int = 25):
+        """Set user's location"""
+        # Validate zip code (basic check)
+        if not zip_code.isdigit() or len(zip_code) != 5:
+            await ctx.send("Please provide a valid 5-digit US zip code. Example: `!setlocation 90210`")
+            return
+
+        user_id = str(ctx.author.id)
+        user_locations[user_id] = {
+            "zip_code": zip_code,
+            "radius_miles": radius,
+            "set_at": datetime.now().isoformat()
+        }
+        save_user_locations()
+
+        embed = discord.Embed(
+            title="Location Saved!",
+            description=f"Your location has been set to **{zip_code}** with a **{radius} mile** search radius.",
+            color=0x00FF00
+        )
+        embed.add_field(name="Next Steps", value="Use `!stores` to find nearby stores or `!stock` to check inventory.")
+        await ctx.send(embed=embed)
+
+    @bot.command(name='mylocation')
+    async def my_location(ctx):
+        """Show user's saved location"""
+        user_id = str(ctx.author.id)
+
+        if user_id not in user_locations:
+            await ctx.send("You haven't set a location yet. Use `!setlocation <zip_code>` to set one.")
+            return
+
+        loc = user_locations[user_id]
+        embed = discord.Embed(
+            title="Your Location",
+            color=0x0099FF
+        )
+        embed.add_field(name="Zip Code", value=loc["zip_code"], inline=True)
+        embed.add_field(name="Search Radius", value=f"{loc['radius_miles']} miles", inline=True)
+        await ctx.send(embed=embed)
+
+    @bot.command(name='stores')
+    async def find_stores(ctx):
+        """Find nearby stores"""
+        user_id = str(ctx.author.id)
+
+        if user_id not in user_locations:
+            await ctx.send("Please set your location first with `!setlocation <zip_code>`")
+            return
+
+        loc = user_locations[user_id]
+        zip_code = loc["zip_code"]
+        radius = loc["radius_miles"]
+
+        # Send "searching" message
+        searching_msg = await ctx.send(f"Searching for stores near {zip_code}...")
+
+        # Find stores (run in thread to avoid blocking)
+        loop = asyncio.get_event_loop()
+        stores = await loop.run_in_executor(
+            None,
+            find_all_nearby_stores,
+            zip_code,
+            radius
+        )
+
+        # Build embed
+        embed = discord.Embed(
+            title=f"Stores Near {zip_code}",
+            description=f"Within {radius} miles",
+            color=0xFFCC00
+        )
+
+        total_stores = 0
+        for retailer, store_list in stores.items():
+            if store_list:
+                total_stores += len(store_list)
+                stores_text = ""
+                for store in store_list[:5]:  # Show top 5 per retailer
+                    stores_text += f"**{store.name}** ({store.distance_miles:.1f} mi)\n"
+                    stores_text += f"{store.city}, {store.state}\n\n"
+
+                embed.add_field(
+                    name=f"{retailer} ({len(store_list)} stores)",
+                    value=stores_text[:1024] if stores_text else "No stores found",
+                    inline=False
+                )
+            else:
+                embed.add_field(
+                    name=retailer,
+                    value="No stores found nearby",
+                    inline=False
+                )
+
+        embed.set_footer(text=f"Total: {total_stores} stores found")
+
+        await searching_msg.edit(content=None, embed=embed)
+
+    @bot.command(name='stock')
+    async def check_stock(ctx, retailer: Optional[str] = None):
+        """Check Pokemon TCG stock at local stores"""
+        user_id = str(ctx.author.id)
+
+        if user_id not in user_locations:
+            await ctx.send("Please set your location first with `!setlocation <zip_code>`")
+            return
+
+        loc = user_locations[user_id]
+        zip_code = loc["zip_code"]
+        radius = loc["radius_miles"]
+
+        # Determine which retailers to check
+        retailers_to_check = []
+        if retailer:
+            retailer_lower = retailer.lower()
+            if "target" in retailer_lower:
+                retailers_to_check = [("Target", TargetLocator)]
+            elif "best" in retailer_lower or "buy" in retailer_lower:
+                retailers_to_check = [("Best Buy", BestBuyLocator)]
+            elif "game" in retailer_lower or "stop" in retailer_lower:
+                retailers_to_check = [("GameStop", GameStopLocator)]
+            elif "walmart" in retailer_lower:
+                retailers_to_check = [("Walmart", WalmartLocator)]
+            else:
+                await ctx.send(f"Unknown retailer: {retailer}. Try: target, bestbuy, gamestop, walmart")
+                return
+        else:
+            # Check all
+            retailers_to_check = [
+                ("Target", TargetLocator),
+                ("Best Buy", BestBuyLocator),
+                ("GameStop", GameStopLocator),
+                ("Walmart", WalmartLocator),
+            ]
+
+        searching_msg = await ctx.send(f"Checking Pokemon TCG stock near {zip_code}...")
+
+        embed = discord.Embed(
+            title="Pokemon TCG Local Stock",
+            description=f"Checking stores near {zip_code}",
+            color=0xFF6600
+        )
+
+        for name, LocatorClass in retailers_to_check:
+            try:
+                locator = LocatorClass(zip_code, radius)
+                stores = locator.find_stores()[:3]  # Check closest 3
+
+                if stores:
+                    store_info = ""
+                    for store in stores:
+                        # Note: Full stock check requires product IDs
+                        # For now, just show store info
+                        store_info += f"**{store.name}**\n"
+                        store_info += f"{store.address}\n"
+                        store_info += f"{store.city}, {store.state} ({store.distance_miles:.1f} mi)\n"
+                        if store.phone:
+                            store_info += f"Phone: {store.phone}\n"
+                        store_info += "\n"
+
+                    embed.add_field(
+                        name=f"{name}",
+                        value=store_info[:1024] or "No stores found",
+                        inline=False
+                    )
+                else:
+                    embed.add_field(name=name, value="No stores found nearby", inline=False)
+
+            except Exception as e:
+                logger.error(f"Error checking {name}: {e}")
+                embed.add_field(name=name, value="Error checking stores", inline=False)
+
+        embed.add_field(
+            name="Tip",
+            value="Call ahead to confirm Pokemon TCG availability. Stock changes frequently!",
+            inline=False
+        )
+
+        embed.set_footer(text="Pokemon Stock Monitor")
+        await searching_msg.edit(content=None, embed=embed)
+
+    @bot.command(name='ping')
+    async def ping(ctx):
+        """Check if bot is alive"""
+        await ctx.send(f"Pong! Latency: {round(bot.latency * 1000)}ms")
+
+
+def run_bot(token: str):
+    """Run the Discord bot"""
+    if not DISCORD_AVAILABLE:
+        print("discord.py is not installed!")
+        print("Run: pip install discord.py")
+        return
+
+    bot.run(token)
+
+
+if __name__ == "__main__":
+    # Try to get token from config or environment
+    token = os.environ.get("DISCORD_BOT_TOKEN")
+
+    if not token:
+        try:
+            from config import DISCORD_BOT_TOKEN
+            token = DISCORD_BOT_TOKEN
+        except ImportError:
+            pass
+
+    if not token:
+        print("=" * 60)
+        print("Discord Bot Setup")
+        print("=" * 60)
+        print()
+        print("To run the bot, you need a Discord Bot Token.")
+        print()
+        print("1. Go to https://discord.com/developers/applications")
+        print("2. Create a New Application")
+        print("3. Go to 'Bot' section, create a bot")
+        print("4. Enable 'Message Content Intent' under Privileged Intents")
+        print("5. Copy the token")
+        print()
+        print("Then either:")
+        print("  - Set DISCORD_BOT_TOKEN in config.py")
+        print("  - Or set DISCORD_BOT_TOKEN environment variable")
+        print()
+        print("To invite bot to your server:")
+        print("  - Go to OAuth2 > URL Generator")
+        print("  - Select 'bot' scope")
+        print("  - Select permissions: Send Messages, Embed Links, Read Message History")
+        print("  - Use the generated URL to invite")
+        print()
+    else:
+        print("Starting Discord bot...")
+        run_bot(token)

src/discord_notifier.py

@@ -0,0 +1,340 @@
+"""
+Discord webhook notifications for stock alerts
+Sends rich embeds with product info and direct links
+Now supports favorites with priority notifications and custom webhooks
+"""
+
+import json
+import logging
+import requests
+from datetime import datetime
+from pathlib import Path
+from typing import Optional, Dict, Set
+from config import DISCORD_WEBHOOK_URL, NOTIFICATIONS_ENABLED, SKIP_DUPLICATE_SKUS
+
+logger = logging.getLogger(__name__)
+
+# Track notified products to avoid duplicates
+NOTIFIED_FILE = Path(__file__).parent.parent / "data" / "notified_products.json"
+_notified_urls: Set[str] = set()
+
+
+def _load_notified():
+    """Load previously notified product URLs"""
+    global _notified_urls
+    try:
+        if NOTIFIED_FILE.exists():
+            with open(NOTIFIED_FILE, 'r') as f:
+                _notified_urls = set(json.load(f))
+    except Exception:
+        _notified_urls = set()
+
+
+def _save_notified():
+    """Save notified product URLs"""
+    try:
+        NOTIFIED_FILE.parent.mkdir(parents=True, exist_ok=True)
+        with open(NOTIFIED_FILE, 'w') as f:
+            json.dump(list(_notified_urls), f)
+    except Exception as e:
+        logger.error(f"Failed to save notified products: {e}")
+
+
+def was_already_notified(product_url: str) -> bool:
+    """Check if we already sent a notification for this product"""
+    if not _notified_urls:
+        _load_notified()
+    return product_url in _notified_urls
+
+
+def mark_as_notified(product_url: str):
+    """Mark a product as notified"""
+    _notified_urls.add(product_url)
+    _save_notified()
+
+# Colors for different notification types
+COLOR_RESTOCK = 0x00FF00  # Green - item back in stock
+COLOR_NEW_DROP = 0x0099FF  # Blue - new product listing
+COLOR_PREORDER = 0xFFAA00  # Orange - pre-order available
+COLOR_ERROR = 0xFF0000  # Red - error notification
+COLOR_FAVORITE_HIGH = 0xFF0000  # Red - high priority favorite
+COLOR_FAVORITE_MEDIUM = 0xFFAA00  # Orange - medium priority favorite
+
+# Site icons/emojis
+SITE_EMOJIS = {
+    "pokemoncenter": "\U0001F7E1",  # Yellow circle
+    "target": "\U0001F534",  # Red circle
+    "walmart": "\U0001F535",  # Blue circle
+    "bestbuy": "\U0001F7E1",  # Yellow circle
+}
+
+# Priority indicators
+PRIORITY_EMOJI = {
+    "high": "\u2B50\u2B50\u2B50",  # Three stars
+    "medium": "\u2B50\u2B50",  # Two stars
+    "low": "\u2B50",  # One star
+    "normal": "",  # No stars
+}
+
+
+def send_stock_alert(
+    product_name: str,
+    product_url: str,
+    price: str,
+    site: str,
+    alert_type: str = "restock",
+    image_url: Optional[str] = None,
+    priority_settings: Optional[Dict] = None,
+):
+    """
+    Send a Discord notification for a stock alert
+
+    Args:
+        product_name: Name of the product
+        product_url: Direct link to the product
+        price: Price string (e.g., "$49.99")
+        site: Site name (pokemoncenter, target, etc.)
+        alert_type: "restock", "new_drop", or "preorder"
+        image_url: Optional product image URL
+        priority_settings: Optional dict with favorite/priority info
+    """
+    # Check if notifications are enabled
+    if not NOTIFICATIONS_ENABLED:
+        logger.debug(f"Notifications disabled - skipping alert for: {product_name}")
+        return True  # Return True so caller thinks it succeeded
+
+    # Skip duplicate notifications for new drops (restocks always go through)
+    if SKIP_DUPLICATE_SKUS and alert_type == "new_drop":
+        if was_already_notified(product_url):
+            logger.debug(f"Already notified about this product - skipping: {product_name}")
+            return True
+
+    # Determine which webhook to use
+    webhook_url = DISCORD_WEBHOOK_URL
+    if priority_settings and priority_settings.get('custom_webhook'):
+        webhook_url = priority_settings['custom_webhook']
+
+    if webhook_url == "YOUR_WEBHOOK_URL_HERE":
+        logger.error("Discord webhook URL not configured! Update config.py")
+        return False
+
+    # Check if this is a favorite
+    is_favorite = priority_settings.get('is_favorite', False) if priority_settings else False
+    priority = priority_settings.get('priority', 'normal') if priority_settings else 'normal'
+
+    # Choose color based on alert type and priority
+    if is_favorite and priority == 'high':
+        color = COLOR_FAVORITE_HIGH
+    elif is_favorite and priority == 'medium':
+        color = COLOR_FAVORITE_MEDIUM
+    elif alert_type == "restock":
+        color = COLOR_RESTOCK
+    elif alert_type == "new_drop":
+        color = COLOR_NEW_DROP
+    elif alert_type == "preorder":
+        color = COLOR_PREORDER
+    else:
+        color = COLOR_RESTOCK
+
+    # Build title with priority indicator
+    priority_indicator = PRIORITY_EMOJI.get(priority, "")
+    favorite_badge = "\u2B50 FAVORITE " if is_favorite else ""
+
+    if alert_type == "restock":
+        title = f"\U0001F6A8 {favorite_badge}RESTOCK ALERT {priority_indicator}"
+    elif alert_type == "new_drop":
+        title = f"\U0001F195 {favorite_badge}NEW DROP {priority_indicator}"
+    elif alert_type == "preorder":
+        title = f"\u23F0 {favorite_badge}PRE-ORDER AVAILABLE {priority_indicator}"
+    else:
+        title = f"\U0001F514 {favorite_badge}STOCK ALERT {priority_indicator}"
+
+    site_emoji = SITE_EMOJIS.get(site.lower(), "\U0001F6D2")
+    site_display = site.replace("pokemoncenter", "Pokemon Center").title()
+
+    # Build the embed
+    embed = {
+        "title": title.strip(),
+        "description": f"**{product_name}**",
+        "url": product_url,
+        "color": color,
+        "fields": [
+            {"name": "Price", "value": price or "See link", "inline": True},
+            {"name": "Store", "value": f"{site_emoji} {site_display}", "inline": True},
+            {"name": "Link", "value": f"[\U0001F6D2 BUY NOW]({product_url})", "inline": False},
+        ],
+        "footer": {"text": "Pokemon Stock Monitor"},
+        "timestamp": datetime.utcnow().isoformat(),
+    }
+
+    # Add priority field for favorites
+    if is_favorite:
+        embed["fields"].insert(0, {
+            "name": "Priority",
+            "value": f"{priority_indicator} {priority.upper()}",
+            "inline": True
+        })
+
+    if image_url:
+        embed["thumbnail"] = {"url": image_url}
+
+    # Ping @everyone for high priority favorites, or just regular ping otherwise
+    if is_favorite and priority == 'high':
+        content = "@everyone \U0001F6A8 HIGH PRIORITY ALERT!"
+    else:
+        content = "@everyone"
+
+    payload = {
+        "content": content,
+        "embeds": [embed],
+    }
+
+    try:
+        response = requests.post(
+            webhook_url,
+            json=payload,
+            timeout=10,
+        )
+        response.raise_for_status()
+        logger.info(f"Discord notification sent for: {product_name} (priority: {priority})")
+        # Track that we notified about this product
+        if alert_type == "new_drop":
+            mark_as_notified(product_url)
+        return True
+    except requests.exceptions.RequestException as e:
+        logger.error(f"Failed to send Discord notification: {e}")
+        return False
+
+
+def send_stock_alert_with_priority(product, alert_type: str = "restock",
+                                   priority_settings: Optional[Dict] = None):
+    """
+    Convenience method to send alert from Product object with priority settings
+
+    Args:
+        product: Product object from scraper
+        alert_type: "restock", "new_drop", or "preorder"
+        priority_settings: Optional dict with favorite/priority info
+    """
+    return send_stock_alert(
+        product_name=product.name,
+        product_url=product.url,
+        price=product.price,
+        site=product.site,
+        alert_type=alert_type,
+        image_url=product.image_url,
+        priority_settings=priority_settings
+    )
+
+
+def send_error_notification(error_message: str, site: str = "Unknown"):
+    """Send an error notification to Discord"""
+    if not NOTIFICATIONS_ENABLED:
+        return True
+    if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
+        return False
+
+    embed = {
+        "title": "\u26A0\uFE0F Monitor Error",
+        "description": error_message,
+        "color": COLOR_ERROR,
+        "fields": [{"name": "Site", "value": site, "inline": True}],
+        "footer": {"text": "Pokemon Stock Monitor"},
+        "timestamp": datetime.utcnow().isoformat(),
+    }
+
+    payload = {"embeds": [embed]}
+
+    try:
+        response = requests.post(DISCORD_WEBHOOK_URL, json=payload, timeout=10)
+        response.raise_for_status()
+        return True
+    except requests.exceptions.RequestException as e:
+        logger.error(f"Failed to send error notification: {e}")
+        return False
+
+
+def send_startup_notification():
+    """Send a notification that the monitor has started"""
+    if not NOTIFICATIONS_ENABLED:
+        return True
+    if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
+        return False
+
+    embed = {
+        "title": "\u2705 Monitor Started",
+        "description": "Pokemon Stock Monitor is now running and watching for restocks!",
+        "color": 0x00FF00,
+        "footer": {"text": "Pokemon Stock Monitor"},
+        "timestamp": datetime.utcnow().isoformat(),
+    }
+
+    payload = {"embeds": [embed]}
+
+    try:
+        response = requests.post(DISCORD_WEBHOOK_URL, json=payload, timeout=10)
+        response.raise_for_status()
+        return True
+    except requests.exceptions.RequestException as e:
+        logger.error(f"Failed to send startup notification: {e}")
+        return False
+
+
+def test_webhook():
+    """Test the Discord webhook connection"""
+    print("Testing Discord webhook...")
+
+    if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
+        print("ERROR: Webhook URL not configured!")
+        print("Edit config.py and set DISCORD_WEBHOOK_URL")
+        return False
+
+    # Send a test notification
+    success = send_stock_alert(
+        product_name="Test Product - Pokemon TCG Booster",
+        product_url="https://www.pokemoncenter.com/test",
+        price="$4.99",
+        site="pokemoncenter",
+        alert_type="restock",
+    )
+
+    if success:
+        print("SUCCESS! Check your Discord channel for the test message.")
+    else:
+        print("FAILED! Check the webhook URL and try again.")
+
+    return success
+
+
+def test_priority_webhook():
+    """Test a high priority favorite notification"""
+    print("Testing high priority webhook...")
+
+    if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
+        print("ERROR: Webhook URL not configured!")
+        return False
+
+    success = send_stock_alert(
+        product_name="Chaos Rising Elite Trainer Box",
+        product_url="https://www.pokemoncenter.com/test",
+        price="$54.99",
+        site="pokemoncenter",
+        alert_type="new_drop",
+        priority_settings={
+            'is_favorite': True,
+            'priority': 'high',
+            'custom_webhook': None
+        }
+    )
+
+    if success:
+        print("SUCCESS! Check your Discord channel for the priority test message.")
+    else:
+        print("FAILED! Check the webhook URL and try again.")
+
+    return success
+
+
+if __name__ == "__main__":
+    # Run webhook test
+    test_webhook()

src/favorites.py

@@ -0,0 +1,192 @@
+"""
+Favorites system for Pokemon Stock Monitor.
+Handles priority products and categories with custom notification settings.
+"""
+
+import logging
+from typing import Optional, List, Dict, Any
+from .database import get_database
+
+logger = logging.getLogger(__name__)
+
+
+# Common category keywords for auto-detection
+CATEGORY_KEYWORDS = {
+    "ETB": ["elite trainer", "etb"],
+    "Booster Bundle": ["booster bundle"],
+    "Booster Box": ["booster box"],
+    "Booster Pack": ["booster pack", "sleeved booster"],
+    "Collection Box": ["collection", "premium collection", "special collection"],
+    "Tin": ["tin", "mini tin"],
+    "Blister": ["blister", "3-pack", "check lane"],
+    "Accessories": ["binder", "album", "sleeves", "deck box", "playmat"],
+}
+
+# Known Pokemon TCG sets for category matching
+KNOWN_SETS = [
+    "chaos rising",
+    "prismatic evolutions",
+    "surging sparks",
+    "twilight masquerade",
+    "temporal forces",
+    "paldean fates",
+    "obsidian flames",
+    "paldea evolved",
+    "scarlet & violet",
+    "crown zenith",
+    "silver tempest",
+    "lost origin",
+    "pokemon go",
+    "astral radiance",
+    "brilliant stars",
+    "fusion strike",
+    "celebrations",
+]
+
+
+class FavoritesManager:
+    """Manages favorites and priority notifications"""
+
+    def __init__(self):
+        self.db = get_database()
+
+    def add_product_favorite(self, url: str, display_name: str = None,
+                             priority: str = "high", notify_discord: bool = True,
+                             notify_sound: bool = False, custom_webhook: str = None) -> int:
+        """Add a product URL as favorite"""
+        return self.db.add_favorite(
+            fav_type="product",
+            value=url,
+            display_name=display_name,
+            priority=priority,
+            notify_discord=notify_discord,
+            notify_sound=notify_sound,
+            custom_webhook=custom_webhook
+        )
+
+    def add_category_favorite(self, category: str, display_name: str = None,
+                              priority: str = "high", notify_discord: bool = True,
+                              notify_sound: bool = False, custom_webhook: str = None) -> int:
+        """Add a category (e.g., 'ETB', 'Chaos Rising') as favorite"""
+        return self.db.add_favorite(
+            fav_type="category",
+            value=category.lower(),
+            display_name=display_name or category,
+            priority=priority,
+            notify_discord=notify_discord,
+            notify_sound=notify_sound,
+            custom_webhook=custom_webhook
+        )
+
+    def remove_favorite(self, favorite_id: int):
+        """Remove a favorite"""
+        self.db.remove_favorite(favorite_id)
+
+    def get_all_favorites(self) -> List[Dict]:
+        """Get all favorites"""
+        return self.db.get_favorites()
+
+    def get_product_favorites(self) -> List[Dict]:
+        """Get product favorites only"""
+        return self.db.get_favorites(fav_type="product")
+
+    def get_category_favorites(self) -> List[Dict]:
+        """Get category favorites only"""
+        return self.db.get_favorites(fav_type="category")
+
+    def update_favorite(self, favorite_id: int, **kwargs):
+        """Update favorite settings"""
+        self.db.update_favorite(favorite_id, **kwargs)
+
+    def check_product_priority(self, url: str, name: str, category: str = None) -> Optional[Dict]:
+        """
+        Check if a product matches any favorites.
+        Returns favorite info if matched, None otherwise.
+        """
+        # Check direct URL match
+        fav = self.db.check_is_favorite(url=url)
+        if fav:
+            logger.debug(f"Product {name} matched favorite URL")
+            return fav
+
+        # Check category match
+        if category:
+            fav = self.db.check_is_favorite(category=category.lower())
+            if fav:
+                logger.debug(f"Product {name} matched category favorite: {category}")
+                return fav
+
+        # Check if product name contains any category favorite keywords
+        name_lower = name.lower()
+        category_favorites = self.get_category_favorites()
+
+        for fav in category_favorites:
+            keyword = fav['value'].lower()
+            if keyword in name_lower:
+                logger.debug(f"Product {name} matched keyword favorite: {keyword}")
+                return fav
+
+        return None
+
+    def get_notification_settings(self, url: str, name: str, category: str = None) -> Dict:
+        """
+        Get notification settings for a product based on favorites.
+        Returns default settings if not a favorite.
+        """
+        favorite = self.check_product_priority(url, name, category)
+
+        if favorite:
+            return {
+                'is_favorite': True,
+                'priority': favorite['priority'],
+                'notify_discord': bool(favorite['notify_discord']),
+                'notify_sound': bool(favorite['notify_sound']),
+                'custom_webhook': favorite['custom_webhook'],
+                'display_name': favorite['display_name']
+            }
+        else:
+            return {
+                'is_favorite': False,
+                'priority': 'normal',
+                'notify_discord': True,
+                'notify_sound': False,
+                'custom_webhook': None,
+                'display_name': None
+            }
+
+    def get_suggested_categories(self) -> List[str]:
+        """Get list of suggested categories for favorites"""
+        categories = list(CATEGORY_KEYWORDS.keys())
+        categories.extend([s.title() for s in KNOWN_SETS])
+        return sorted(set(categories))
+
+    def detect_matching_categories(self, name: str) -> List[str]:
+        """Detect which categories a product name matches"""
+        name_lower = name.lower()
+        matches = []
+
+        # Check product type categories
+        for category, keywords in CATEGORY_KEYWORDS.items():
+            for keyword in keywords:
+                if keyword in name_lower:
+                    matches.append(category)
+                    break
+
+        # Check set names
+        for set_name in KNOWN_SETS:
+            if set_name in name_lower:
+                matches.append(set_name.title())
+
+        return matches
+
+
+# Global favorites manager instance
+_favorites: FavoritesManager = None
+
+
+def get_favorites_manager() -> FavoritesManager:
+    """Get the global favorites manager instance"""
+    global _favorites
+    if _favorites is None:
+        _favorites = FavoritesManager()
+    return _favorites

src/news/__init__.py

@@ -0,0 +1,9 @@
+"""
+News aggregation module for Pokemon Stock Monitor.
+Fetches news from Twitter, Pokemon.com, and manual Discord input.
+"""
+
+from .base import NewsArticle
+from .sentiment import analyze_sentiment, extract_keywords
+
+__all__ = ['NewsArticle', 'analyze_sentiment', 'extract_keywords']

src/news/base.py

@@ -0,0 +1,96 @@
+"""
+Base classes and data structures for news aggregation.
+"""
+
+from dataclasses import dataclass, field
+from datetime import datetime
+from typing import Optional, List
+
+
+@dataclass
+class NewsArticle:
+    """Represents a news article from any source"""
+    source: str  # 'twitter', 'pokemon_official', 'discord_manual'
+    source_account: str  # '@PokemonRestocks', 'Pokemon.com', 'PokePings Discord'
+    content: str  # Tweet text, article summary, or message content
+    author: str  # Username or account name
+    published_at: datetime
+
+    # Optional fields
+    external_id: Optional[str] = None  # Tweet ID, article URL slug
+    title: Optional[str] = None  # For official news articles
+    url: Optional[str] = None  # Link to original source
+    image_url: Optional[str] = None  # Attached image if any
+
+    # Sentiment analysis results (populated after analysis)
+    sentiment_score: Optional[float] = None  # -1.0 to 1.0
+    sentiment_label: Optional[str] = None  # 'positive', 'negative', 'neutral'
+    keywords: List[str] = field(default_factory=list)  # Extracted keywords
+
+    # Correlation with stock events (populated after correlation)
+    related_product_ids: List[int] = field(default_factory=list)
+    is_drop_related: bool = False
+    is_restock_related: bool = False
+
+    def __hash__(self):
+        """Hash by external_id or content for deduplication"""
+        if self.external_id:
+            return hash(self.external_id)
+        return hash((self.source, self.content[:100], self.published_at))
+
+    def __eq__(self, other):
+        if not isinstance(other, NewsArticle):
+            return False
+        if self.external_id and other.external_id:
+            return self.external_id == other.external_id
+        return (self.source == other.source and
+                self.content[:100] == other.content[:100] and
+                self.published_at == other.published_at)
+
+    def to_dict(self) -> dict:
+        """Convert to dictionary for database storage"""
+        return {
+            'source': self.source,
+            'source_account': self.source_account,
+            'external_id': self.external_id,
+            'title': self.title,
+            'content': self.content,
+            'url': self.url,
+            'author': self.author,
+            'image_url': self.image_url,
+            'published_at': self.published_at.isoformat() if self.published_at else None,
+            'sentiment_score': self.sentiment_score,
+            'sentiment_label': self.sentiment_label,
+            'keywords': self.keywords,
+            'related_product_ids': self.related_product_ids,
+            'is_drop_related': self.is_drop_related,
+            'is_restock_related': self.is_restock_related
+        }
+
+    @classmethod
+    def from_dict(cls, data: dict) -> 'NewsArticle':
+        """Create from dictionary (database row)"""
+        published_at = data.get('published_at')
+        if published_at and isinstance(published_at, str):
+            try:
+                published_at = datetime.fromisoformat(published_at)
+            except ValueError:
+                published_at = datetime.now()
+
+        return cls(
+            source=data['source'],
+            source_account=data.get('source_account', ''),
+            content=data['content'],
+            author=data.get('author', ''),
+            published_at=published_at or datetime.now(),
+            external_id=data.get('external_id'),
+            title=data.get('title'),
+            url=data.get('url'),
+            image_url=data.get('image_url'),
+            sentiment_score=data.get('sentiment_score'),
+            sentiment_label=data.get('sentiment_label'),
+            keywords=data.get('keywords', []),
+            related_product_ids=data.get('related_product_ids', []),
+            is_drop_related=data.get('is_drop_related', False),
+            is_restock_related=data.get('is_restock_related', False)
+        )

src/news/pokemon_fetcher.py

@@ -0,0 +1,222 @@
+"""
+Fetches official Pokemon TCG news from Pokemon.com
+"""
+
+import logging
+import re
+import requests
+from datetime import datetime
+from typing import List, Optional
+from bs4 import BeautifulSoup
+
+from .base import NewsArticle
+from .sentiment import analyze_article
+
+logger = logging.getLogger(__name__)
+
+# Pokemon.com TCG news URL
+POKEMON_NEWS_URL = "https://www.pokemon.com/us/pokemon-tcg-news/"
+
+# Request headers to mimic browser
+HEADERS = {
+    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
+    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
+    'Accept-Language': 'en-US,en;q=0.5',
+}
+
+
+class PokemonNewsFetcher:
+    """Fetches news from Pokemon.com TCG news page"""
+
+    def __init__(self):
+        self.session = requests.Session()
+        self.session.headers.update(HEADERS)
+
+    def fetch_latest(self, limit: int = 20) -> List[NewsArticle]:
+        """
+        Fetch latest Pokemon TCG news articles.
+
+        Args:
+            limit: Maximum number of articles to fetch
+
+        Returns:
+            List of NewsArticle objects
+        """
+        try:
+            response = self.session.get(POKEMON_NEWS_URL, timeout=30)
+            response.raise_for_status()
+            return self._parse_news_page(response.text, limit)
+        except requests.RequestException as e:
+            logger.error(f"Failed to fetch Pokemon news: {e}")
+            return []
+
+    def _parse_news_page(self, html: str, limit: int) -> List[NewsArticle]:
+        """Parse the news page HTML and extract articles"""
+        soup = BeautifulSoup(html, 'html.parser')
+        articles = []
+
+        # Find article containers - Pokemon.com uses various class names
+        # Try multiple selectors for robustness
+        article_containers = soup.select('article, .news-article, .article-item, .content-block')
+
+        if not article_containers:
+            # Fallback: look for links in the main content area
+            article_containers = soup.select('.main-content a, .news-list a, section a')
+
+        for container in article_containers[:limit]:
+            try:
+                article = self._parse_article(container)
+                if article:
+                    articles.append(article)
+            except Exception as e:
+                logger.debug(f"Failed to parse article: {e}")
+                continue
+
+        logger.info(f"Fetched {len(articles)} Pokemon.com news articles")
+        return articles
+
+    def _parse_article(self, container) -> Optional[NewsArticle]:
+        """Parse a single article container"""
+        # Extract title
+        title_elem = container.select_one('h1, h2, h3, h4, .title, .headline')
+        if not title_elem:
+            # Check if container itself is a link with text
+            title = container.get_text(strip=True)
+            if not title or len(title) < 10:
+                return None
+        else:
+            title = title_elem.get_text(strip=True)
+
+        # Extract URL
+        link = container.select_one('a[href]')
+        if not link:
+            if container.name == 'a':
+                link = container
+            else:
+                return None
+
+        url = link.get('href', '')
+        if url and not url.startswith('http'):
+            url = f"https://www.pokemon.com{url}"
+
+        # Skip non-TCG articles
+        if url and '/pokemon-tcg' not in url.lower() and 'tcg' not in title.lower():
+            return None
+
+        # Extract summary/description
+        summary_elem = container.select_one('p, .summary, .description, .excerpt')
+        summary = summary_elem.get_text(strip=True) if summary_elem else title
+
+        # Extract image
+        img_elem = container.select_one('img')
+        image_url = None
+        if img_elem:
+            image_url = img_elem.get('src') or img_elem.get('data-src')
+            if image_url and not image_url.startswith('http'):
+                image_url = f"https://www.pokemon.com{image_url}"
+
+        # Extract date (if available)
+        date_elem = container.select_one('time, .date, .published-date')
+        published_at = datetime.now()
+        if date_elem:
+            date_str = date_elem.get('datetime') or date_elem.get_text(strip=True)
+            published_at = self._parse_date(date_str) or datetime.now()
+
+        # Generate external ID from URL
+        external_id = f"pokemon_{self._url_to_id(url)}"
+
+        # Analyze sentiment
+        analysis = analyze_article(summary, title)
+
+        article = NewsArticle(
+            source='pokemon_official',
+            source_account='Pokemon.com',
+            external_id=external_id,
+            title=title,
+            content=summary,
+            url=url,
+            author='The Pokemon Company',
+            image_url=image_url,
+            published_at=published_at,
+            sentiment_score=analysis['sentiment_score'],
+            sentiment_label=analysis['sentiment_label'],
+            keywords=analysis['keywords'],
+            is_drop_related=analysis['is_drop_related'],
+            is_restock_related=analysis['is_restock_related']
+        )
+
+        return article
+
+    def _parse_date(self, date_str: str) -> Optional[datetime]:
+        """Parse various date formats"""
+        if not date_str:
+            return None
+
+        # Try ISO format first
+        try:
+            return datetime.fromisoformat(date_str.replace('Z', '+00:00'))
+        except ValueError:
+            pass
+
+        # Try common formats
+        formats = [
+            '%B %d, %Y',  # January 15, 2024
+            '%b %d, %Y',  # Jan 15, 2024
+            '%m/%d/%Y',   # 01/15/2024
+            '%Y-%m-%d',   # 2024-01-15
+        ]
+
+        for fmt in formats:
+            try:
+                return datetime.strptime(date_str.strip(), fmt)
+            except ValueError:
+                continue
+
+        return None
+
+    def _url_to_id(self, url: str) -> str:
+        """Convert URL to a simple ID"""
+        if not url:
+            return str(hash(datetime.now()))
+
+        # Extract the last path segment
+        parts = url.rstrip('/').split('/')
+        slug = parts[-1] if parts else ''
+
+        # Remove query params
+        slug = slug.split('?')[0]
+
+        # If slug is too long or not useful, use hash
+        if not slug or len(slug) > 100:
+            return str(abs(hash(url)))
+
+        return slug
+
+
+def fetch_pokemon_news(limit: int = 20) -> List[NewsArticle]:
+    """
+    Convenience function to fetch Pokemon.com news.
+
+    Args:
+        limit: Maximum number of articles to fetch
+
+    Returns:
+        List of NewsArticle objects
+    """
+    fetcher = PokemonNewsFetcher()
+    return fetcher.fetch_latest(limit)
+
+
+# Allow running as a standalone script for testing
+if __name__ == '__main__':
+    logging.basicConfig(level=logging.INFO)
+
+    print("Fetching Pokemon.com TCG news...")
+    articles = fetch_pokemon_news(limit=10)
+
+    for i, article in enumerate(articles, 1):
+        print(f"\n{i}. {article.title}")
+        print(f"   Source: {article.source_account}")
+        print(f"   Sentiment: {article.sentiment_label} ({article.sentiment_score:.2f})")
+        print(f"   Keywords: {', '.join(article.keywords) if article.keywords else 'None'}")
+        print(f"   URL: {article.url}")

src/news/sentiment.py

@@ -0,0 +1,218 @@
+"""
+Sentiment analysis for Pokemon TCG news articles.
+Uses VADER (Valence Aware Dictionary and sEntiment Reasoner) from NLTK.
+"""
+
+import re
+import logging
+from typing import Tuple, List, Optional
+
+logger = logging.getLogger(__name__)
+
+# Try to import NLTK's VADER
+try:
+    import nltk
+    from nltk.sentiment.vader import SentimentIntensityAnalyzer
+    NLTK_AVAILABLE = True
+except ImportError:
+    NLTK_AVAILABLE = False
+    logger.warning("NLTK not installed. Sentiment analysis will use fallback method.")
+
+
+# Keywords that indicate stock-related news
+STOCK_KEYWORDS = {
+    'positive': [
+        'restock', 'restocked', 'restocking',
+        'live', 'in stock', 'instock', 'in-stock',
+        'available', 'back in stock', 'just dropped',
+        'drop', 'dropped', 'dropping', 'new drop',
+        'go go go', 'hurry', 'quick', 'fast',
+        'preorder', 'pre-order', 'pre order',
+        'release', 'released', 'launching'
+    ],
+    'negative': [
+        'sold out', 'soldout', 'oos', 'out of stock',
+        'unavailable', 'gone', 'missed', 'too late',
+        'cancelled', 'canceled', 'delayed',
+        'scalper', 'scalped', 'bot', 'botted'
+    ],
+    'neutral': [
+        'announcement', 'announced', 'reveal', 'revealed',
+        'coming soon', 'upcoming', 'rumor', 'leak',
+        'spoiler', 'preview', 'teaser'
+    ]
+}
+
+# Pokemon TCG product keywords for extraction
+PRODUCT_KEYWORDS = [
+    'etb', 'elite trainer box', 'elite trainer',
+    'booster box', 'booster bundle', 'booster pack',
+    'collection', 'premium collection', 'super premium',
+    'tin', 'blister', 'pack', 'box',
+    'prismatic evolutions', 'surging sparks', 'stellar crown',
+    'twilight masquerade', 'temporal forces', 'paldean fates',
+    'obsidian flames', 'scarlet violet', '151', 'sv',
+    'charizard', 'pikachu', 'mewtwo', 'mew',
+    'pokemon center', 'target', 'walmart', 'best buy', 'gamestop'
+]
+
+
+def _ensure_vader_data():
+    """Download VADER lexicon if not present"""
+    if not NLTK_AVAILABLE:
+        return False
+    try:
+        nltk.data.find('sentiment/vader_lexicon.zip')
+        return True
+    except LookupError:
+        try:
+            logger.info("Downloading VADER lexicon...")
+            nltk.download('vader_lexicon', quiet=True)
+            return True
+        except Exception as e:
+            logger.error(f"Failed to download VADER lexicon: {e}")
+            return False
+
+
+def analyze_sentiment(text: str) -> Tuple[float, str]:
+    """
+    Analyze sentiment of text using VADER.
+
+    Args:
+        text: The text to analyze
+
+    Returns:
+        Tuple of (compound_score, label)
+        - compound_score: float from -1.0 (negative) to 1.0 (positive)
+        - label: 'positive', 'negative', or 'neutral'
+    """
+    if not text:
+        return 0.0, 'neutral'
+
+    # Try VADER first
+    if NLTK_AVAILABLE and _ensure_vader_data():
+        try:
+            sia = SentimentIntensityAnalyzer()
+            scores = sia.polarity_scores(text)
+            compound = scores['compound']
+
+            if compound >= 0.05:
+                label = 'positive'
+            elif compound <= -0.05:
+                label = 'negative'
+            else:
+                label = 'neutral'
+
+            return compound, label
+        except Exception as e:
+            logger.warning(f"VADER analysis failed: {e}, using fallback")
+
+    # Fallback: simple keyword-based sentiment
+    return _fallback_sentiment(text)
+
+
+def _fallback_sentiment(text: str) -> Tuple[float, str]:
+    """Simple keyword-based sentiment analysis as fallback"""
+    text_lower = text.lower()
+
+    positive_count = sum(1 for kw in STOCK_KEYWORDS['positive'] if kw in text_lower)
+    negative_count = sum(1 for kw in STOCK_KEYWORDS['negative'] if kw in text_lower)
+
+    # Simple scoring
+    if positive_count > negative_count:
+        score = min(0.5 + (positive_count * 0.1), 1.0)
+        return score, 'positive'
+    elif negative_count > positive_count:
+        score = max(-0.5 - (negative_count * 0.1), -1.0)
+        return score, 'negative'
+    else:
+        return 0.0, 'neutral'
+
+
+def extract_keywords(text: str) -> List[str]:
+    """
+    Extract relevant Pokemon TCG keywords from text.
+
+    Args:
+        text: The text to analyze
+
+    Returns:
+        List of found keywords
+    """
+    if not text:
+        return []
+
+    text_lower = text.lower()
+    found = []
+
+    for keyword in PRODUCT_KEYWORDS:
+        if keyword in text_lower:
+            found.append(keyword)
+
+    # Remove duplicates while preserving order
+    seen = set()
+    unique = []
+    for kw in found:
+        if kw not in seen:
+            seen.add(kw)
+            unique.append(kw)
+
+    return unique
+
+
+def detect_stock_status(text: str) -> Tuple[bool, bool]:
+    """
+    Detect if news is related to drops or restocks.
+
+    Args:
+        text: The text to analyze
+
+    Returns:
+        Tuple of (is_drop_related, is_restock_related)
+    """
+    if not text:
+        return False, False
+
+    text_lower = text.lower()
+
+    # Check for drop-related keywords
+    drop_keywords = ['new drop', 'just dropped', 'dropping', 'release', 'launched', 'preorder', 'pre-order']
+    is_drop = any(kw in text_lower for kw in drop_keywords)
+
+    # Check for restock-related keywords
+    restock_keywords = ['restock', 'back in stock', 'restocked', 'restocking', 'available again']
+    is_restock = any(kw in text_lower for kw in restock_keywords)
+
+    # Generic "in stock" or "live" could be either
+    if not is_drop and not is_restock:
+        if any(kw in text_lower for kw in ['live', 'in stock', 'available', 'go go']):
+            # Default to restock if not clearly a new release
+            is_restock = True
+
+    return is_drop, is_restock
+
+
+def analyze_article(text: str, title: str = None) -> dict:
+    """
+    Full analysis of a news article.
+
+    Args:
+        text: Article content
+        title: Optional title for additional context
+
+    Returns:
+        Dictionary with analysis results
+    """
+    full_text = f"{title or ''} {text}".strip()
+
+    sentiment_score, sentiment_label = analyze_sentiment(full_text)
+    keywords = extract_keywords(full_text)
+    is_drop, is_restock = detect_stock_status(full_text)
+
+    return {
+        'sentiment_score': sentiment_score,
+        'sentiment_label': sentiment_label,
+        'keywords': keywords,
+        'is_drop_related': is_drop,
+        'is_restock_related': is_restock
+    }

src/product_tracker.py

@@ -0,0 +1,246 @@
+"""
+Product tracker - keeps track of known products to detect new drops and restocks
+Now integrates with SQLite database for historical tracking.
+"""
+
+import json
+import logging
+from pathlib import Path
+from typing import Dict, List, Set, Optional, Tuple
+from dataclasses import asdict
+from datetime import datetime
+
+from scrapers.base import Product
+from .database import get_database
+from .favorites import get_favorites_manager
+
+logger = logging.getLogger(__name__)
+
+# File to store known products (kept for backwards compatibility)
+PRODUCTS_FILE = Path(__file__).parent.parent / "data" / "products.json"
+
+
+class ProductTracker:
+    """Tracks known products to detect new listings and stock changes"""
+
+    def __init__(self, products_file: Path = PRODUCTS_FILE):
+        self.products_file = products_file
+        self.products: Dict[str, dict] = {}  # URL -> product data
+        self.db = get_database()
+        self.favorites = get_favorites_manager()
+        self.load()
+
+    def load(self):
+        """Load products from file"""
+        if self.products_file.exists():
+            try:
+                with open(self.products_file, "r", encoding="utf-8") as f:
+                    self.products = json.load(f)
+                logger.info(f"Loaded {len(self.products)} tracked products")
+            except (json.JSONDecodeError, IOError) as e:
+                logger.error(f"Error loading products file: {e}")
+                self.products = {}
+        else:
+            self.products = {}
+            logger.info("No existing products file, starting fresh")
+
+    def save(self):
+        """Save products to file"""
+        try:
+            with open(self.products_file, "w", encoding="utf-8") as f:
+                json.dump(self.products, f, indent=2, ensure_ascii=False)
+            logger.debug(f"Saved {len(self.products)} products")
+        except IOError as e:
+            logger.error(f"Error saving products file: {e}")
+
+    def process_products(self, products: List[Product]) -> Tuple[List[Product], List[Product]]:
+        """
+        Process a list of scraped products and detect changes.
+        Also logs events to database for stats tracking.
+
+        Args:
+            products: List of products from scraper
+
+        Returns:
+            Tuple of (new_products, restocked_products)
+        """
+        new_products = []
+        restocked_products = []
+
+        for product in products:
+            url = product.url
+            now = datetime.now().isoformat()
+
+            # Get or create product in database
+            db_product_id = self.db.get_or_create_product(
+                url=url,
+                name=product.name,
+                site=product.site,
+                product_id=product.product_id,
+                image_url=product.image_url,
+                price=product.price,
+                in_stock=product.in_stock
+            )
+
+            if url not in self.products:
+                # New product!
+                new_products.append(product)
+                self.products[url] = {
+                    "name": product.name,
+                    "url": url,
+                    "price": product.price,
+                    "in_stock": product.in_stock,
+                    "image_url": product.image_url,
+                    "site": product.site,
+                    "product_id": product.product_id,
+                    "first_seen": now,
+                    "last_seen": now,
+                    "last_in_stock": now if product.in_stock else None,
+                    "db_id": db_product_id,
+                }
+                logger.info(f"NEW PRODUCT: {product.name}")
+
+                # Log new drop event to database
+                self.db.record_stock_event(db_product_id, "new_drop")
+                if product.in_stock:
+                    self.db.record_stock_event(db_product_id, "in_stock")
+
+                # Record initial price
+                if product.price:
+                    self.db.record_price(db_product_id, product.price)
+
+            else:
+                # Existing product - check for changes
+                existing = self.products[url]
+                was_in_stock = existing.get("in_stock", False)
+                old_price = existing.get("price")
+
+                # Update last seen
+                existing["last_seen"] = now
+                existing["image_url"] = product.image_url or existing.get("image_url")
+                existing["db_id"] = db_product_id
+
+                # Check for price change
+                if product.price and product.price != old_price:
+                    existing["price"] = product.price
+                    self.db.record_price(db_product_id, product.price)
+                    logger.debug(f"Price change for {product.name}: {old_price} -> {product.price}")
+
+                # Check for restock
+                if product.in_stock and not was_in_stock:
+                    # RESTOCK!
+                    restocked_products.append(product)
+                    existing["last_in_stock"] = now
+                    logger.info(f"RESTOCK: {product.name}")
+
+                    # Log restock event to database
+                    self.db.record_stock_event(db_product_id, "restock")
+                    self.db.record_stock_event(db_product_id, "in_stock")
+
+                # Check for out of stock
+                elif not product.in_stock and was_in_stock:
+                    # Just went out of stock
+                    self.db.record_stock_event(db_product_id, "out_of_stock")
+                    logger.debug(f"Out of stock: {product.name}")
+
+                existing["in_stock"] = product.in_stock
+                self.products[url] = existing
+
+        self.save()
+        return new_products, restocked_products
+
+    def process_products_with_priority(self, products: List[Product]) -> Tuple[List[Tuple[Product, dict]], List[Tuple[Product, dict]]]:
+        """
+        Process products and return with priority/favorite info.
+
+        Returns:
+            Tuple of (new_products_with_priority, restocked_products_with_priority)
+            Each item is a tuple of (Product, notification_settings)
+        """
+        new_products, restocked_products = self.process_products(products)
+
+        def add_priority(product: Product) -> Tuple[Product, dict]:
+            # Get product from DB for category
+            db_product = self.db.get_product_by_url(product.url)
+            category = db_product.get('category') if db_product else None
+
+            settings = self.favorites.get_notification_settings(
+                url=product.url,
+                name=product.name,
+                category=category
+            )
+            return (product, settings)
+
+        new_with_priority = [add_priority(p) for p in new_products]
+        restocked_with_priority = [add_priority(p) for p in restocked_products]
+
+        return new_with_priority, restocked_with_priority
+
+    def log_check(self, site: str, products_found: int, new_count: int,
+                  restock_count: int, duration_ms: int, success: bool = True,
+                  error_message: str = None):
+        """Log a monitoring check to database"""
+        self.db.log_check(
+            site=site,
+            products_found=products_found,
+            new_products=new_count,
+            restocks=restock_count,
+            duration_ms=duration_ms,
+            success=success,
+            error_message=error_message
+        )
+
+    def get_known_urls(self) -> Set[str]:
+        """Get all known product URLs"""
+        return set(self.products.keys())
+
+    def get_product(self, url: str) -> Optional[dict]:
+        """Get a specific product by URL"""
+        return self.products.get(url)
+
+    def mark_out_of_stock(self, url: str):
+        """Mark a product as out of stock"""
+        if url in self.products:
+            self.products[url]["in_stock"] = False
+
+            # Also update database
+            db_product = self.db.get_product_by_url(url)
+            if db_product:
+                self.db.record_stock_event(db_product['id'], "out_of_stock")
+
+            self.save()
+
+    def clear(self):
+        """Clear all tracked products (JSON only, database preserved for history)"""
+        self.products = {}
+        self.save()
+        logger.info("Cleared all tracked products from JSON")
+
+    def get_stats(self) -> dict:
+        """Get tracking statistics from both JSON and database"""
+        # JSON stats
+        total = len(self.products)
+        in_stock = sum(1 for p in self.products.values() if p.get("in_stock", False))
+        out_of_stock = total - in_stock
+
+        # Database stats
+        db_stats = self.db.get_dashboard_stats()
+
+        return {
+            "total_products": total,
+            "in_stock": in_stock,
+            "out_of_stock": out_of_stock,
+            # From database
+            "new_drops_today": db_stats.get("new_drops_today", 0),
+            "new_drops_week": db_stats.get("new_drops_week", 0),
+            "restocks_today": db_stats.get("restocks_today", 0),
+            "restocks_week": db_stats.get("restocks_week", 0),
+            "last_check": db_stats.get("last_check"),
+            "favorites_count": db_stats.get("favorites_count", 0),
+        }
+
+    def migrate_to_database(self):
+        """Migrate existing products.json to database"""
+        if self.products_file.exists():
+            self.db.migrate_from_json(str(self.products_file))
+            logger.info("Migration to database complete")

src/scraper_state.py

@@ -0,0 +1,264 @@
+"""
+Scraper State Manager
+Manages runtime state for scrapers - enabling/disabling, running status, etc.
+"""
+
+import json
+import os
+import subprocess
+import sys
+import threading
+import time
+from datetime import datetime
+from pathlib import Path
+
+STATE_FILE = Path(__file__).parent.parent / "data" / "scraper_state.json"
+
+# Default state
+DEFAULT_STATE = {
+    "scrapers": {
+        "pokemoncenter": {"enabled": False, "running": False, "last_run": None, "last_error": None},
+        "target": {"enabled": True, "running": False, "last_run": None, "last_error": None},
+        "gamestop": {"enabled": False, "running": False, "last_run": None, "last_error": None},
+        "walmart": {"enabled": False, "running": False, "last_run": None, "last_error": None},
+        "bestbuy": {"enabled": False, "running": False, "last_run": None, "last_error": None},
+    },
+    "monitor_running": False,
+    "monitor_pid": None,
+    "check_interval": 60,
+    "last_check": None,
+}
+
+
+class ScraperStateManager:
+    """Manages scraper state with file persistence"""
+
+    _instance = None
+    _lock = threading.Lock()
+    _monitor_process = None
+
+    def __new__(cls):
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+            cls._instance._initialized = False
+        return cls._instance
+
+    def __init__(self):
+        if self._initialized:
+            return
+        self._initialized = True
+        self.state = self._load_state()
+
+    def _load_state(self) -> dict:
+        """Load state from file or return defaults"""
+        try:
+            if STATE_FILE.exists():
+                with open(STATE_FILE, 'r') as f:
+                    saved = json.load(f)
+                    # Merge with defaults in case new fields added
+                    state = DEFAULT_STATE.copy()
+                    state.update(saved)
+                    # Ensure all scrapers exist
+                    for scraper in DEFAULT_STATE["scrapers"]:
+                        if scraper not in state["scrapers"]:
+                            state["scrapers"][scraper] = DEFAULT_STATE["scrapers"][scraper]
+                    return state
+        except Exception:
+            pass
+        return DEFAULT_STATE.copy()
+
+    def _save_state(self):
+        """Save state to file"""
+        with self._lock:
+            try:
+                with open(STATE_FILE, 'w') as f:
+                    json.dump(self.state, f, indent=2, default=str)
+            except Exception as e:
+                print(f"Error saving scraper state: {e}")
+
+    def get_state(self) -> dict:
+        """Get current state"""
+        # Check if monitor is still running
+        if self.state["monitor_pid"]:
+            try:
+                # Check if process exists
+                import psutil
+                if not psutil.pid_exists(self.state["monitor_pid"]):
+                    self.state["monitor_running"] = False
+                    self.state["monitor_pid"] = None
+                    for scraper in self.state["scrapers"]:
+                        self.state["scrapers"][scraper]["running"] = False
+                    self._save_state()
+            except ImportError:
+                pass  # psutil not installed, skip check
+
+        return self.state.copy()
+
+    def set_scraper_enabled(self, scraper: str, enabled: bool) -> bool:
+        """Enable or disable a scraper"""
+        if scraper not in self.state["scrapers"]:
+            return False
+
+        self.state["scrapers"][scraper]["enabled"] = enabled
+        self._save_state()
+        return True
+
+    def get_enabled_scrapers(self) -> list:
+        """Get list of enabled scraper names"""
+        return [
+            name for name, data in self.state["scrapers"].items()
+            if data.get("enabled", False)
+        ]
+
+    def set_check_interval(self, seconds: int) -> bool:
+        """Set check interval"""
+        if seconds < 10:
+            return False
+        self.state["check_interval"] = seconds
+        self._save_state()
+        return True
+
+    def record_check(self, scraper: str, success: bool, error: str = None):
+        """Record a scraper check"""
+        if scraper in self.state["scrapers"]:
+            self.state["scrapers"][scraper]["last_run"] = datetime.now().isoformat()
+            if not success:
+                self.state["scrapers"][scraper]["last_error"] = error
+            else:
+                self.state["scrapers"][scraper]["last_error"] = None
+        self.state["last_check"] = datetime.now().isoformat()
+        self._save_state()
+
+    def start_monitor(self) -> dict:
+        """Start the monitor process"""
+        if self.state["monitor_running"]:
+            return {"success": False, "error": "Monitor already running"}
+
+        enabled = self.get_enabled_scrapers()
+        if not enabled:
+            return {"success": False, "error": "No scrapers enabled"}
+
+        try:
+            # Start main.py as a subprocess
+            project_dir = Path(__file__).parent.parent
+            main_script = project_dir / "main.py"
+
+            # Create a modified config for enabled scrapers
+            self._update_config_file(enabled)
+
+            # Start the process
+            if sys.platform == 'win32':
+                # Windows: use CREATE_NEW_PROCESS_GROUP for proper handling
+                process = subprocess.Popen(
+                    [sys.executable, str(main_script)],
+                    cwd=str(project_dir),
+                    creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
+                    stdout=subprocess.PIPE,
+                    stderr=subprocess.PIPE,
+                )
+            else:
+                process = subprocess.Popen(
+                    [sys.executable, str(main_script)],
+                    cwd=str(project_dir),
+                    start_new_session=True,
+                    stdout=subprocess.PIPE,
+                    stderr=subprocess.PIPE,
+                )
+
+            self._monitor_process = process
+            self.state["monitor_running"] = True
+            self.state["monitor_pid"] = process.pid
+
+            for scraper in enabled:
+                self.state["scrapers"][scraper]["running"] = True
+
+            self._save_state()
+
+            return {"success": True, "pid": process.pid, "scrapers": enabled}
+
+        except Exception as e:
+            return {"success": False, "error": str(e)}
+
+    def stop_monitor(self) -> dict:
+        """Stop the monitor process"""
+        if not self.state["monitor_running"] and not self.state["monitor_pid"]:
+            return {"success": False, "error": "Monitor not running"}
+
+        try:
+            pid = self.state["monitor_pid"]
+
+            if pid:
+                # On Windows, use taskkill for reliable process termination
+                if sys.platform == 'win32':
+                    try:
+                        # Kill the process tree forcefully
+                        subprocess.run(
+                            ['taskkill', '/F', '/T', '/PID', str(pid)],
+                            capture_output=True,
+                            timeout=10
+                        )
+                    except Exception:
+                        pass
+                else:
+                    # On Unix, use psutil
+                    try:
+                        import psutil
+                        proc = psutil.Process(pid)
+                        # Kill child processes first
+                        children = proc.children(recursive=True)
+                        for child in children:
+                            try:
+                                child.kill()  # Use kill instead of terminate
+                            except:
+                                pass
+                        # Kill main process
+                        proc.kill()
+                        proc.wait(timeout=5)
+                    except:
+                        pass
+
+            self.state["monitor_running"] = False
+            self.state["monitor_pid"] = None
+            for scraper in self.state["scrapers"]:
+                self.state["scrapers"][scraper]["running"] = False
+            self._save_state()
+
+            return {"success": True}
+
+        except Exception as e:
+            # Still update state even if kill failed
+            self.state["monitor_running"] = False
+            self.state["monitor_pid"] = None
+            for scraper in self.state["scrapers"]:
+                self.state["scrapers"][scraper]["running"] = False
+            self._save_state()
+            return {"success": True, "warning": f"Process may need manual cleanup: {str(e)}"}
+
+    def _update_config_file(self, enabled_scrapers: list):
+        """Update config.py with enabled scrapers"""
+        config_path = Path(__file__).parent.parent / "config.py"
+
+        try:
+            with open(config_path, 'r') as f:
+                content = f.read()
+
+            # Find and replace SITES_ENABLED line
+            import re
+            new_line = f'SITES_ENABLED = {json.dumps(enabled_scrapers)}'
+            pattern = r'SITES_ENABLED\s*=\s*\[.*?\]'
+            content = re.sub(pattern, new_line, content)
+
+            # Also update CHECK_INTERVAL_SECONDS
+            interval_line = f'CHECK_INTERVAL_SECONDS = {self.state["check_interval"]}'
+            interval_pattern = r'CHECK_INTERVAL_SECONDS\s*=\s*\d+'
+            content = re.sub(interval_pattern, interval_line, content)
+
+            with open(config_path, 'w') as f:
+                f.write(content)
+
+        except Exception as e:
+            print(f"Error updating config: {e}")
+
+
+# Singleton instance
+scraper_state = ScraperStateManager()

src/store_locator.py

@@ -0,0 +1,382 @@
+"""
+Store Locator Module
+
+Finds Pokemon TCG products at local retail stores.
+Supports Target, Best Buy, GameStop, Walmart.
+"""
+
+import json
+import requests
+import logging
+from typing import List, Dict, Optional
+from dataclasses import dataclass
+from pathlib import Path
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# User location config
+CONFIG_FILE = Path(__file__).parent / "location_config.json"
+
+
+@dataclass
+class Store:
+    name: str
+    retailer: str
+    address: str
+    city: str
+    state: str
+    zip_code: str
+    distance_miles: float
+    phone: Optional[str] = None
+    store_id: Optional[str] = None
+
+
+@dataclass
+class StoreStock:
+    store: Store
+    product_name: str
+    product_url: str
+    in_stock: bool
+    quantity: Optional[int] = None
+    price: Optional[str] = None
+
+
+class StoreLocator:
+    """Base class for store locators"""
+
+    def __init__(self, zip_code: str, radius_miles: int = 25):
+        self.zip_code = zip_code
+        self.radius_miles = radius_miles
+        self.session = requests.Session()
+        self.session.headers.update({
+            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
+        })
+
+    def find_stores(self) -> List[Store]:
+        """Find nearby stores - override in subclasses"""
+        raise NotImplementedError
+
+    def check_stock(self, product_id: str) -> List[StoreStock]:
+        """Check stock at nearby stores - override in subclasses"""
+        raise NotImplementedError
+
+
+class TargetLocator(StoreLocator):
+    """Target store locator and stock checker"""
+
+    BASE_URL = "https://redsky.target.com"
+
+    def find_stores(self) -> List[Store]:
+        """Find nearby Target stores"""
+        url = f"{self.BASE_URL}/v3/stores/nearby/{self.zip_code}"
+        params = {
+            "limit": 20,
+            "within": self.radius_miles,
+            "key": "9f36aeafbe60771e321a7cc95a78140772ab3e96"  # Public API key
+        }
+
+        try:
+            response = self.session.get(url, params=params, timeout=10)
+            if response.status_code == 200:
+                data = response.json()
+                stores = []
+                for loc in data.get("locations", []):
+                    addr = loc.get("address", {})
+                    stores.append(Store(
+                        name=loc.get("location_name", "Target"),
+                        retailer="Target",
+                        address=addr.get("address_line1", ""),
+                        city=addr.get("city", ""),
+                        state=addr.get("state", ""),
+                        zip_code=addr.get("postal_code", ""),
+                        distance_miles=loc.get("distance", 0),
+                        phone=loc.get("telephone", ""),
+                        store_id=str(loc.get("location_id", ""))
+                    ))
+                return stores
+        except Exception as e:
+            logger.error(f"Error finding Target stores: {e}")
+
+        return []
+
+    def check_stock(self, tcin: str, store_ids: List[str] = None) -> List[StoreStock]:
+        """
+        Check Target stock for a product (TCIN).
+
+        Popular Pokemon TCINs:
+        - 89776270: Pokemon TCG general
+        - Search for specific products on Target.com and get TCIN from URL
+        """
+        if not store_ids:
+            stores = self.find_stores()
+            store_ids = [s.store_id for s in stores[:5]]  # Check closest 5
+
+        results = []
+
+        for store_id in store_ids:
+            url = f"{self.BASE_URL}/v3/fulfillment/stores/{store_id}/products/{tcin}"
+            params = {
+                "key": "9f36aeafbe60771e321a7cc95a78140772ab3e96"
+            }
+
+            try:
+                response = self.session.get(url, params=params, timeout=10)
+                if response.status_code == 200:
+                    data = response.json()
+                    product = data.get("product", {})
+                    avail = product.get("fulfillment", {}).get("store_options", [])
+
+                    for opt in avail:
+                        if opt.get("location_id") == store_id:
+                            in_stock = opt.get("in_store_only", {}).get("availability_status") == "IN_STOCK"
+                            results.append(StoreStock(
+                                store=Store(
+                                    name="Target",
+                                    retailer="Target",
+                                    address="",
+                                    city="",
+                                    state="",
+                                    zip_code="",
+                                    distance_miles=0,
+                                    store_id=store_id
+                                ),
+                                product_name=product.get("item", {}).get("product_description", {}).get("title", "Unknown"),
+                                product_url=f"https://www.target.com/p/-/A-{tcin}",
+                                in_stock=in_stock,
+                                price=product.get("price", {}).get("current_retail_min")
+                            ))
+            except Exception as e:
+                logger.error(f"Error checking Target stock: {e}")
+
+        return results
+
+
+class BestBuyLocator(StoreLocator):
+    """Best Buy store locator and stock checker"""
+
+    def find_stores(self) -> List[Store]:
+        """Find nearby Best Buy stores"""
+        url = "https://www.bestbuy.com/site/store-locator/v1/stores"
+        params = {
+            "postalCode": self.zip_code,
+            "radius": self.radius_miles
+        }
+
+        try:
+            response = self.session.get(url, params=params, timeout=10)
+            if response.status_code == 200:
+                data = response.json()
+                stores = []
+                for store in data.get("stores", []):
+                    stores.append(Store(
+                        name=store.get("name", "Best Buy"),
+                        retailer="Best Buy",
+                        address=store.get("address", ""),
+                        city=store.get("city", ""),
+                        state=store.get("state", ""),
+                        zip_code=store.get("postalCode", ""),
+                        distance_miles=store.get("distance", 0),
+                        phone=store.get("phone", ""),
+                        store_id=str(store.get("storeId", ""))
+                    ))
+                return stores
+        except Exception as e:
+            logger.error(f"Error finding Best Buy stores: {e}")
+
+        return []
+
+    def check_stock(self, sku: str) -> List[StoreStock]:
+        """
+        Check Best Buy stock for a product (SKU).
+
+        Example: Search for Pokemon on bestbuy.com, get SKU from product page
+        """
+        stores = self.find_stores()[:5]
+        store_ids = [s.store_id for s in stores]
+
+        if not store_ids:
+            return []
+
+        url = f"https://www.bestbuy.com/fulfillment/v1/stores/{','.join(store_ids)}/sku/{sku}"
+
+        try:
+            response = self.session.get(url, timeout=10)
+            if response.status_code == 200:
+                data = response.json()
+                results = []
+                for store_data in data.get("stores", []):
+                    store = next((s for s in stores if s.store_id == str(store_data.get("storeId"))), None)
+                    if store:
+                        results.append(StoreStock(
+                            store=store,
+                            product_name=data.get("productName", "Unknown"),
+                            product_url=f"https://www.bestbuy.com/site/-/{sku}.p",
+                            in_stock=store_data.get("storeAvailability", {}).get("inStoreAvailable", False),
+                            quantity=store_data.get("storeAvailability", {}).get("quantity")
+                        ))
+                return results
+        except Exception as e:
+            logger.error(f"Error checking Best Buy stock: {e}")
+
+        return []
+
+
+class GameStopLocator(StoreLocator):
+    """GameStop store locator"""
+
+    def find_stores(self) -> List[Store]:
+        """Find nearby GameStop stores"""
+        url = "https://www.gamestop.com/api/location/v1/stores/search"
+        params = {
+            "postalCode": self.zip_code,
+            "radius": self.radius_miles,
+            "count": 20
+        }
+
+        try:
+            response = self.session.get(url, params=params, timeout=10)
+            if response.status_code == 200:
+                data = response.json()
+                stores = []
+                for store in data.get("stores", []):
+                    stores.append(Store(
+                        name=store.get("storeName", "GameStop"),
+                        retailer="GameStop",
+                        address=store.get("address1", ""),
+                        city=store.get("city", ""),
+                        state=store.get("state", ""),
+                        zip_code=store.get("postalCode", ""),
+                        distance_miles=store.get("distance", 0),
+                        phone=store.get("phone", ""),
+                        store_id=str(store.get("storeNumber", ""))
+                    ))
+                return stores
+        except Exception as e:
+            logger.error(f"Error finding GameStop stores: {e}")
+
+        return []
+
+
+class WalmartLocator(StoreLocator):
+    """Walmart store locator"""
+
+    def find_stores(self) -> List[Store]:
+        """Find nearby Walmart stores"""
+        url = "https://www.walmart.com/store/finder/electrode/api/stores"
+        params = {
+            "singleLineAddr": self.zip_code,
+            "distance": self.radius_miles
+        }
+
+        try:
+            response = self.session.get(url, params=params, timeout=10)
+            if response.status_code == 200:
+                data = response.json()
+                stores = []
+                for store in data.get("payload", {}).get("storesData", {}).get("stores", []):
+                    addr = store.get("address", {})
+                    stores.append(Store(
+                        name=store.get("displayName", "Walmart"),
+                        retailer="Walmart",
+                        address=addr.get("streetAddress", ""),
+                        city=addr.get("city", ""),
+                        state=addr.get("state", ""),
+                        zip_code=addr.get("postalCode", ""),
+                        distance_miles=store.get("distance", 0),
+                        phone=store.get("phone", ""),
+                        store_id=str(store.get("id", ""))
+                    ))
+                return stores
+        except Exception as e:
+            logger.error(f"Error finding Walmart stores: {e}")
+
+        return []
+
+
+# Pokemon TCG product IDs for various retailers
+POKEMON_PRODUCTS = {
+    "target": {
+        "pokemon_tcg_general": "89776270",
+        "etb_search": "pokemon+elite+trainer+box",
+    },
+    "bestbuy": {
+        "pokemon_cards": "6542167",
+    },
+    "gamestop": {
+        "pokemon_tcg": "pokemon-trading-cards",
+    },
+    "walmart": {
+        "pokemon_tcg": "pokemon-trading-card-game",
+    }
+}
+
+
+def load_location_config() -> Dict:
+    """Load user location configuration"""
+    if CONFIG_FILE.exists():
+        with open(CONFIG_FILE, 'r') as f:
+            return json.load(f)
+    return {"zip_code": None, "radius_miles": 25}
+
+
+def save_location_config(zip_code: str, radius_miles: int = 25):
+    """Save user location configuration"""
+    config = {
+        "zip_code": zip_code,
+        "radius_miles": radius_miles
+    }
+    with open(CONFIG_FILE, 'w') as f:
+        json.dump(config, f, indent=2)
+    logger.info(f"Location saved: {zip_code}, radius: {radius_miles} miles")
+
+
+def find_all_nearby_stores(zip_code: str, radius_miles: int = 25) -> Dict[str, List[Store]]:
+    """Find all nearby stores across all supported retailers"""
+    results = {}
+
+    locators = [
+        ("Target", TargetLocator(zip_code, radius_miles)),
+        ("Best Buy", BestBuyLocator(zip_code, radius_miles)),
+        ("GameStop", GameStopLocator(zip_code, radius_miles)),
+        ("Walmart", WalmartLocator(zip_code, radius_miles)),
+    ]
+
+    for name, locator in locators:
+        try:
+            stores = locator.find_stores()
+            results[name] = stores
+            logger.info(f"Found {len(stores)} {name} stores near {zip_code}")
+        except Exception as e:
+            logger.error(f"Error with {name}: {e}")
+            results[name] = []
+
+    return results
+
+
+def format_stores_for_discord(stores_by_retailer: Dict[str, List[Store]]) -> str:
+    """Format store list for Discord message"""
+    lines = ["**Nearby Stores:**\n"]
+
+    for retailer, stores in stores_by_retailer.items():
+        if stores:
+            lines.append(f"\n**{retailer}** ({len(stores)} stores)")
+            for store in stores[:3]:  # Show top 3
+                lines.append(f"  - {store.name} ({store.distance_miles:.1f} mi)")
+                lines.append(f"    {store.address}, {store.city}, {store.state}")
+
+    return "\n".join(lines)
+
+
+if __name__ == "__main__":
+    # Test with a sample zip code
+    test_zip = "90210"
+    print(f"Finding stores near {test_zip}...")
+
+    stores = find_all_nearby_stores(test_zip, 25)
+
+    for retailer, store_list in stores.items():
+        print(f"\n{retailer}: {len(store_list)} stores found")
+        for store in store_list[:3]:
+            print(f"  - {store.name} ({store.distance_miles:.1f} mi)")
+            print(f"    {store.address}, {store.city}, {store.state}")

tests/__init__.py

@@ -0,0 +1 @@
+# Test modules

tests/test_api.py

@@ -0,0 +1,68 @@
+"""
+Test Pokemon Center API endpoints directly
+"""
+
+import requests
+import json
+
+# Headers extracted from HAR capture
+HEADERS = {
+    "Accept": "application/json",
+    "Accept-Version": "1",
+    "Content-Type": "application/json",
+    "X-Store-Locale": "en-us",
+    "X-Store-Scope": "pokemon",
+    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36",
+    "Referer": "https://www.pokemoncenter.com/category/tcg-cards",
+}
+
+# Test endpoints
+ENDPOINTS = [
+    # Category listing
+    ("GET", "https://www.pokemoncenter.com/site/resourceapi/category/new-releases"),
+
+    # Product details (example SKU)
+    ("GET", "https://www.pokemoncenter.com/tpci-ecommweb-api/product/699-17113"),
+
+    # Product status
+    ("GET", "https://www.pokemoncenter.com/tpci-ecommweb-api/product/status/qgqvbkjwhe4s2mjxgeytg="),
+]
+
+print("=" * 70)
+print("Pokemon Center API Test")
+print("=" * 70)
+print()
+
+for method, url in ENDPOINTS:
+    print(f"[{method}] {url[:80]}...")
+
+    try:
+        if method == "GET":
+            response = requests.get(url, headers=HEADERS, timeout=10)
+        else:
+            response = requests.post(url, headers=HEADERS, timeout=10)
+
+        print(f"  Status: {response.status_code}")
+        print(f"  Content-Type: {response.headers.get('Content-Type', 'N/A')}")
+
+        if response.status_code == 200:
+            try:
+                data = response.json()
+                print(f"  Response keys: {list(data.keys())[:5]}...")
+
+                # Show a preview
+                preview = json.dumps(data, indent=2)[:500]
+                print(f"  Preview:\n{preview}")
+            except:
+                print(f"  Raw: {response.text[:200]}")
+        else:
+            print(f"  Response: {response.text[:200]}")
+
+    except Exception as e:
+        print(f"  Error: {e}")
+
+    print()
+
+print("=" * 70)
+print("If APIs return 200, we can monitor without a browser!")
+print("=" * 70)

tests/test_dashboard_api.py

@@ -0,0 +1,407 @@
+"""
+Unit tests for Dashboard REST API endpoints.
+"""
+
+import pytest
+import tempfile
+import os
+import sys
+import json
+
+# Add project root to path
+project_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+sys.path.insert(0, project_root)
+
+from flask import Flask
+from src.database import Database
+
+# Import api module directly to avoid circular imports via dashboard/__init__.py
+import importlib.util
+spec = importlib.util.spec_from_file_location(
+    "dashboard_api",
+    os.path.join(project_root, "dashboard", "api.py")
+)
+dashboard_api = importlib.util.module_from_spec(spec)
+sys.modules["dashboard_api"] = dashboard_api
+spec.loader.exec_module(dashboard_api)
+api_bp = dashboard_api.api_bp
+
+
+@pytest.fixture
+def app():
+    """Create test Flask app"""
+    app = Flask(__name__)
+    app.config['TESTING'] = True
+    app.register_blueprint(api_bp, url_prefix='/api')
+
+    # Use temp database
+    with tempfile.NamedTemporaryFile(suffix='.db', delete=False) as f:
+        db_path = f.name
+
+    # Monkey-patch the database
+    import src.database as db_module
+    db_module._db = Database(db_path)
+
+    # Also reset the favorites manager so it picks up the new database
+    import src.favorites as fav_module
+    fav_module._favorites = None  # Reset so it gets recreated with new db
+
+    yield app
+
+    # Cleanup
+    os.unlink(db_path)
+
+
+@pytest.fixture
+def client(app):
+    """Create test client"""
+    return app.test_client()
+
+
+class TestStatsEndpoints:
+    """Tests for stats and health endpoints"""
+
+    def test_get_stats(self, client):
+        """Test /api/stats endpoint"""
+        response = client.get('/api/stats')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'total_products' in data
+        assert 'in_stock_count' in data
+        assert 'new_drops_today' in data
+
+    def test_get_health(self, client):
+        """Test /api/health endpoint"""
+        response = client.get('/api/health')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert data['status'] == 'ok'
+
+
+class TestProductsEndpoints:
+    """Tests for product endpoints"""
+
+    def test_get_products_empty(self, client):
+        """Test getting products when empty"""
+        response = client.get('/api/products')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'products' in data
+        assert 'total' in data
+
+    def test_get_products_with_filters(self, client):
+        """Test product filtering"""
+        response = client.get('/api/products?site=pokemoncenter&in_stock=true')
+
+        assert response.status_code == 200
+
+    def test_get_product_not_found(self, client):
+        """Test getting non-existent product"""
+        response = client.get('/api/products/99999')
+
+        assert response.status_code == 404
+
+
+class TestEventsEndpoints:
+    """Tests for events endpoints"""
+
+    def test_get_events(self, client):
+        """Test /api/events endpoint"""
+        response = client.get('/api/events')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'events' in data
+
+    def test_get_events_with_type_filter(self, client):
+        """Test filtering events by type"""
+        response = client.get('/api/events?type=restock&type=new_drop')
+
+        assert response.status_code == 200
+
+
+class TestAnalyticsEndpoints:
+    """Tests for analytics endpoints"""
+
+    def test_get_drop_timing(self, client):
+        """Test /api/analytics/drops endpoint"""
+        response = client.get('/api/analytics/drops')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'drop_timing' in data
+
+    def test_get_stock_duration(self, client):
+        """Test /api/analytics/stock endpoint"""
+        response = client.get('/api/analytics/stock')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'stock_duration' in data
+
+    def test_get_selling_rates(self, client):
+        """Test /api/analytics/selling-rates endpoint"""
+        response = client.get('/api/analytics/selling-rates')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'products' in data
+
+    def test_get_site_stats(self, client):
+        """Test /api/analytics/sites endpoint"""
+        response = client.get('/api/analytics/sites')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'sites' in data
+
+
+class TestExtensionSyncEndpoint:
+    """Tests for Chrome extension sync endpoint"""
+
+    def test_sync_empty_data(self, client):
+        """Test syncing with no data returns error"""
+        response = client.post('/api/extension/sync',
+                               content_type='application/json')
+
+        assert response.status_code == 400
+
+    def test_sync_skus(self, client):
+        """Test syncing SKUs from extension"""
+        data = {
+            'skus': ['699-17113', '191-85953', '100-12345'],
+            'products': [],
+            'events': []
+        }
+
+        response = client.post('/api/extension/sync',
+                               data=json.dumps(data),
+                               content_type='application/json')
+
+        assert response.status_code == 200
+        result = json.loads(response.data)
+
+        assert result['success'] is True
+        assert result['total_skus'] == 3
+        assert result['new_skus_added'] == 3
+
+    def test_sync_products(self, client):
+        """Test syncing products from extension"""
+        data = {
+            'skus': [],
+            'products': [
+                {
+                    'url': 'https://www.pokemoncenter.com/product/699-17113',
+                    'name': 'Pokemon ETB Prismatic Evolutions',
+                    'price': '$49.99',
+                    'inStock': True,
+                    'imageUrl': 'https://example.com/image.jpg',
+                    'productId': '699-17113'
+                }
+            ],
+            'events': []
+        }
+
+        response = client.post('/api/extension/sync',
+                               data=json.dumps(data),
+                               content_type='application/json')
+
+        assert response.status_code == 200
+        result = json.loads(response.data)
+
+        assert result['success'] is True
+        assert result['total_products'] >= 1
+
+    def test_sync_events(self, client):
+        """Test syncing events (restocks, drops) from extension"""
+        # First sync a product
+        product_data = {
+            'skus': [],
+            'products': [
+                {
+                    'url': 'https://www.pokemoncenter.com/product/test-123',
+                    'name': 'Test Product',
+                    'price': '$29.99',
+                    'inStock': True
+                }
+            ],
+            'events': []
+        }
+        client.post('/api/extension/sync',
+                    data=json.dumps(product_data),
+                    content_type='application/json')
+
+        # Now sync events
+        event_data = {
+            'skus': [],
+            'products': [],
+            'events': [
+                {
+                    'type': 'restock',
+                    'url': 'https://www.pokemoncenter.com/product/test-123',
+                    'name': 'Test Product',
+                    'price': '$29.99',
+                    'timestamp': '2024-01-15T10:00:00Z'
+                },
+                {
+                    'type': 'out_of_stock',
+                    'url': 'https://www.pokemoncenter.com/product/test-123',
+                    'name': 'Test Product',
+                    'timestamp': '2024-01-15T10:30:00Z'
+                }
+            ]
+        }
+
+        response = client.post('/api/extension/sync',
+                               data=json.dumps(event_data),
+                               content_type='application/json')
+
+        assert response.status_code == 200
+        result = json.loads(response.data)
+
+        assert result['success'] is True
+        assert result['events_processed'] == 2
+
+    def test_sync_price_change_event(self, client):
+        """Test syncing price change events"""
+        # First create the product
+        client.post('/api/extension/sync',
+                    data=json.dumps({
+                        'products': [{
+                            'url': 'https://www.pokemoncenter.com/product/price-test',
+                            'name': 'Price Test Product',
+                            'price': '$39.99',
+                            'inStock': True
+                        }]
+                    }),
+                    content_type='application/json')
+
+        # Now send price change event
+        event_data = {
+            'events': [
+                {
+                    'type': 'price_change',
+                    'url': 'https://www.pokemoncenter.com/product/price-test',
+                    'oldPrice': '$39.99',
+                    'newPrice': '$34.99',
+                    'timestamp': '2024-01-15T12:00:00Z'
+                }
+            ]
+        }
+
+        response = client.post('/api/extension/sync',
+                               data=json.dumps(event_data),
+                               content_type='application/json')
+
+        assert response.status_code == 200
+        result = json.loads(response.data)
+        assert result['events_processed'] == 1
+
+
+class TestExtensionDataEndpoints:
+    """Tests for extension data retrieval endpoints"""
+
+    def test_get_extension_skus(self, client):
+        """Test getting extension SKUs"""
+        response = client.get('/api/extension/skus')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'skus' in data
+        assert 'total' in data
+
+    def test_get_extension_products(self, client):
+        """Test getting extension products"""
+        response = client.get('/api/extension/products')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'products' in data
+        assert 'total' in data
+
+    def test_get_extension_stats(self, client):
+        """Test getting extension stats"""
+        response = client.get('/api/extension/stats')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'total_skus' in data
+        assert 'total_products' in data
+        assert 'source' in data
+        assert data['source'] == 'chrome_extension'
+
+    def test_clear_extension_data(self, client):
+        """Test clearing extension data"""
+        response = client.post('/api/extension/clear')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert data['success'] is True
+
+
+class TestFavoritesEndpoints:
+    """Tests for favorites CRUD endpoints"""
+
+    def test_add_favorite(self, client):
+        """Test adding a favorite"""
+        data = {
+            'type': 'product',
+            'value': 'https://example.com/fav-product',
+            'display_name': 'My Favorite ETB',
+            'priority': 'high'
+        }
+
+        response = client.post('/api/favorites',
+                               data=json.dumps(data),
+                               content_type='application/json')
+
+        assert response.status_code == 201
+        result = json.loads(response.data)
+
+        assert result['success'] is True
+        assert 'id' in result
+
+    def test_get_favorites(self, client):
+        """Test getting all favorites"""
+        response = client.get('/api/favorites')
+
+        assert response.status_code == 200
+        data = json.loads(response.data)
+
+        assert 'favorites' in data
+
+    def test_delete_favorite(self, client):
+        """Test deleting a favorite"""
+        # First add one
+        add_response = client.post('/api/favorites',
+                                   data=json.dumps({
+                                       'type': 'category',
+                                       'value': 'ETB'
+                                   }),
+                                   content_type='application/json')
+
+        fav_id = json.loads(add_response.data)['id']
+
+        # Now delete it
+        response = client.delete(f'/api/favorites/{fav_id}')
+
+        assert response.status_code == 200
+
+
+if __name__ == "__main__":
+    pytest.main([__file__, "-v"])

tests/test_database.py

@@ -0,0 +1,300 @@
+"""
+Unit tests for database operations.
+"""
+
+import pytest
+import tempfile
+import os
+from datetime import datetime, timedelta
+
+import sys
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from src.database import Database
+
+
+@pytest.fixture
+def db():
+    """Create a temporary database for testing"""
+    with tempfile.NamedTemporaryFile(suffix='.db', delete=False) as f:
+        db_path = f.name
+
+    database = Database(db_path)
+    yield database
+
+    # Cleanup
+    os.unlink(db_path)
+
+
+class TestProductOperations:
+    """Tests for product CRUD operations"""
+
+    def test_create_product(self, db):
+        """Test creating a new product"""
+        product_id = db.get_or_create_product(
+            url="https://example.com/product/123",
+            name="Test Pokemon Card",
+            site="pokemoncenter",
+            product_id="123",
+            price="$19.99",
+            in_stock=True
+        )
+
+        assert product_id is not None
+        assert product_id > 0
+
+    def test_get_product_by_url(self, db):
+        """Test retrieving product by URL"""
+        url = "https://example.com/product/456"
+        db.get_or_create_product(
+            url=url,
+            name="Pikachu ETB",
+            site="target",
+            in_stock=True
+        )
+
+        product = db.get_product_by_url(url)
+
+        assert product is not None
+        assert product['name'] == "Pikachu ETB"
+        assert product['site'] == "target"
+
+    def test_update_product_price(self, db):
+        """Test updating product price"""
+        product_id = db.get_or_create_product(
+            url="https://example.com/product/789",
+            name="Booster Box",
+            site="bestbuy",
+            price="$99.99",
+            in_stock=True
+        )
+
+        # Update price
+        db.update_product_price(product_id, "$89.99")
+
+        product = db.get_product(product_id)
+        assert product['current_price'] == "$89.99"
+
+        # Check price history was recorded
+        history = db.get_price_history(product_id, days=1)
+        assert len(history) >= 1
+
+    def test_update_product_stock(self, db):
+        """Test updating product stock status"""
+        product_id = db.get_or_create_product(
+            url="https://example.com/product/stock-test",
+            name="Limited Edition",
+            site="pokemoncenter",
+            in_stock=True
+        )
+
+        # Mark out of stock
+        db.update_product_stock(product_id, False)
+
+        product = db.get_product(product_id)
+        assert product['in_stock'] == 0  # SQLite stores as 0/1
+
+    def test_category_detection(self, db):
+        """Test automatic category detection from product name"""
+        test_cases = [
+            ("Pokemon Scarlet Elite Trainer Box", "ETB"),
+            ("Charizard Booster Bundle", "Booster Bundle"),
+            ("Display Booster Box 36 Packs", "Booster Box"),
+            ("Sleeved Booster Pack", "Booster Pack"),
+            ("Premium Collection Box", "Collection Box"),
+            ("Pokemon Trading Card Tin", "Tin"),
+            ("Card Binder Album", "Accessories"),
+            ("Random Pokemon Item", "Other"),
+        ]
+
+        for name, expected_category in test_cases:
+            product_id = db.get_or_create_product(
+                url=f"https://example.com/{name.replace(' ', '-')}",
+                name=name,
+                site="test",
+                in_stock=True
+            )
+            product = db.get_product(product_id)
+            assert product['category'] == expected_category, f"Failed for: {name}"
+
+
+class TestStockEvents:
+    """Tests for stock event tracking"""
+
+    def test_record_stock_event(self, db):
+        """Test recording stock events"""
+        product_id = db.get_or_create_product(
+            url="https://example.com/event-test",
+            name="Event Test Product",
+            site="test",
+            in_stock=True
+        )
+
+        db.record_stock_event(product_id, "new_drop")
+        db.record_stock_event(product_id, "out_of_stock")
+        db.record_stock_event(product_id, "restock")
+
+        events = db.get_recent_events(limit=10)
+
+        # Should have 3 events
+        product_events = [e for e in events if e['product_id'] == product_id]
+        assert len(product_events) == 3
+
+    def test_events_today_count(self, db):
+        """Test counting events from today"""
+        product_id = db.get_or_create_product(
+            url="https://example.com/today-test",
+            name="Today Test",
+            site="test",
+            in_stock=True
+        )
+
+        db.record_stock_event(product_id, "new_drop")
+        db.record_stock_event(product_id, "restock")
+
+        new_drops = db.get_events_today("new_drop")
+        restocks = db.get_events_today("restock")
+
+        assert new_drops >= 1
+        assert restocks >= 1
+
+    def test_selling_rate_calculation(self, db):
+        """Test that selling rate can be calculated from events"""
+        product_id = db.get_or_create_product(
+            url="https://example.com/sellrate-test",
+            name="Fast Seller",
+            site="pokemoncenter",
+            in_stock=True
+        )
+
+        # Simulate: drop -> sold out
+        db.record_stock_event(product_id, "new_drop")
+        db.record_stock_event(product_id, "out_of_stock")
+
+        # Get stock duration stats
+        stats = db.get_stock_duration_stats()
+        # Should return at least an empty list (stats depend on timing)
+        assert isinstance(stats, list)
+
+
+class TestPriceHistory:
+    """Tests for price history tracking"""
+
+    def test_record_price_change(self, db):
+        """Test that price changes are recorded"""
+        product_id = db.get_or_create_product(
+            url="https://example.com/price-test",
+            name="Price Test",
+            site="test",
+            price="$49.99",
+            in_stock=True
+        )
+
+        # Record initial price
+        db.record_price(product_id, "$49.99")
+
+        # Change price
+        db.record_price(product_id, "$39.99")
+
+        history = db.get_price_history(product_id, days=1)
+
+        # Should have at least 2 price points
+        assert len(history) >= 1
+
+    def test_duplicate_price_not_recorded(self, db):
+        """Test that same price is not recorded multiple times"""
+        product_id = db.get_or_create_product(
+            url="https://example.com/dup-price-test",
+            name="Dup Price Test",
+            site="test",
+            in_stock=True
+        )
+
+        # Record same price multiple times
+        db.record_price(product_id, "$29.99")
+        db.record_price(product_id, "$29.99")
+        db.record_price(product_id, "$29.99")
+
+        history = db.get_price_history(product_id, days=1)
+
+        # Should only have 1 entry
+        assert len(history) == 1
+
+
+class TestDashboardStats:
+    """Tests for dashboard statistics"""
+
+    def test_get_dashboard_stats(self, db):
+        """Test getting dashboard stats summary"""
+        # Add some test data
+        for i in range(5):
+            product_id = db.get_or_create_product(
+                url=f"https://example.com/stats-{i}",
+                name=f"Stats Product {i}",
+                site="pokemoncenter",
+                in_stock=(i % 2 == 0)
+            )
+            db.record_stock_event(product_id, "new_drop")
+
+        stats = db.get_dashboard_stats()
+
+        assert 'total_products' in stats
+        assert 'in_stock_count' in stats
+        assert 'new_drops_today' in stats
+        assert stats['total_products'] >= 5
+
+    def test_get_site_stats(self, db):
+        """Test per-site statistics"""
+        sites = ['target', 'bestbuy', 'pokemoncenter']
+
+        for site in sites:
+            db.get_or_create_product(
+                url=f"https://{site}.com/test",
+                name=f"{site} Product",
+                site=site,
+                in_stock=True
+            )
+
+        stats = db.get_site_stats()
+
+        assert len(stats) == 3
+        site_names = [s['site'] for s in stats]
+        assert 'target' in site_names
+        assert 'bestbuy' in site_names
+
+
+class TestFavorites:
+    """Tests for favorites functionality"""
+
+    def test_add_favorite(self, db):
+        """Test adding a favorite"""
+        fav_id = db.add_favorite(
+            fav_type="product",
+            value="https://example.com/fav-product",
+            display_name="My Favorite Card",
+            priority="high"
+        )
+
+        assert fav_id is not None
+
+        favorite = db.get_favorite(fav_id)
+        assert favorite['display_name'] == "My Favorite Card"
+
+    def test_check_is_favorite(self, db):
+        """Test checking if product is favorited"""
+        url = "https://example.com/check-fav"
+
+        # Not a favorite yet
+        result = db.check_is_favorite(url=url)
+        assert result is None
+
+        # Add as favorite
+        db.add_favorite(fav_type="product", value=url)
+
+        # Now should be found
+        result = db.check_is_favorite(url=url)
+        assert result is not None
+
+
+if __name__ == "__main__":
+    pytest.main([__file__, "-v"])

tests/test_product_filter.py

@@ -0,0 +1,175 @@
+"""
+Unit tests for Pokemon product validation filter.
+"""
+
+import pytest
+import os
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from scrapers.base import BaseScraper, Product
+
+
+def make_product(name: str, url: str = "", site: str = "test") -> Product:
+    """Helper to create Product with required fields"""
+    return Product(name=name, url=url, price=None, in_stock=True, site=site)
+
+
+class MockScraper(BaseScraper):
+    """Mock scraper for testing base class methods"""
+
+    site_name = "test"
+
+    def scrape_category_page(self, url: str):
+        return []
+
+    def check_product_stock(self, product):
+        return True, "$19.99"
+
+
+@pytest.fixture
+def scraper():
+    return MockScraper()
+
+
+class TestPokemonProductFilter:
+    """Tests for is_pokemon_product validation"""
+
+    def test_valid_pokemon_products(self, scraper):
+        """Test that valid Pokemon products are accepted"""
+        valid_products = [
+            make_product("Pokemon Scarlet & Violet Elite Trainer Box"),
+            make_product("Pikachu V Collection Box"),
+            make_product("TCG Booster Pack Prismatic Evolutions"),
+            make_product("Charizard Premium Collection"),
+            make_product("Pokemon ETB Paldean Fates"),
+            make_product("Pokémon Trading Card Game Tin"),  # Unicode é
+            make_product("Mewtwo VMAX Box Set"),
+            make_product("Eevee Heroes Booster Box"),
+        ]
+
+        for product in valid_products:
+            assert scraper.is_pokemon_product(product), f"Should accept: {product.name}"
+
+    def test_invalid_non_pokemon_products(self, scraper):
+        """Test that non-Pokemon products are rejected"""
+        invalid_products = [
+            make_product("Ice Cube Tray Silicone Mold"),
+            make_product("Barbie Dream House Playset"),
+            make_product("Hot Wheels Track Builder"),
+            make_product("LEGO Star Wars Set"),
+            make_product("Kitchen Appliance Blender"),
+            make_product("Transformers Action Figure"),
+            make_product("Room Essentials Bedding Set"),
+            make_product("Threshold Furniture Table"),
+            make_product("Random Gaming Accessory"),
+        ]
+
+        for product in invalid_products:
+            assert not scraper.is_pokemon_product(product), f"Should reject: {product.name}"
+
+    def test_exclusion_takes_priority(self, scraper):
+        """Test that exclusion terms override Pokemon terms"""
+        # This has "pokemon" but also "ice cube" - should be rejected
+        product = make_product("Pokemon Ice Cube Tray Silicone")
+        assert not scraper.is_pokemon_product(product)
+
+    def test_case_insensitive(self, scraper):
+        """Test that matching is case insensitive"""
+        products = [
+            make_product("POKEMON ELITE TRAINER BOX"),
+            make_product("pokemon scarlet booster"),
+            make_product("PoKeMoN ChArIzArD"),
+        ]
+
+        for product in products:
+            assert scraper.is_pokemon_product(product), f"Should accept: {product.name}"
+
+
+class TestFilterPokemonProducts:
+    """Tests for filter_pokemon_products batch filtering"""
+
+    def test_filter_mixed_products(self, scraper):
+        """Test filtering a mix of valid and invalid products"""
+        products = [
+            make_product("Pokemon Booster Pack", url="1"),
+            make_product("Ice Cube Tray", url="2"),
+            make_product("Charizard Collection", url="3"),
+            make_product("Barbie Doll", url="4"),
+            make_product("Pikachu Tin", url="5"),
+        ]
+
+        filtered = scraper.filter_pokemon_products(products)
+
+        assert len(filtered) == 3
+        urls = [p.url for p in filtered]
+        assert "1" in urls
+        assert "3" in urls
+        assert "5" in urls
+        assert "2" not in urls
+        assert "4" not in urls
+
+    def test_filter_empty_list(self, scraper):
+        """Test filtering empty list returns empty"""
+        filtered = scraper.filter_pokemon_products([])
+        assert filtered == []
+
+    def test_filter_all_valid(self, scraper):
+        """Test filtering when all products are valid"""
+        products = [
+            make_product("Pokemon ETB", url="1"),
+            make_product("Pokemon Booster", url="2"),
+        ]
+
+        filtered = scraper.filter_pokemon_products(products)
+        assert len(filtered) == 2
+
+    def test_filter_all_invalid(self, scraper):
+        """Test filtering when all products are invalid"""
+        products = [
+            make_product("Random Item", url="1"),
+            make_product("Another Thing", url="2"),
+        ]
+
+        filtered = scraper.filter_pokemon_products(products)
+        assert len(filtered) == 0
+
+
+class TestProductDataclass:
+    """Tests for Product dataclass"""
+
+    def test_product_creation(self):
+        """Test creating a Product with all fields"""
+        product = Product(
+            name="Test Product",
+            url="https://example.com/test",
+            price="$29.99",
+            in_stock=True,
+            site="pokemoncenter",
+            image_url="https://example.com/image.jpg",
+            product_id="123"
+        )
+
+        assert product.name == "Test Product"
+        assert product.price == "$29.99"
+        assert product.in_stock is True
+
+    def test_product_minimal(self):
+        """Test Product with only required fields"""
+        product = Product(
+            name="Minimal Product",
+            url="https://example.com",
+            price=None,
+            in_stock=False
+        )
+
+        assert product.name == "Minimal Product"
+        assert product.price is None
+        assert product.in_stock is False
+        assert product.image_url is None
+        assert product.product_id is None
+
+
+if __name__ == "__main__":
+    pytest.main([__file__, "-v"])

tests/test_stealth.py

@@ -0,0 +1,111 @@
+"""
+Test script for the stealth browser module
+Run this to verify undetected-chromedriver is working
+"""
+
+import sys
+import time
+
+print("=" * 60)
+print("Stealth Browser Test")
+print("=" * 60)
+print()
+
+# Check dependencies
+print("1. Checking dependencies...")
+try:
+    import undetected_chromedriver as uc
+    print("   [OK] undetected-chromedriver installed")
+except ImportError:
+    print("   [ERROR] undetected-chromedriver not installed!")
+    print("   Run: pip install undetected-chromedriver")
+    sys.exit(1)
+
+try:
+    from selenium.webdriver.common.action_chains import ActionChains
+    print("   [OK] selenium installed")
+except ImportError:
+    print("   [ERROR] selenium not installed!")
+    print("   Run: pip install selenium")
+    sys.exit(1)
+
+# Import our stealth browser
+print()
+print("2. Importing stealth browser module...")
+try:
+    from stealth_browser import StealthBrowser, get_stealth_browser
+    print("   [OK] stealth_browser module loaded")
+except Exception as e:
+    print(f"   [ERROR] Failed to import: {e}")
+    sys.exit(1)
+
+# Test browser launch
+print()
+print("3. Launching stealth browser...")
+print("   (A Chrome window should open)")
+print()
+
+browser = None
+try:
+    browser = StealthBrowser(
+        headless=False,
+        session_name="test_session"
+    )
+    browser.start()
+    print("   [OK] Browser started successfully!")
+
+    # Test navigation
+    print()
+    print("4. Testing navigation to Pokemon Center...")
+    print("   (Watch the browser window)")
+
+    url = "https://www.pokemoncenter.com/category/tcg-cards"
+    html = browser.get_page(url, wait_time=5)
+
+    print(f"   [OK] Page loaded - {len(html)} bytes")
+
+    # Check for CAPTCHA
+    print()
+    print("5. Checking for bot detection...")
+    if browser.check_for_captcha():
+        print("   [!] CAPTCHA/Challenge detected!")
+        print("   The browser window is open - solve it manually if needed")
+        print("   Waiting up to 60 seconds...")
+        browser.wait_for_captcha_solve(timeout=60)
+    else:
+        print("   [OK] No CAPTCHA detected! Stealth mode working.")
+
+    # Show some page info
+    print()
+    print("6. Page analysis...")
+    from bs4 import BeautifulSoup
+    soup = BeautifulSoup(html, 'html.parser')
+    title = soup.title.string if soup.title else "No title"
+    print(f"   Page title: {title}")
+
+    # Count potential product elements
+    products = soup.select("[data-testid='product-card'], .product-card, a[href*='/product/']")
+    print(f"   Potential product elements: {len(products)}")
+
+    print()
+    print("=" * 60)
+    print("TEST COMPLETE")
+    print("=" * 60)
+    print()
+    print("The browser window will stay open for 10 seconds so you can inspect it.")
+    print("Press Ctrl+C to close early.")
+
+    time.sleep(10)
+
+except KeyboardInterrupt:
+    print("\n   Interrupted by user")
+except Exception as e:
+    print(f"   [ERROR] {e}")
+    import traceback
+    traceback.print_exc()
+finally:
+    if browser:
+        print()
+        print("Closing browser...")
+        browser.stop()
+        print("Done!")

tools/__init__.py

@@ -0,0 +1 @@
+# Utility and experimental scripts

tools/analyze_har.py

@@ -0,0 +1,336 @@
+"""
+HAR File Analyzer for Pokemon Center API Discovery
+
+Analyzes a HAR (HTTP Archive) file exported from Chrome DevTools
+to find API endpoints used by Pokemon Center.
+
+Usage:
+    1. Export HAR from Chrome DevTools Network tab
+    2. Save as pokemon_network.har in this folder
+    3. Run: python analyze_har.py
+
+Or specify a different file:
+    python analyze_har.py my_capture.har
+"""
+
+import json
+import sys
+import re
+from pathlib import Path
+from urllib.parse import urlparse, parse_qs
+from collections import defaultdict
+
+# Patterns that indicate interesting API endpoints
+INTERESTING_PATTERNS = [
+    r'/api/',
+    r'/graphql',
+    r'/v\d+/',
+    r'product',
+    r'catalog',
+    r'search',
+    r'inventory',
+    r'stock',
+    r'\.json$',
+    r'algolia',
+    r'contentful',
+    r'commercetools',
+    r'demandware',
+    r'sfcc',
+]
+
+# Skip these resource types
+SKIP_TYPES = [
+    'image', 'stylesheet', 'font', 'script', 'media',
+    '.png', '.jpg', '.jpeg', '.gif', '.svg', '.ico',
+    '.css', '.woff', '.woff2', '.ttf',
+    '.js',  # Usually not APIs, but we'll catch fetch calls
+]
+
+def should_skip(url, mime_type=''):
+    """Check if we should skip this request"""
+    url_lower = url.lower()
+    mime_lower = mime_type.lower()
+
+    for skip in SKIP_TYPES:
+        if skip in url_lower or skip in mime_lower:
+            return True
+    return False
+
+def is_interesting(url):
+    """Check if URL matches patterns we care about"""
+    url_lower = url.lower()
+    for pattern in INTERESTING_PATTERNS:
+        if re.search(pattern, url_lower):
+            return True
+    return False
+
+def extract_json_preview(content, max_length=500):
+    """Try to extract and preview JSON content"""
+    if not content:
+        return None
+
+    try:
+        # HAR stores content as text or base64
+        text = content.get('text', '')
+        if not text:
+            return None
+
+        # Try to parse as JSON
+        data = json.loads(text)
+
+        # Return a preview
+        preview = json.dumps(data, indent=2)
+        if len(preview) > max_length:
+            return preview[:max_length] + "\n... (truncated)"
+        return preview
+    except:
+        return None
+
+def analyze_har(har_path):
+    """Analyze a HAR file for API endpoints"""
+
+    print(f"\nLoading {har_path}...")
+
+    with open(har_path, 'r', encoding='utf-8') as f:
+        har_data = json.load(f)
+
+    entries = har_data.get('log', {}).get('entries', [])
+    print(f"Found {len(entries)} network requests")
+
+    # Categorize requests
+    api_calls = []
+    json_responses = []
+    graphql_calls = []
+    third_party_apis = []
+
+    domains = defaultdict(int)
+
+    for entry in entries:
+        request = entry.get('request', {})
+        response = entry.get('response', {})
+
+        url = request.get('url', '')
+        method = request.get('method', 'GET')
+        status = response.get('status', 0)
+        mime_type = response.get('content', {}).get('mimeType', '')
+
+        # Track domains
+        parsed = urlparse(url)
+        domains[parsed.netloc] += 1
+
+        # Skip resources we don't care about
+        if should_skip(url, mime_type):
+            continue
+
+        # Check for JSON responses
+        if 'json' in mime_type.lower():
+            content_preview = extract_json_preview(response.get('content', {}))
+            json_responses.append({
+                'url': url,
+                'method': method,
+                'status': status,
+                'preview': content_preview,
+                'headers': {h['name']: h['value'] for h in request.get('headers', [])},
+                'response_headers': {h['name']: h['value'] for h in response.get('headers', [])}
+            })
+
+        # Check for GraphQL
+        if 'graphql' in url.lower():
+            post_data = request.get('postData', {})
+            graphql_calls.append({
+                'url': url,
+                'method': method,
+                'body': post_data.get('text', ''),
+                'status': status
+            })
+
+        # Check for interesting patterns
+        if is_interesting(url):
+            api_calls.append({
+                'url': url,
+                'method': method,
+                'status': status,
+                'mime': mime_type
+            })
+
+        # Check for third-party APIs
+        if any(tp in url.lower() for tp in ['algolia', 'contentful', 'commercetools', 'sfcc']):
+            third_party_apis.append({
+                'url': url,
+                'method': method,
+                'status': status
+            })
+
+    # Print analysis
+    print("\n" + "=" * 70)
+    print("HAR ANALYSIS RESULTS")
+    print("=" * 70)
+
+    # Domains summary
+    print("\n[DOMAINS CONTACTED]")
+    print("-" * 40)
+    for domain, count in sorted(domains.items(), key=lambda x: -x[1])[:15]:
+        print(f"  {count:4d} requests  {domain}")
+
+    # GraphQL calls
+    if graphql_calls:
+        print("\n[GRAPHQL ENDPOINTS]")
+        print("-" * 40)
+        for call in graphql_calls:
+            print(f"\n  [{call['method']}] {call['url']}")
+            print(f"  Status: {call['status']}")
+            if call['body']:
+                try:
+                    body = json.loads(call['body'])
+                    if 'query' in body:
+                        query_preview = body['query'][:200].replace('\n', ' ')
+                        print(f"  Query: {query_preview}...")
+                    if 'operationName' in body:
+                        print(f"  Operation: {body['operationName']}")
+                except:
+                    print(f"  Body: {call['body'][:200]}...")
+
+    # JSON responses (potential APIs)
+    if json_responses:
+        print("\n[JSON RESPONSES - Potential APIs]")
+        print("-" * 40)
+
+        # Filter to most interesting ones
+        product_related = [r for r in json_responses if 'product' in r['url'].lower()]
+        other_json = [r for r in json_responses if 'product' not in r['url'].lower()][:10]
+
+        for resp in product_related + other_json:
+            print(f"\n  [{resp['method']}] {resp['url'][:100]}")
+            print(f"  Status: {resp['status']}")
+
+            if resp['preview']:
+                print(f"  Preview:")
+                for line in resp['preview'].split('\n')[:10]:
+                    print(f"    {line}")
+
+    # Third-party APIs
+    if third_party_apis:
+        print("\n[THIRD-PARTY APIs]")
+        print("-" * 40)
+        for api in third_party_apis:
+            print(f"  [{api['method']}] {api['url'][:100]}")
+
+    # Interesting endpoints
+    if api_calls:
+        print("\n[OTHER INTERESTING ENDPOINTS]")
+        print("-" * 40)
+        seen = set()
+        for call in api_calls:
+            # Dedupe by base URL
+            parsed = urlparse(call['url'])
+            base = f"{parsed.scheme}://{parsed.netloc}{parsed.path}"
+            if base in seen:
+                continue
+            seen.add(base)
+
+            print(f"  [{call['method']}] {call['url'][:100]}")
+            print(f"      Status: {call['status']}, Type: {call['mime']}")
+
+    # Save detailed results
+    output_file = Path(har_path).stem + "_analysis.json"
+    output_path = Path(har_path).parent / output_file
+
+    with open(output_path, 'w') as f:
+        json.dump({
+            'summary': {
+                'total_requests': len(entries),
+                'json_responses': len(json_responses),
+                'graphql_calls': len(graphql_calls),
+                'third_party_apis': len(third_party_apis),
+                'interesting_endpoints': len(api_calls)
+            },
+            'domains': dict(domains),
+            'json_responses': json_responses,
+            'graphql_calls': graphql_calls,
+            'third_party_apis': third_party_apis,
+            'api_calls': api_calls
+        }, f, indent=2)
+
+    print(f"\n[SAVED] Detailed results saved to: {output_path}")
+
+    # Recommendations
+    print("\n" + "=" * 70)
+    print("RECOMMENDATIONS")
+    print("=" * 70)
+
+    if graphql_calls:
+        print("""
+[OK] GraphQL endpoint found! This is likely the main data source.
+   Next steps:
+   1. Examine the query structure in the analysis JSON
+   2. Test if the endpoint works without cookies/auth
+   3. Build a monitor that polls this endpoint
+""")
+
+    if any('algolia' in api['url'].lower() for api in third_party_apis):
+        print("""
+[OK] Algolia search detected! This is often used for product search.
+   Next steps:
+   1. Find the Algolia App ID and Search API Key (usually in page source)
+   2. Query Algolia directly - very fast and low detection risk
+   3. Search for "apiKey" or "applicationId" in the page source
+""")
+
+    if json_responses and not graphql_calls:
+        print("""
+[INFO] JSON responses found but no GraphQL. Check the analysis JSON for:
+   - Responses containing product arrays
+   - URLs with /api/ or /v1/, /v2/ patterns
+   - Look for pagination parameters (page, limit, offset)
+""")
+
+    if not json_responses and not graphql_calls:
+        print("""
+[WARNING] No obvious API endpoints found. Possible reasons:
+   - Server-side rendering (data embedded in HTML)
+   - API calls blocked by bot protection
+   - Need to scroll/interact more to trigger lazy loading
+
+Try:
+   - Scrolling more on the page before exporting HAR
+   - Clicking on product filters
+   - Looking at individual product pages
+""")
+
+def main():
+    # Default HAR file location
+    default_har = Path(__file__).parent / "pokemon_network.har"
+
+    # Check command line args
+    if len(sys.argv) > 1:
+        har_path = Path(sys.argv[1])
+    else:
+        har_path = default_har
+
+    if not har_path.exists():
+        print("=" * 70)
+        print("HAR File Analyzer")
+        print("=" * 70)
+        print(f"""
+No HAR file found at: {har_path}
+
+To capture a HAR file:
+1. Open Chrome and go to: https://www.pokemoncenter.com/category/tcg-cards
+2. Press F12 to open DevTools
+3. Go to the Network tab
+4. Check "Preserve log" checkbox
+5. Refresh the page (F5)
+6. Scroll down to load more products
+7. Right-click in Network panel → "Save all as HAR with content"
+8. Save as: {default_har}
+9. Run this script again
+
+Or specify a HAR file:
+    python analyze_har.py path/to/your/file.har
+""")
+        return
+
+    analyze_har(har_path)
+
+if __name__ == "__main__":
+    main()

tools/api_discovery.py

@@ -0,0 +1,267 @@
+"""
+API Discovery Tool for Pokemon Center
+Attempts to find and test API endpoints that could be used instead of browser scraping.
+"""
+
+import requests
+import json
+from urllib.parse import urljoin
+
+# Common API patterns to try
+BASE_URL = "https://www.pokemoncenter.com"
+
+# Headers to mimic a real browser
+HEADERS = {
+    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
+    "Accept": "application/json, text/plain, */*",
+    "Accept-Language": "en-US,en;q=0.9",
+    "Referer": "https://www.pokemoncenter.com/",
+    "Origin": "https://www.pokemoncenter.com",
+}
+
+# Common API endpoint patterns for e-commerce sites
+API_PATTERNS = [
+    # REST API patterns
+    "/api/products",
+    "/api/v1/products",
+    "/api/v2/products",
+    "/api/catalog/products",
+    "/api/search",
+    "/api/inventory",
+
+    # GraphQL
+    "/graphql",
+    "/api/graphql",
+
+    # Common e-commerce platforms
+    "/rest/V1/products",  # Magento
+    "/_api/products",  # Wix
+    "/cdn/shop/products.json",  # Shopify pattern
+    "/products.json",  # Shopify
+
+    # Search APIs
+    "/api/search/products",
+    "/search/suggest",
+    "/api/autocomplete",
+
+    # Algolia (very common for e-commerce search)
+    # Note: Algolia requires app ID and API key from the page
+]
+
+# Pokemon TCG specific search terms
+SEARCH_TERMS = ["pokemon", "tcg", "cards", "booster", "etb"]
+
+
+def test_endpoint(url: str, method: str = "GET", data: dict = None) -> dict:
+    """Test an API endpoint"""
+    try:
+        if method == "GET":
+            response = requests.get(url, headers=HEADERS, timeout=10)
+        else:
+            response = requests.post(url, headers=HEADERS, json=data, timeout=10)
+
+        return {
+            "url": url,
+            "status": response.status_code,
+            "content_type": response.headers.get("content-type", ""),
+            "size": len(response.content),
+            "sample": response.text[:500] if response.status_code == 200 else None
+        }
+    except Exception as e:
+        return {
+            "url": url,
+            "status": "error",
+            "error": str(e)
+        }
+
+
+def discover_apis():
+    """Attempt to discover API endpoints"""
+    print("=" * 60)
+    print("Pokemon Center API Discovery")
+    print("=" * 60)
+    print()
+
+    results = []
+
+    # Test common patterns
+    print("Testing common API patterns...")
+    for pattern in API_PATTERNS:
+        url = urljoin(BASE_URL, pattern)
+        result = test_endpoint(url)
+        results.append(result)
+
+        if result["status"] == 200:
+            print(f"  [OK] {url}")
+            print(f"       Content-Type: {result['content_type']}")
+            print(f"       Size: {result['size']} bytes")
+        elif result["status"] != "error" and result["status"] < 500:
+            print(f"  [{result['status']}] {url}")
+
+    # Test search with query params
+    print()
+    print("Testing search endpoints...")
+    search_patterns = [
+        "/api/search?q=pokemon",
+        "/api/products?search=tcg",
+        "/api/catalog?category=tcg-cards",
+        "/search?q=pokemon+tcg",
+    ]
+
+    for pattern in search_patterns:
+        url = urljoin(BASE_URL, pattern)
+        result = test_endpoint(url)
+        results.append(result)
+
+        if result["status"] == 200:
+            print(f"  [OK] {url}")
+
+    # Look for Algolia configuration
+    print()
+    print("Checking for Algolia search...")
+    # Algolia is often exposed in page source
+    try:
+        response = requests.get(BASE_URL, headers=HEADERS, timeout=15)
+        if "algolia" in response.text.lower():
+            print("  [!] Algolia detected in page source")
+            # Try to extract app ID and search key
+            import re
+            app_id = re.search(r'["\']?algolia[_-]?app[_-]?id["\']?\s*[:=]\s*["\']([A-Z0-9]+)["\']', response.text, re.I)
+            api_key = re.search(r'["\']?algolia[_-]?(?:search[_-]?)?(?:api[_-]?)?key["\']?\s*[:=]\s*["\']([a-f0-9]+)["\']', response.text, re.I)
+
+            if app_id:
+                print(f"      App ID: {app_id.group(1)}")
+            if api_key:
+                print(f"      Search Key: {api_key.group(1)}")
+
+        # Check for other API clues
+        if "graphql" in response.text.lower():
+            print("  [!] GraphQL detected in page source")
+
+        if "__NEXT_DATA__" in response.text:
+            print("  [!] Next.js detected - may have API routes at /api/*")
+
+        if "window.__INITIAL_STATE__" in response.text or "window.__PRELOADED_STATE__" in response.text:
+            print("  [!] Pre-rendered state detected - data may be in page source")
+
+    except Exception as e:
+        print(f"  Error checking main page: {e}")
+
+    # Summary
+    print()
+    print("=" * 60)
+    print("Summary")
+    print("=" * 60)
+
+    working = [r for r in results if r.get("status") == 200]
+    if working:
+        print(f"Found {len(working)} potentially working endpoints:")
+        for r in working:
+            print(f"  - {r['url']}")
+    else:
+        print("No direct API endpoints found.")
+        print()
+        print("Alternative approaches to consider:")
+        print("  1. Monitor sitemap.xml for new products")
+        print("  2. Use Google Shopping API or similar aggregators")
+        print("  3. Check if they have an RSS feed")
+        print("  4. Use a service like Distill.io for change detection")
+        print("  5. Proxy rotation with residential IPs")
+        print("  6. Lower check frequency + add human-like delays")
+
+    return results
+
+
+def check_sitemap():
+    """Check sitemap for product URLs"""
+    print()
+    print("Checking sitemap...")
+
+    sitemap_urls = [
+        "/sitemap.xml",
+        "/sitemap_index.xml",
+        "/sitemaps/sitemap.xml",
+        "/robots.txt",  # Often contains sitemap location
+    ]
+
+    for pattern in sitemap_urls:
+        url = urljoin(BASE_URL, pattern)
+        result = test_endpoint(url)
+
+        if result["status"] == 200:
+            print(f"  [OK] {url}")
+            if "sitemap" in result.get("sample", "").lower():
+                print(f"       Contains sitemap references")
+            if "product" in result.get("sample", "").lower():
+                print(f"       Contains product references")
+
+
+def check_rss():
+    """Check for RSS feeds"""
+    print()
+    print("Checking for RSS/Atom feeds...")
+
+    feed_urls = [
+        "/feed",
+        "/rss",
+        "/feed.xml",
+        "/rss.xml",
+        "/atom.xml",
+        "/blog/feed",
+        "/news/feed",
+    ]
+
+    for pattern in feed_urls:
+        url = urljoin(BASE_URL, pattern)
+        result = test_endpoint(url)
+
+        if result["status"] == 200 and ("xml" in result.get("content_type", "") or "rss" in result.get("content_type", "")):
+            print(f"  [OK] {url}")
+
+
+if __name__ == "__main__":
+    discover_apis()
+    check_sitemap()
+    check_rss()
+
+    print()
+    print("=" * 60)
+    print("Next Steps")
+    print("=" * 60)
+    print("""
+To avoid bot detection, consider these strategies:
+
+1. API-BASED MONITORING (if endpoints found):
+   - Call API endpoints directly with requests
+   - Much faster and less detectable than browser
+   - Can check more frequently
+
+2. SITEMAP MONITORING:
+   - Parse sitemap.xml periodically
+   - Detect new product URLs without visiting pages
+   - Very low detection risk
+
+3. HASH-BASED CHANGE DETECTION:
+   - Fetch page, hash content
+   - Only alert when hash changes
+   - Reduces unnecessary processing
+
+4. RESIDENTIAL PROXY ROTATION:
+   - Use services like Bright Data, Oxylabs
+   - Rotate IPs to avoid blocks
+   - More expensive but reliable
+
+5. HUMAN-LIKE BEHAVIOR:
+   - Random delays between 60-180 seconds
+   - Vary user agent strings
+   - Add mouse movements and scrolling
+   - Use real browser cookies
+
+6. THIRD-PARTY ALERTS:
+   - Discord servers that track Pokemon Center
+   - Stock alert services (NowInStock, etc.)
+   - Browser extensions like Distill.io
+
+Run this script to see what APIs are available:
+  python api_discovery.py
+""")

tools/api_monitor.py

@@ -0,0 +1,239 @@
+"""
+Pokemon Center API Monitor
+
+Uses cookies from a browser session to make API calls directly.
+Much lighter than full browser scraping once cookies are established.
+
+Workflow:
+1. Run warmup_session.py first to get valid cookies
+2. This script uses those cookies to call APIs directly
+3. Falls back to browser if cookies expire
+"""
+
+import json
+import time
+import pickle
+import logging
+import requests
+from pathlib import Path
+from datetime import datetime
+from typing import Optional, Dict, List, Any
+
+logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
+logger = logging.getLogger(__name__)
+
+# Session/cookie storage
+SESSION_DIR = Path(__file__).parent / "sessions"
+COOKIE_FILE = SESSION_DIR / "pokemoncenter_cookies.pkl"
+
+# API endpoints discovered from HAR analysis
+API_BASE = "https://www.pokemoncenter.com"
+ENDPOINTS = {
+    "product": "/tpci-ecommweb-api/product/{sku}",
+    "status": "/tpci-ecommweb-api/product/status/{encoded_id}",
+    "category": "/site/resourceapi/category/{category}",
+    "reviews": "/tpci-ecommweb-api/review/get-product-scores",
+}
+
+# Required headers from HAR capture
+BASE_HEADERS = {
+    "Accept": "application/json",
+    "Accept-Version": "1",
+    "Content-Type": "application/json",
+    "X-Store-Locale": "en-us",
+    "X-Store-Scope": "pokemon",
+    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36",
+    "Referer": "https://www.pokemoncenter.com/category/tcg-cards",
+    "Sec-Fetch-Dest": "empty",
+    "Sec-Fetch-Mode": "cors",
+    "Sec-Fetch-Site": "same-origin",
+}
+
+# Known product SKUs for TCG (build this list over time)
+KNOWN_TCG_SKUS_FILE = Path(__file__).parent / "known_tcg_skus.json"
+
+
+class PokemonCenterAPI:
+    """Direct API client for Pokemon Center using session cookies"""
+
+    def __init__(self):
+        self.session = requests.Session()
+        self.session.headers.update(BASE_HEADERS)
+        self.cookies_loaded = False
+        self.last_cookie_refresh = None
+
+    def load_cookies(self) -> bool:
+        """Load cookies from the saved session file"""
+        if not COOKIE_FILE.exists():
+            logger.warning(f"No cookie file found at {COOKIE_FILE}")
+            logger.info("Run warmup_session.py first to create a valid session")
+            return False
+
+        try:
+            with open(COOKIE_FILE, 'rb') as f:
+                cookies = pickle.load(f)
+
+            # Add cookies to session
+            for cookie in cookies:
+                self.session.cookies.set(
+                    cookie['name'],
+                    cookie['value'],
+                    domain=cookie.get('domain', '.pokemoncenter.com'),
+                    path=cookie.get('path', '/')
+                )
+
+            logger.info(f"Loaded {len(cookies)} cookies from session file")
+            self.cookies_loaded = True
+            self.last_cookie_refresh = datetime.now()
+            return True
+
+        except Exception as e:
+            logger.error(f"Failed to load cookies: {e}")
+            return False
+
+    def _make_request(self, method: str, url: str, **kwargs) -> Optional[requests.Response]:
+        """Make a request with error handling"""
+        try:
+            if method.upper() == "GET":
+                response = self.session.get(url, timeout=15, **kwargs)
+            else:
+                response = self.session.post(url, timeout=15, **kwargs)
+
+            # Check for blocking
+            if response.status_code == 403:
+                if "captcha" in response.text.lower() or "blocked" in response.text.lower():
+                    logger.warning("Request blocked - cookies may have expired")
+                    self.cookies_loaded = False
+                    return None
+
+            return response
+
+        except requests.RequestException as e:
+            logger.error(f"Request failed: {e}")
+            return None
+
+    def get_product(self, sku: str) -> Optional[Dict]:
+        """Get product details by SKU"""
+        if not self.cookies_loaded:
+            if not self.load_cookies():
+                return None
+
+        url = f"{API_BASE}{ENDPOINTS['product'].format(sku=sku)}"
+        response = self._make_request("GET", url)
+
+        if response and response.status_code == 200:
+            return response.json()
+        return None
+
+    def get_product_status(self, encoded_id: str) -> Optional[Dict]:
+        """Get product availability status"""
+        if not self.cookies_loaded:
+            if not self.load_cookies():
+                return None
+
+        url = f"{API_BASE}{ENDPOINTS['status'].format(encoded_id=encoded_id)}"
+        response = self._make_request("GET", url)
+
+        if response and response.status_code == 200:
+            return response.json()
+        return None
+
+    def get_category(self, category: str = "new-releases") -> Optional[Dict]:
+        """Get category listing (potential goldmine for new drops!)"""
+        if not self.cookies_loaded:
+            if not self.load_cookies():
+                return None
+
+        url = f"{API_BASE}{ENDPOINTS['category'].format(category=category)}"
+        response = self._make_request("GET", url)
+
+        if response and response.status_code == 200:
+            return response.json()
+        return None
+
+    def get_review_scores(self, sku_list: List[str]) -> Optional[Dict]:
+        """Get review scores for multiple SKUs"""
+        if not self.cookies_loaded:
+            if not self.load_cookies():
+                return None
+
+        skus = ",".join(sku_list)
+        url = f"{API_BASE}{ENDPOINTS['reviews']}?skuList={skus}"
+        response = self._make_request("GET", url)
+
+        if response and response.status_code == 200:
+            return response.json()
+        return None
+
+
+def load_known_skus() -> List[str]:
+    """Load list of known TCG SKUs"""
+    if KNOWN_TCG_SKUS_FILE.exists():
+        with open(KNOWN_TCG_SKUS_FILE, 'r') as f:
+            return json.load(f)
+    return []
+
+
+def save_known_skus(skus: List[str]):
+    """Save list of known TCG SKUs"""
+    with open(KNOWN_TCG_SKUS_FILE, 'w') as f:
+        json.dump(skus, f, indent=2)
+
+
+def test_api():
+    """Test the API client"""
+    print("=" * 70)
+    print("Pokemon Center API Monitor Test")
+    print("=" * 70)
+    print()
+
+    api = PokemonCenterAPI()
+
+    if not api.load_cookies():
+        print("\nNo valid cookies found!")
+        print("Please run: python warmup_session.py")
+        print("Then try again.")
+        return
+
+    print("\n[1] Testing category endpoint (new-releases)...")
+    category_data = api.get_category("new-releases")
+    if category_data:
+        print("    SUCCESS! Category data retrieved.")
+        print(f"    Keys: {list(category_data.keys())[:5]}")
+
+        # Save for analysis
+        with open("category_response.json", "w") as f:
+            json.dump(category_data, f, indent=2)
+        print("    Saved to category_response.json")
+    else:
+        print("    FAILED - cookies may have expired")
+
+    print("\n[2] Testing product endpoint...")
+    # Try a known SKU from our HAR capture
+    product_data = api.get_product("699-17113")
+    if product_data:
+        print("    SUCCESS! Product data retrieved.")
+        print(f"    Keys: {list(product_data.keys())[:5]}")
+    else:
+        print("    FAILED - cookies may have expired")
+
+    print("\n[3] Testing review scores endpoint...")
+    reviews = api.get_review_scores(["699-17113", "191-85953"])
+    if reviews:
+        print("    SUCCESS! Review scores retrieved.")
+        print(f"    Data: {reviews}")
+    else:
+        print("    FAILED - cookies may have expired")
+
+    print()
+    print("=" * 70)
+    if category_data or product_data:
+        print("API access working! Can monitor without full browser scraping.")
+        print("Cookies will expire eventually - re-run warmup when needed.")
+    else:
+        print("API blocked. Need fresh cookies from browser session.")
+    print("=" * 70)
+
+
+if __name__ == "__main__":
+    test_api()

tools/backend_monitor.py

@@ -0,0 +1,370 @@
+"""
+Pokemon Center Backend Monitor
+
+Detects NEW products added to the API before they're publicly announced.
+This is how accounts like @pokepullzhq detect drops early.
+
+Strategy:
+1. Maintain a list of all known product SKUs
+2. Periodically check the API for current products
+3. Compare: Any new SKUs = potential silent drop
+4. Alert immediately on new detections
+
+Usage:
+1. First run warmup_session.py to get valid cookies
+2. Then run: python backend_monitor.py
+"""
+
+import json
+import time
+import pickle
+import logging
+import requests
+from pathlib import Path
+from datetime import datetime
+from typing import Optional, Dict, List, Set
+
+# Setup logging
+logging.basicConfig(
+    level=logging.INFO,
+    format='%(asctime)s - %(levelname)s - %(message)s'
+)
+logger = logging.getLogger(__name__)
+
+# File paths
+DATA_DIR = Path(__file__).parent.parent / "data"
+SESSION_DIR = DATA_DIR / "sessions"
+COOKIE_FILE = SESSION_DIR / "pokemoncenter_cookies.pkl"
+KNOWN_SKUS_FILE = DATA_DIR / "known_skus.json"
+DETECTIONS_LOG = DATA_DIR / "detections.json"
+
+# API configuration
+API_BASE = "https://www.pokemoncenter.com"
+
+# Categories to monitor for new products
+MONITOR_CATEGORIES = [
+    "new-releases",
+    "tcg-cards",
+    # Add more as needed
+]
+
+# Headers required for API calls
+HEADERS = {
+    "Accept": "application/json",
+    "Accept-Version": "1",
+    "Content-Type": "application/json",
+    "X-Store-Locale": "en-us",
+    "X-Store-Scope": "pokemon",
+    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36",
+    "Referer": "https://www.pokemoncenter.com/",
+}
+
+# Check interval (seconds)
+CHECK_INTERVAL = 60
+
+
+class BackendMonitor:
+    """Monitors Pokemon Center API for new product drops"""
+
+    def __init__(self):
+        self.session = requests.Session()
+        self.session.headers.update(HEADERS)
+        self.known_skus: Set[str] = set()
+        self.cookies_valid = False
+        self.detections: List[Dict] = []
+
+    def load_cookies(self) -> bool:
+        """Load cookies from warmup session"""
+        if not COOKIE_FILE.exists():
+            logger.error(f"No cookie file found at {COOKIE_FILE}")
+            logger.info("Run warmup_session.py first!")
+            return False
+
+        try:
+            with open(COOKIE_FILE, 'rb') as f:
+                cookies = pickle.load(f)
+
+            for cookie in cookies:
+                self.session.cookies.set(
+                    cookie['name'],
+                    cookie['value'],
+                    domain=cookie.get('domain', '.pokemoncenter.com'),
+                    path=cookie.get('path', '/')
+                )
+
+            logger.info(f"Loaded {len(cookies)} cookies")
+            self.cookies_valid = True
+            return True
+
+        except Exception as e:
+            logger.error(f"Failed to load cookies: {e}")
+            return False
+
+    def load_known_skus(self):
+        """Load previously seen SKUs"""
+        if KNOWN_SKUS_FILE.exists():
+            with open(KNOWN_SKUS_FILE, 'r') as f:
+                data = json.load(f)
+                self.known_skus = set(data.get('skus', []))
+            logger.info(f"Loaded {len(self.known_skus)} known SKUs")
+        else:
+            logger.info("No known SKUs file - will create on first run")
+            self.known_skus = set()
+
+    def save_known_skus(self):
+        """Save known SKUs to file"""
+        with open(KNOWN_SKUS_FILE, 'w') as f:
+            json.dump({
+                'skus': list(self.known_skus),
+                'last_updated': datetime.now().isoformat(),
+                'count': len(self.known_skus)
+            }, f, indent=2)
+
+    def log_detection(self, sku: str, product_info: Dict):
+        """Log a new product detection"""
+        detection = {
+            'sku': sku,
+            'detected_at': datetime.now().isoformat(),
+            'product_info': product_info
+        }
+        self.detections.append(detection)
+
+        # Append to detections log file
+        detections = []
+        if DETECTIONS_LOG.exists():
+            with open(DETECTIONS_LOG, 'r') as f:
+                detections = json.load(f)
+
+        detections.append(detection)
+
+        with open(DETECTIONS_LOG, 'w') as f:
+            json.dump(detections, f, indent=2)
+
+    def get_category_products(self, category: str) -> Optional[Dict]:
+        """Fetch products from a category endpoint"""
+        url = f"{API_BASE}/site/resourceapi/category/{category}"
+
+        try:
+            response = self.session.get(url, timeout=15)
+
+            if response.status_code == 403:
+                logger.warning("API blocked - cookies may have expired")
+                self.cookies_valid = False
+                return None
+
+            if response.status_code == 200:
+                return response.json()
+
+            logger.warning(f"Unexpected status {response.status_code} for {category}")
+            return None
+
+        except Exception as e:
+            logger.error(f"Error fetching {category}: {e}")
+            return None
+
+    def get_product_details(self, sku: str) -> Optional[Dict]:
+        """Get full details for a specific product"""
+        url = f"{API_BASE}/tpci-ecommweb-api/product/{sku}"
+
+        try:
+            response = self.session.get(url, timeout=15)
+
+            if response.status_code == 200:
+                return response.json()
+
+            return None
+
+        except Exception as e:
+            logger.error(f"Error fetching product {sku}: {e}")
+            return None
+
+    def extract_skus_from_response(self, data: Dict) -> Set[str]:
+        """Extract product SKUs from API response"""
+        skus = set()
+
+        # The response structure varies - try multiple approaches
+        # This will need adjustment based on actual API response
+
+        def find_skus(obj, depth=0):
+            """Recursively find SKU-like values"""
+            if depth > 10:  # Prevent infinite recursion
+                return
+
+            if isinstance(obj, dict):
+                # Look for SKU fields
+                for key in ['sku', 'skuCode', 'productId', 'id', 'code']:
+                    if key in obj:
+                        value = obj[key]
+                        if isinstance(value, str) and self._looks_like_sku(value):
+                            skus.add(value)
+
+                # Look in nested objects
+                for value in obj.values():
+                    find_skus(value, depth + 1)
+
+            elif isinstance(obj, list):
+                for item in obj:
+                    find_skus(item, depth + 1)
+
+        find_skus(data)
+        return skus
+
+    def _looks_like_sku(self, value: str) -> bool:
+        """Check if a string looks like a Pokemon Center SKU"""
+        # SKUs we've seen: 699-17113, 191-85953, 10-10191-109
+        if not value:
+            return False
+
+        # Must contain digits and possibly hyphens
+        has_digit = any(c.isdigit() for c in value)
+        reasonable_length = 5 <= len(value) <= 20
+
+        return has_digit and reasonable_length
+
+    def check_for_new_products(self) -> List[Dict]:
+        """Main check - look for new SKUs across all categories"""
+        if not self.cookies_valid:
+            if not self.load_cookies():
+                return []
+
+        new_products = []
+        current_skus = set()
+
+        for category in MONITOR_CATEGORIES:
+            logger.debug(f"Checking category: {category}")
+
+            data = self.get_category_products(category)
+            if data:
+                skus = self.extract_skus_from_response(data)
+                current_skus.update(skus)
+                logger.debug(f"  Found {len(skus)} SKUs in {category}")
+
+        if not current_skus:
+            logger.warning("No SKUs found - API may not be returning data")
+            return []
+
+        # Find new SKUs
+        new_skus = current_skus - self.known_skus
+
+        if new_skus:
+            logger.info(f"!!! DETECTED {len(new_skus)} NEW SKU(s) !!!")
+
+            for sku in new_skus:
+                # Get full product details
+                details = self.get_product_details(sku)
+
+                product_info = {
+                    'sku': sku,
+                    'details': details,
+                    'detected_at': datetime.now().isoformat()
+                }
+
+                # Try to extract name from details
+                name = "Unknown Product"
+                if details:
+                    # Look for name in various places
+                    name = (
+                        details.get('name') or
+                        details.get('displayName') or
+                        details.get('definition', {}).get('display-name') or
+                        sku
+                    )
+
+                logger.info(f"  NEW: {sku} - {name}")
+                self.log_detection(sku, product_info)
+                new_products.append(product_info)
+
+                # Add to known SKUs
+                self.known_skus.add(sku)
+
+            # Save updated known SKUs
+            self.save_known_skus()
+
+        else:
+            logger.info(f"Check complete - {len(current_skus)} products, no new drops")
+
+        return new_products
+
+    def send_discord_alert(self, product: Dict):
+        """Send Discord notification for new product"""
+        # Import from your existing discord_notifier
+        try:
+            from src.discord_notifier import send_notification
+            # You'd format and send the alert here
+            logger.info(f"Discord alert sent for {product['sku']}")
+        except ImportError:
+            logger.warning("Discord notifier not available")
+
+    def run(self):
+        """Main monitoring loop"""
+        print("=" * 60)
+        print("Pokemon Center Backend Monitor")
+        print("=" * 60)
+        print()
+        print("This monitors for NEW products added to the API.")
+        print("New SKUs = potential silent drops before announcement!")
+        print()
+        print(f"Check interval: {CHECK_INTERVAL} seconds")
+        print(f"Monitoring categories: {', '.join(MONITOR_CATEGORIES)}")
+        print()
+        print("Press Ctrl+C to stop")
+        print("=" * 60)
+        print()
+
+        # Load known SKUs
+        self.load_known_skus()
+
+        # Load cookies
+        if not self.load_cookies():
+            print("\nERROR: No valid cookies!")
+            print("Run: python warmup_session.py")
+            return
+
+        # First check - populate known SKUs if empty
+        if not self.known_skus:
+            logger.info("First run - building initial SKU database...")
+            self.check_for_new_products()
+            logger.info(f"Baseline established with {len(self.known_skus)} products")
+            print()
+
+        # Main loop
+        check_count = 0
+        while True:
+            try:
+                check_count += 1
+                logger.info(f"--- Check #{check_count} ---")
+
+                new_products = self.check_for_new_products()
+
+                if new_products:
+                    print()
+                    print("!" * 60)
+                    print("!!! NEW PRODUCT DETECTED !!!")
+                    print("!" * 60)
+                    for p in new_products:
+                        print(f"  SKU: {p['sku']}")
+                        # Send Discord alert
+                        self.send_discord_alert(p)
+                    print("!" * 60)
+                    print()
+
+                # Wait for next check
+                time.sleep(CHECK_INTERVAL)
+
+            except KeyboardInterrupt:
+                print("\nStopping monitor...")
+                self.save_known_skus()
+                break
+
+            except Exception as e:
+                logger.error(f"Error in main loop: {e}")
+                time.sleep(CHECK_INTERVAL)
+
+
+def main():
+    monitor = BackendMonitor()
+    monitor.run()
+
+
+if __name__ == "__main__":
+    main()

tools/capture_api_calls.py

@@ -0,0 +1,242 @@
+"""
+API Call Capture Tool for Pokemon Center
+
+This script opens Pokemon Center and logs ALL network requests,
+helping identify backend APIs used for product data.
+
+Usage:
+    python capture_api_calls.py
+
+Output:
+    - api_captures.json: All captured API calls
+    - Console output with interesting endpoints
+"""
+
+import json
+import time
+import re
+from datetime import datetime
+from pathlib import Path
+
+print("=" * 70)
+print("Pokemon Center API Capture Tool")
+print("=" * 70)
+print()
+
+# Check for selenium
+try:
+    from selenium import webdriver
+    from selenium.webdriver.chrome.options import Options
+    from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
+except ImportError:
+    print("ERROR: selenium not installed")
+    print("Run: pip install selenium")
+    exit(1)
+
+# Storage for captured requests
+captured_requests = []
+interesting_patterns = [
+    r'api',
+    r'graphql',
+    r'product',
+    r'catalog',
+    r'search',
+    r'inventory',
+    r'stock',
+    r'algolia',
+    r'contentful',
+    r'commercetools',
+    r'\.json',
+]
+
+def is_interesting(url):
+    """Check if URL matches patterns we care about"""
+    url_lower = url.lower()
+    for pattern in interesting_patterns:
+        if re.search(pattern, url_lower):
+            return True
+    return False
+
+def setup_browser():
+    """Setup Chrome with network logging enabled"""
+    options = Options()
+    options.add_argument("--disable-blink-features=AutomationControlled")
+    options.add_argument("--window-size=1920,1080")
+
+    # Enable performance logging to capture network requests
+    options.set_capability('goog:loggingPrefs', {'performance': 'ALL'})
+
+    driver = webdriver.Chrome(options=options)
+    return driver
+
+def extract_network_requests(driver):
+    """Extract network requests from Chrome performance logs"""
+    logs = driver.get_log('performance')
+    requests = []
+
+    for entry in logs:
+        try:
+            log = json.loads(entry['message'])['message']
+
+            # We want Network.requestWillBeSent and Network.responseReceived
+            if log['method'] == 'Network.requestWillBeSent':
+                request = log['params']['request']
+                requests.append({
+                    'type': 'request',
+                    'url': request.get('url', ''),
+                    'method': request.get('method', ''),
+                    'headers': request.get('headers', {}),
+                    'postData': request.get('postData', None),
+                    'timestamp': entry['timestamp']
+                })
+
+            elif log['method'] == 'Network.responseReceived':
+                response = log['params']['response']
+                requests.append({
+                    'type': 'response',
+                    'url': response.get('url', ''),
+                    'status': response.get('status', 0),
+                    'mimeType': response.get('mimeType', ''),
+                    'headers': response.get('headers', {}),
+                    'timestamp': entry['timestamp']
+                })
+
+        except Exception:
+            pass
+
+    return requests
+
+def analyze_requests(requests):
+    """Analyze captured requests for interesting patterns"""
+    api_calls = []
+    seen_urls = set()
+
+    for req in requests:
+        url = req.get('url', '')
+
+        # Skip if already seen or not interesting
+        if url in seen_urls:
+            continue
+        if not is_interesting(url):
+            continue
+
+        # Skip common non-API resources
+        if any(ext in url for ext in ['.png', '.jpg', '.gif', '.css', '.woff', '.svg', '.ico']):
+            continue
+
+        seen_urls.add(url)
+        api_calls.append(req)
+
+    return api_calls
+
+def main():
+    driver = None
+
+    try:
+        print("Starting Chrome with network logging...")
+        driver = setup_browser()
+
+        print("Navigating to Pokemon Center TCG page...")
+        print("(This may trigger a CAPTCHA - solve it if needed)")
+        print()
+
+        # Navigate to the TCG category
+        driver.get("https://www.pokemoncenter.com/category/tcg-cards")
+
+        print("Waiting for page to load...")
+        time.sleep(10)
+
+        # Scroll down to trigger lazy loading
+        print("Scrolling to load more content...")
+        for i in range(3):
+            driver.execute_script("window.scrollBy(0, 800);")
+            time.sleep(2)
+
+        # Wait a bit more
+        time.sleep(5)
+
+        print()
+        print("Extracting network requests...")
+        requests = extract_network_requests(driver)
+        print(f"Captured {len(requests)} total network events")
+
+        # Analyze for interesting APIs
+        api_calls = analyze_requests(requests)
+        print(f"Found {len(api_calls)} potentially interesting API calls")
+
+        # Save all captures
+        output_file = Path(__file__).parent / "api_captures.json"
+        with open(output_file, 'w') as f:
+            json.dump({
+                'captured_at': datetime.now().isoformat(),
+                'page_url': driver.current_url,
+                'total_requests': len(requests),
+                'interesting_calls': api_calls,
+                'all_requests': requests
+            }, f, indent=2)
+
+        print(f"\nSaved full capture to: {output_file}")
+
+        # Display interesting findings
+        print()
+        print("=" * 70)
+        print("INTERESTING API CALLS FOUND")
+        print("=" * 70)
+
+        if not api_calls:
+            print("No obvious API calls detected.")
+            print("This could mean:")
+            print("  1. Data is server-rendered (no client API)")
+            print("  2. API calls use non-standard paths")
+            print("  3. Bot protection blocked the content")
+        else:
+            for call in api_calls:
+                url = call.get('url', '')
+                method = call.get('method', 'GET')
+                status = call.get('status', '-')
+                mime = call.get('mimeType', '')
+
+                # Truncate long URLs
+                display_url = url[:100] + '...' if len(url) > 100 else url
+
+                print(f"\n[{method}] {display_url}")
+                if status != '-':
+                    print(f"    Status: {status}, Type: {mime}")
+
+                # Check for key patterns
+                if 'graphql' in url.lower():
+                    print("    ⚡ GraphQL endpoint!")
+                if 'algolia' in url.lower():
+                    print("    🔍 Algolia search!")
+                if 'product' in url.lower():
+                    print("    📦 Product data!")
+
+        print()
+        print("=" * 70)
+        print("NEXT STEPS")
+        print("=" * 70)
+        print("""
+1. Review api_captures.json for the full data
+2. Look for JSON responses containing product info
+3. Test promising endpoints with curl/requests
+4. Note any required headers (auth tokens, API keys)
+
+To test an endpoint manually:
+    curl -H "User-Agent: Mozilla/5.0..." "https://api.example.com/endpoint"
+""")
+
+        # Keep browser open for manual inspection
+        print()
+        input("Browser is still open. Press ENTER to close and exit...")
+
+    except Exception as e:
+        print(f"Error: {e}")
+        import traceback
+        traceback.print_exc()
+    finally:
+        if driver:
+            driver.quit()
+            print("Browser closed.")
+
+if __name__ == "__main__":
+    main()

tools/smart_monitor.py

@@ -0,0 +1,415 @@
+"""
+Smart Pokemon Center Monitor
+
+Uses stealth browser with adaptive modes and human-like behavior.
+Detects new products before they're announced.
+
+Modes:
+- STEALTH: Normal monitoring (~1 min intervals, human-like)
+- ALERT: Fast checking when new product detected (~20 sec)
+- COOLDOWN: Gradual return to stealth after alert
+
+Usage:
+    python smart_monitor.py
+"""
+
+import json
+import time
+import random
+import logging
+from pathlib import Path
+from datetime import datetime, timedelta
+from typing import Optional, Set, List, Dict
+from enum import Enum
+
+# Setup logging
+logging.basicConfig(
+    level=logging.INFO,
+    format='%(asctime)s [%(levelname)s] %(message)s',
+    datefmt='%H:%M:%S'
+)
+logger = logging.getLogger(__name__)
+
+# File paths
+DATA_DIR = Path(__file__).parent.parent / "data"
+KNOWN_SKUS_FILE = DATA_DIR / "known_skus.json"
+DETECTIONS_FILE = DATA_DIR / "detections.json"
+
+# URLs to monitor
+MONITOR_URLS = [
+    "https://www.pokemoncenter.com/category/tcg-cards?sort=newest",
+    "https://www.pokemoncenter.com/category/new-releases",
+]
+
+
+class MonitorMode(Enum):
+    STEALTH = "stealth"
+    ALERT = "alert"
+    COOLDOWN = "cooldown"
+
+
+class SmartMonitor:
+    """Adaptive monitor with human-like behavior"""
+
+    def __init__(self):
+        self.browser = None
+        self.known_skus: Set[str] = set()
+        self.mode = MonitorMode.STEALTH
+        self.alert_triggered_at: Optional[datetime] = None
+        self.check_count = 0
+        self.last_detection: Optional[Dict] = None
+
+        # Mode timing settings
+        self.timing = {
+            MonitorMode.STEALTH: (45, 90),      # 45-90 seconds
+            MonitorMode.ALERT: (15, 30),         # 15-30 seconds
+            MonitorMode.COOLDOWN: (60, 120),     # 60-120 seconds
+        }
+
+        # Alert mode duration
+        self.alert_duration = timedelta(minutes=30)
+        self.cooldown_duration = timedelta(minutes=15)
+
+    def start_browser(self):
+        """Start the stealth browser"""
+        if self.browser:
+            return
+
+        logger.info("Starting stealth browser...")
+
+        try:
+            from stealth_browser import StealthBrowser
+            self.browser = StealthBrowser(
+                headless=False,
+                session_name="smart_monitor"
+            )
+            self.browser.start()
+            logger.info("Browser started successfully")
+        except Exception as e:
+            logger.error(f"Failed to start browser: {e}")
+            raise
+
+    def stop_browser(self):
+        """Stop the browser"""
+        if self.browser:
+            logger.info("Stopping browser...")
+            self.browser.stop()
+            self.browser = None
+
+    def load_known_skus(self):
+        """Load previously seen SKUs"""
+        if KNOWN_SKUS_FILE.exists():
+            with open(KNOWN_SKUS_FILE, 'r') as f:
+                data = json.load(f)
+                self.known_skus = set(data.get('skus', []))
+            logger.info(f"Loaded {len(self.known_skus)} known SKUs")
+        else:
+            self.known_skus = set()
+            logger.info("No known SKUs file - starting fresh")
+
+    def save_known_skus(self):
+        """Save known SKUs"""
+        with open(KNOWN_SKUS_FILE, 'w') as f:
+            json.dump({
+                'skus': list(self.known_skus),
+                'updated': datetime.now().isoformat(),
+                'count': len(self.known_skus)
+            }, f, indent=2)
+
+    def log_detection(self, sku: str, name: str, url: str):
+        """Log a new product detection"""
+        detection = {
+            'sku': sku,
+            'name': name,
+            'url': url,
+            'detected_at': datetime.now().isoformat(),
+            'mode': self.mode.value
+        }
+
+        self.last_detection = detection
+
+        # Load existing detections
+        detections = []
+        if DETECTIONS_FILE.exists():
+            with open(DETECTIONS_FILE, 'r') as f:
+                detections = json.load(f)
+
+        detections.append(detection)
+
+        with open(DETECTIONS_FILE, 'w') as f:
+            json.dump(detections, f, indent=2)
+
+        logger.info(f"Detection logged: {sku}")
+
+    def get_interval(self) -> float:
+        """Get randomized check interval based on current mode"""
+        min_sec, max_sec = self.timing[self.mode]
+
+        # Add gaussian jitter for more natural timing
+        base = (min_sec + max_sec) / 2
+        jitter = random.gauss(0, (max_sec - min_sec) / 4)
+        interval = base + jitter
+
+        # Clamp to bounds
+        return max(min_sec * 0.8, min(max_sec * 1.2, interval))
+
+    def maybe_do_human_action(self):
+        """Occasionally do something human-like"""
+        if not self.browser or not self.browser.driver:
+            return
+
+        action = random.random()
+
+        if action < 0.3:
+            # Scroll randomly
+            self.browser.human_scroll()
+        elif action < 0.4:
+            # Small mouse movement
+            self.browser.human_mouse_move()
+        elif action < 0.45:
+            # Longer pause (human distraction)
+            pause = random.uniform(3, 8)
+            logger.debug(f"Human pause: {pause:.1f}s")
+            time.sleep(pause)
+
+    def maybe_take_break(self) -> bool:
+        """Occasionally take a longer break"""
+        # 3% chance of a break
+        if random.random() < 0.03:
+            break_time = random.randint(120, 300)  # 2-5 minutes
+            logger.info(f"Taking a break for {break_time}s (human-like pause)")
+            time.sleep(break_time)
+            return True
+        return False
+
+    def extract_skus_from_page(self) -> Set[str]:
+        """Extract product SKUs from the current page"""
+        if not self.browser or not self.browser.driver:
+            return set()
+
+        skus = set()
+
+        try:
+            # Get page source and parse
+            from bs4 import BeautifulSoup
+            html = self.browser.driver.page_source
+            soup = BeautifulSoup(html, 'html.parser')
+
+            # Look for product links - Pokemon Center format: /product/SKU/name
+            import re
+            product_links = soup.select('a[href*="/product/"]')
+
+            for link in product_links:
+                href = link.get('href', '')
+                # Extract SKU from URL like /product/699-17113/product-name
+                match = re.search(r'/product/([0-9]+-?[0-9]+)', href)
+                if match:
+                    skus.add(match.group(1))
+
+            # Also look for data attributes
+            for elem in soup.select('[data-sku], [data-product-id]'):
+                sku = elem.get('data-sku') or elem.get('data-product-id')
+                if sku and re.match(r'^[0-9]+-?[0-9]+', sku):
+                    skus.add(sku)
+
+        except Exception as e:
+            logger.error(f"Error extracting SKUs: {e}")
+
+        return skus
+
+    def check_page(self, url: str) -> Set[str]:
+        """Load a page and extract SKUs"""
+        if not self.browser:
+            self.start_browser()
+
+        try:
+            # Human-like delay before navigation
+            self.browser.human_delay(0.5, 2.0)
+
+            # Navigate
+            logger.debug(f"Loading: {url}")
+            self.browser.driver.get(url)
+
+            # Wait for page load
+            time.sleep(random.uniform(3, 6))
+
+            # Check for CAPTCHA
+            if self.browser.check_for_captcha():
+                logger.warning("CAPTCHA detected!")
+                print("\n" + "!" * 50)
+                print("CAPTCHA DETECTED - Please solve it in the browser")
+                print("!" * 50 + "\n")
+                self.browser.wait_for_captcha_solve(timeout=120)
+
+            # Human actions
+            self.maybe_do_human_action()
+
+            # Extract SKUs
+            skus = self.extract_skus_from_page()
+
+            return skus
+
+        except Exception as e:
+            logger.error(f"Error checking page: {e}")
+            return set()
+
+    def update_mode(self):
+        """Update monitoring mode based on state"""
+        now = datetime.now()
+
+        if self.mode == MonitorMode.ALERT:
+            # Check if alert period is over
+            if self.alert_triggered_at:
+                elapsed = now - self.alert_triggered_at
+                if elapsed > self.alert_duration:
+                    logger.info("Alert period over, entering cooldown")
+                    self.mode = MonitorMode.COOLDOWN
+
+        elif self.mode == MonitorMode.COOLDOWN:
+            # Check if cooldown is over
+            if self.alert_triggered_at:
+                elapsed = now - self.alert_triggered_at
+                if elapsed > (self.alert_duration + self.cooldown_duration):
+                    logger.info("Cooldown over, returning to stealth mode")
+                    self.mode = MonitorMode.STEALTH
+                    self.alert_triggered_at = None
+
+    def trigger_alert_mode(self):
+        """Switch to alert mode"""
+        logger.info("!!! ENTERING ALERT MODE - Faster checks !!!")
+        self.mode = MonitorMode.ALERT
+        self.alert_triggered_at = datetime.now()
+
+    def send_discord_alert(self, sku: str, name: str, url: str):
+        """Send Discord notification"""
+        try:
+            from src.discord_notifier import DiscordNotifier
+            from config import DISCORD_WEBHOOK_URL
+
+            if DISCORD_WEBHOOK_URL and DISCORD_WEBHOOK_URL != "YOUR_WEBHOOK_URL_HERE":
+                notifier = DiscordNotifier(DISCORD_WEBHOOK_URL)
+                # Create a simple product dict
+                product = {
+                    'name': name,
+                    'url': f"https://www.pokemoncenter.com/product/{sku}",
+                    'price': 'Check site',
+                    'site': 'pokemoncenter',
+                }
+                notifier.send_stock_alert(product, "NEW BACKEND DETECTION")
+                logger.info("Discord alert sent!")
+        except Exception as e:
+            logger.warning(f"Could not send Discord alert: {e}")
+
+    def run_check(self) -> List[str]:
+        """Run a single check across all monitored URLs"""
+        self.check_count += 1
+        all_skus = set()
+        new_skus = []
+
+        logger.info(f"Check #{self.check_count} | Mode: {self.mode.value.upper()}")
+
+        for url in MONITOR_URLS:
+            skus = self.check_page(url)
+            all_skus.update(skus)
+
+            # Brief pause between pages
+            if url != MONITOR_URLS[-1]:
+                time.sleep(random.uniform(2, 5))
+
+        logger.info(f"Found {len(all_skus)} total SKUs")
+
+        # Find new SKUs
+        if self.known_skus:  # Only check if we have a baseline
+            new = all_skus - self.known_skus
+            if new:
+                for sku in new:
+                    logger.info(f"!!! NEW SKU DETECTED: {sku}")
+                    new_skus.append(sku)
+
+                    # Log and alert
+                    url = f"https://www.pokemoncenter.com/product/{sku}"
+                    self.log_detection(sku, f"New Product {sku}", url)
+                    self.send_discord_alert(sku, f"New Product {sku}", url)
+
+                # Trigger alert mode
+                self.trigger_alert_mode()
+
+        # Update known SKUs
+        self.known_skus.update(all_skus)
+        self.save_known_skus()
+
+        return new_skus
+
+    def run(self):
+        """Main monitoring loop"""
+        print()
+        print("=" * 60)
+        print("  SMART POKEMON CENTER MONITOR")
+        print("=" * 60)
+        print()
+        print("Modes:")
+        print(f"  STEALTH: {self.timing[MonitorMode.STEALTH]} sec (normal)")
+        print(f"  ALERT:   {self.timing[MonitorMode.ALERT]} sec (after detection)")
+        print(f"  COOLDOWN: {self.timing[MonitorMode.COOLDOWN]} sec (transition)")
+        print()
+        print("Press Ctrl+C to stop")
+        print("=" * 60)
+        print()
+
+        try:
+            # Initialize
+            self.load_known_skus()
+            self.start_browser()
+
+            # First check - build baseline if needed
+            if not self.known_skus:
+                logger.info("First run - building SKU baseline...")
+                self.run_check()
+                logger.info(f"Baseline: {len(self.known_skus)} products")
+                print()
+
+            # Main loop
+            while True:
+                # Maybe take a break
+                if self.maybe_take_break():
+                    continue
+
+                # Run check
+                new_skus = self.run_check()
+
+                if new_skus:
+                    print()
+                    print("!" * 60)
+                    print("!!! NEW PRODUCT(S) DETECTED !!!")
+                    for sku in new_skus:
+                        print(f"  -> {sku}")
+                    print("!" * 60)
+                    print()
+
+                # Update mode
+                self.update_mode()
+
+                # Get interval and wait
+                interval = self.get_interval()
+                logger.info(f"Next check in {interval:.0f}s")
+                time.sleep(interval)
+
+        except KeyboardInterrupt:
+            print("\nStopping monitor...")
+        except Exception as e:
+            logger.error(f"Monitor error: {e}")
+            import traceback
+            traceback.print_exc()
+        finally:
+            self.save_known_skus()
+            self.stop_browser()
+            print("Monitor stopped.")
+
+
+def main():
+    monitor = SmartMonitor()
+    monitor.run()
+
+
+if __name__ == "__main__":
+    main()

tools/stealth_browser.py

@@ -0,0 +1,539 @@
+"""
+Stealth Browser Module
+Uses undetected-chromedriver to bypass bot detection (Imperva, Cloudflare, etc.)
+Includes session persistence, proxy rotation, and human-like behavior.
+"""
+
+import os
+import json
+import time
+import random
+import logging
+import pickle
+from pathlib import Path
+from typing import Optional, List, Dict, Any
+from datetime import datetime, timedelta
+
+logger = logging.getLogger(__name__)
+
+# Data directory
+DATA_DIR = Path(__file__).parent.parent / "data"
+
+# Session storage directory
+SESSION_DIR = DATA_DIR / "sessions"
+SESSION_DIR.mkdir(parents=True, exist_ok=True)
+
+# Proxy configuration file
+PROXY_FILE = DATA_DIR / "proxies.json"
+
+
+class ProxyRotator:
+    """Manages proxy rotation for requests"""
+
+    def __init__(self, proxy_file: Path = PROXY_FILE):
+        self.proxies: List[Dict] = []
+        self.current_index = 0
+        self.failed_proxies: Dict[str, datetime] = {}
+        self.cooldown_minutes = 30
+
+        self._load_proxies(proxy_file)
+
+    def _load_proxies(self, proxy_file: Path):
+        """Load proxies from configuration file"""
+        if proxy_file.exists():
+            try:
+                with open(proxy_file, 'r') as f:
+                    data = json.load(f)
+                    self.proxies = data.get("proxies", [])
+                    logger.info(f"Loaded {len(self.proxies)} proxies")
+            except Exception as e:
+                logger.warning(f"Failed to load proxies: {e}")
+
+    def get_proxy(self) -> Optional[Dict]:
+        """Get next available proxy"""
+        if not self.proxies:
+            return None
+
+        # Clean up expired cooldowns
+        now = datetime.now()
+        self.failed_proxies = {
+            k: v for k, v in self.failed_proxies.items()
+            if now - v < timedelta(minutes=self.cooldown_minutes)
+        }
+
+        # Find next working proxy
+        attempts = 0
+        while attempts < len(self.proxies):
+            proxy = self.proxies[self.current_index]
+            proxy_key = f"{proxy.get('host')}:{proxy.get('port')}"
+
+            self.current_index = (self.current_index + 1) % len(self.proxies)
+
+            if proxy_key not in self.failed_proxies:
+                return proxy
+
+            attempts += 1
+
+        # All proxies in cooldown, return first one anyway
+        return self.proxies[0] if self.proxies else None
+
+    def mark_failed(self, proxy: Dict):
+        """Mark a proxy as failed (temporary cooldown)"""
+        if proxy:
+            proxy_key = f"{proxy.get('host')}:{proxy.get('port')}"
+            self.failed_proxies[proxy_key] = datetime.now()
+            logger.warning(f"Proxy {proxy_key} marked as failed")
+
+
+class StealthBrowser:
+    """
+    Stealth browser using undetected-chromedriver.
+    Designed to bypass Imperva/Incapsula and similar bot protection.
+    """
+
+    def __init__(
+        self,
+        headless: bool = False,
+        proxy: Optional[Dict] = None,
+        user_data_dir: Optional[str] = None,
+        session_name: str = "default"
+    ):
+        self.headless = headless
+        self.proxy = proxy
+        self.user_data_dir = user_data_dir
+        self.session_name = session_name
+        self.driver = None
+        self._setup_complete = False
+
+    def _get_chrome_options(self):
+        """Configure Chrome options for stealth"""
+        import undetected_chromedriver as uc
+
+        options = uc.ChromeOptions()
+
+        # Basic stealth settings - compatible with newer Chrome versions
+        options.add_argument("--disable-blink-features=AutomationControlled")
+        options.add_argument("--disable-dev-shm-usage")
+        options.add_argument("--no-sandbox")
+        options.add_argument("--disable-infobars")
+
+        # Window size (realistic resolution)
+        options.add_argument("--window-size=1920,1080")
+
+        # User data directory for session persistence
+        if self.user_data_dir:
+            options.add_argument(f"--user-data-dir={self.user_data_dir}")
+
+        # Proxy configuration
+        if self.proxy:
+            proxy_str = self._format_proxy(self.proxy)
+            if proxy_str:
+                options.add_argument(f"--proxy-server={proxy_str}")
+
+        # Headless mode (note: more detectable)
+        if self.headless:
+            options.add_argument("--headless=new")
+
+        return options
+
+    def _format_proxy(self, proxy: Dict) -> Optional[str]:
+        """Format proxy dict into Chrome proxy string"""
+        if not proxy:
+            return None
+
+        host = proxy.get("host")
+        port = proxy.get("port")
+
+        if not host or not port:
+            return None
+
+        protocol = proxy.get("protocol", "http")
+        return f"{protocol}://{host}:{port}"
+
+    def start(self):
+        """Start the browser"""
+        if self.driver:
+            return
+
+        try:
+            import undetected_chromedriver as uc
+
+            options = self._get_chrome_options()
+
+            # Create driver with version_main to match installed Chrome version
+            # This prevents "ChromeDriver only supports Chrome version X" errors
+            self.driver = uc.Chrome(
+                options=options,
+                use_subprocess=True,
+                version_main=146,  # Match user's Chrome version
+            )
+
+            # Set realistic viewport
+            self.driver.set_window_size(1920, 1080)
+
+            # Load saved cookies if they exist
+            self._load_cookies()
+
+            self._setup_complete = True
+            logger.info("Stealth browser started successfully")
+
+        except Exception as e:
+            logger.error(f"Failed to start stealth browser: {e}")
+            raise
+
+    def stop(self):
+        """Stop the browser and save session"""
+        if self.driver:
+            try:
+                self._save_cookies()
+                self.driver.quit()
+            except Exception as e:
+                logger.warning(f"Error stopping browser: {e}")
+            finally:
+                self.driver = None
+                self._setup_complete = False
+
+    def _get_cookie_file(self) -> Path:
+        """Get path to cookie file for this session"""
+        return SESSION_DIR / f"{self.session_name}_cookies.pkl"
+
+    def _save_cookies(self):
+        """Save cookies to file for session persistence"""
+        if not self.driver:
+            return
+
+        try:
+            cookies = self.driver.get_cookies()
+            cookie_file = self._get_cookie_file()
+
+            with open(cookie_file, 'wb') as f:
+                pickle.dump(cookies, f)
+
+            logger.debug(f"Saved {len(cookies)} cookies to {cookie_file}")
+        except Exception as e:
+            logger.warning(f"Failed to save cookies: {e}")
+
+    def _load_cookies(self):
+        """Load cookies from file"""
+        cookie_file = self._get_cookie_file()
+
+        if not cookie_file.exists():
+            return
+
+        try:
+            with open(cookie_file, 'rb') as f:
+                cookies = pickle.load(f)
+
+            # Need to visit domain first before adding cookies
+            # This will be done when navigating to the actual page
+
+            self._pending_cookies = cookies
+            logger.debug(f"Loaded {len(cookies)} cookies from {cookie_file}")
+        except Exception as e:
+            logger.warning(f"Failed to load cookies: {e}")
+            self._pending_cookies = []
+
+    def _apply_pending_cookies(self, domain: str):
+        """Apply loaded cookies after visiting domain"""
+        if not hasattr(self, '_pending_cookies') or not self._pending_cookies:
+            return
+
+        for cookie in self._pending_cookies:
+            try:
+                # Only add cookies for matching domain
+                if domain in cookie.get('domain', ''):
+                    self.driver.add_cookie(cookie)
+            except Exception:
+                pass  # Some cookies may fail, that's ok
+
+        self._pending_cookies = []
+
+    def human_delay(self, min_seconds: float = 1.0, max_seconds: float = 3.0):
+        """Add human-like random delay"""
+        delay = random.uniform(min_seconds, max_seconds)
+        time.sleep(delay)
+
+    def human_scroll(self):
+        """Scroll like a human would"""
+        if not self.driver:
+            return
+
+        # Random scroll amount
+        scroll_amount = random.randint(200, 600)
+
+        # Smooth scroll
+        self.driver.execute_script(f"""
+            window.scrollBy({{
+                top: {scroll_amount},
+                behavior: 'smooth'
+            }});
+        """)
+
+        self.human_delay(0.5, 1.5)
+
+    def human_mouse_move(self):
+        """Simulate mouse movement (basic)"""
+        if not self.driver:
+            return
+
+        # Move mouse to random position
+        try:
+            from selenium.webdriver.common.action_chains import ActionChains
+
+            actions = ActionChains(self.driver)
+
+            # Random coordinates within viewport
+            x = random.randint(100, 800)
+            y = random.randint(100, 600)
+
+            # Move by offset from current position
+            actions.move_by_offset(x, y).perform()
+
+            # Reset position
+            actions.move_by_offset(-x, -y).perform()
+        except Exception:
+            pass  # Mouse movement is optional
+
+    def get_page(self, url: str, wait_time: float = 5.0) -> str:
+        """
+        Navigate to URL with human-like behavior.
+
+        Args:
+            url: URL to navigate to
+            wait_time: Time to wait for page load
+
+        Returns:
+            Page HTML content
+        """
+        if not self.driver:
+            self.start()
+
+        try:
+            # Pre-navigation delay
+            self.human_delay(0.5, 1.5)
+
+            # Navigate
+            self.driver.get(url)
+
+            # Apply any pending cookies
+            from urllib.parse import urlparse
+            domain = urlparse(url).netloc
+            self._apply_pending_cookies(domain)
+
+            # Wait for page load
+            time.sleep(wait_time)
+
+            # Human-like behavior
+            self.human_scroll()
+            self.human_delay(1, 2)
+            self.human_mouse_move()
+
+            # Get page content
+            html = self.driver.page_source
+
+            # Save cookies after successful page load
+            self._save_cookies()
+
+            return html
+
+        except Exception as e:
+            logger.error(f"Error getting page {url}: {e}")
+            raise
+
+    def check_for_captcha(self) -> bool:
+        """Check if page has a CAPTCHA challenge blocking content"""
+        if not self.driver:
+            return False
+
+        page_source = self.driver.page_source.lower()
+
+        # First check: Did the page load actual content?
+        # If we see product elements, it's NOT a CAPTCHA page
+        content_loaded_indicators = [
+            'class="product-card',
+            'data-product-id',
+            'data-sku',
+            '/product/',
+            'add to cart',
+            'product-grid',
+            'product-list',
+        ]
+
+        for indicator in content_loaded_indicators:
+            if indicator in page_source:
+                # Page has actual content - no CAPTCHA
+                return False
+
+        # Only flag as CAPTCHA if we see blocking indicators AND no content
+        captcha_indicators = [
+            "verify you are human",
+            "press & hold",
+            "press and hold",
+            "checking your browser",
+            "just a moment",
+            "enable javascript and cookies",
+            "access denied",
+            "blocked",
+            "challenge-running",
+            "cf-browser-verification",
+            "ddos-guard",
+        ]
+
+        for indicator in captcha_indicators:
+            if indicator in page_source:
+                return True
+
+        # Also check if page is suspiciously empty (might be blocked)
+        if len(page_source) < 5000 and "pokemoncenter" not in page_source:
+            return True
+
+        return False
+
+    def wait_for_captcha_solve(self, timeout: int = 120):
+        """
+        Wait for user to solve CAPTCHA manually.
+        Only works in non-headless mode.
+        """
+        if self.headless:
+            logger.warning("Cannot solve CAPTCHA in headless mode")
+            return False
+
+        logger.info("CAPTCHA detected! Please solve it manually...")
+        print("\n" + "=" * 50)
+        print("CAPTCHA DETECTED!")
+        print("Please solve the CAPTCHA in the browser window.")
+        print("=" * 50 + "\n")
+
+        start_time = time.time()
+
+        while time.time() - start_time < timeout:
+            if not self.check_for_captcha():
+                logger.info("CAPTCHA solved!")
+                self._save_cookies()  # Save session after solving
+                return True
+
+            time.sleep(2)
+
+        logger.warning("CAPTCHA solve timeout")
+        return False
+
+    def screenshot(self, filename: str = "screenshot.png"):
+        """Take a screenshot for debugging"""
+        if self.driver:
+            try:
+                self.driver.save_screenshot(filename)
+                logger.info(f"Screenshot saved to {filename}")
+            except Exception as e:
+                logger.warning(f"Failed to save screenshot: {e}")
+
+
+class StealthBrowserPool:
+    """
+    Manages multiple stealth browser instances with proxy rotation.
+    """
+
+    def __init__(
+        self,
+        pool_size: int = 1,
+        use_proxies: bool = False,
+        headless: bool = False
+    ):
+        self.pool_size = pool_size
+        self.use_proxies = use_proxies
+        self.headless = headless
+        self.browsers: List[StealthBrowser] = []
+        self.proxy_rotator = ProxyRotator() if use_proxies else None
+        self.current_index = 0
+
+    def get_browser(self) -> StealthBrowser:
+        """Get a browser from the pool"""
+        # Create browser if pool is empty
+        if not self.browsers:
+            proxy = self.proxy_rotator.get_proxy() if self.proxy_rotator else None
+            browser = StealthBrowser(
+                headless=self.headless,
+                proxy=proxy,
+                session_name=f"pool_{self.current_index}"
+            )
+            browser.start()
+            self.browsers.append(browser)
+            return browser
+
+        # Rotate through browsers
+        browser = self.browsers[self.current_index]
+        self.current_index = (self.current_index + 1) % len(self.browsers)
+        return browser
+
+    def shutdown_all(self):
+        """Shutdown all browsers in pool"""
+        for browser in self.browsers:
+            try:
+                browser.stop()
+            except Exception:
+                pass
+        self.browsers = []
+
+
+# Global instances
+_stealth_browser: Optional[StealthBrowser] = None
+_browser_pool: Optional[StealthBrowserPool] = None
+
+
+def get_stealth_browser(
+    headless: bool = False,
+    session_name: str = "pokemoncenter"
+) -> StealthBrowser:
+    """Get or create the global stealth browser instance"""
+    global _stealth_browser
+
+    if _stealth_browser is None:
+        _stealth_browser = StealthBrowser(
+            headless=headless,
+            session_name=session_name
+        )
+
+    if not _stealth_browser._setup_complete:
+        _stealth_browser.start()
+
+    return _stealth_browser
+
+
+def shutdown_stealth_browser():
+    """Shutdown the global stealth browser"""
+    global _stealth_browser
+
+    if _stealth_browser:
+        _stealth_browser.stop()
+        _stealth_browser = None
+
+
+def create_proxy_config_template():
+    """Create a template proxies.json file"""
+    template = {
+        "proxies": [
+            {
+                "host": "proxy1.example.com",
+                "port": 8080,
+                "protocol": "http",
+                "username": "user",
+                "password": "pass"
+            },
+            {
+                "host": "proxy2.example.com",
+                "port": 8080,
+                "protocol": "http",
+                "username": "user",
+                "password": "pass"
+            }
+        ],
+        "_comment": "Add your residential proxies here. Recommended providers: Bright Data, Oxylabs, Smartproxy"
+    }
+
+    if not PROXY_FILE.exists():
+        with open(PROXY_FILE, 'w') as f:
+            json.dump(template, f, indent=2)
+        logger.info(f"Created proxy template at {PROXY_FILE}")
+
+
+# Create template on import
+create_proxy_config_template()

tools/warmup.py

@@ -0,0 +1,74 @@
+"""
+Warmup tool for sites with CAPTCHA/bot protection.
+Run this before the main monitor to solve CAPTCHAs manually.
+
+Usage:
+    python tools/warmup.py gamestop
+    python tools/warmup.py all
+"""
+
+import sys
+import os
+
+# Add parent directory to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import logging
+from scrapers.gamestop import warmup_gamestop
+
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+)
+logger = logging.getLogger(__name__)
+
+
+def warmup_all():
+    """Warm up all sites that need it"""
+    results = {}
+
+    logger.info("=" * 50)
+    logger.info("Starting warmup for GameStop...")
+    logger.info("=" * 50)
+    results["gamestop"] = warmup_gamestop()
+
+    # Add more sites here as needed
+    # results["pokemoncenter"] = warmup_pokemoncenter()
+
+    logger.info("=" * 50)
+    logger.info("Warmup Results:")
+    for site, success in results.items():
+        status = "OK" if success else "FAILED"
+        logger.info(f"  {site}: {status}")
+    logger.info("=" * 50)
+
+    return all(results.values())
+
+
+def main():
+    if len(sys.argv) < 2:
+        print(__doc__)
+        print("\nAvailable sites: gamestop, all")
+        sys.exit(1)
+
+    site = sys.argv[1].lower()
+
+    if site == "gamestop":
+        success = warmup_gamestop()
+    elif site == "all":
+        success = warmup_all()
+    else:
+        print(f"Unknown site: {site}")
+        print("Available sites: gamestop, all")
+        sys.exit(1)
+
+    if success:
+        logger.info("Warmup completed successfully!")
+        sys.exit(0)
+    else:
+        logger.error("Warmup failed or timed out")
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()

tools/warmup_session.py

@@ -0,0 +1,119 @@
+"""
+Session Warmup Tool for Pokemon Center
+
+Run this BEFORE starting the monitor to:
+1. Open a stealth browser
+2. Let you manually browse and solve any CAPTCHAs
+3. Save cookies/session for the monitor to reuse
+
+Usage:
+    python warmup_session.py
+
+After running:
+1. Browse Pokemon Center normally for a few minutes
+2. Add items to cart, look at products, etc.
+3. Solve any CAPTCHAs that appear
+4. Press Enter in this terminal when done
+5. The session will be saved and reused by the monitor
+"""
+
+import sys
+import time
+
+print("=" * 60)
+print("Pokemon Center Session Warmup")
+print("=" * 60)
+print()
+
+# Import stealth browser
+try:
+    from .stealth_browser import StealthBrowser
+except ImportError as e:
+    print(f"Error importing stealth browser: {e}")
+    sys.exit(1)
+
+print("Starting stealth browser...")
+print("This will open a Chrome window.")
+print()
+
+browser = StealthBrowser(
+    headless=False,
+    session_name="pokemoncenter"  # Same name used by monitor
+)
+
+try:
+    browser.start()
+    print("[OK] Browser started!")
+    print()
+
+    # Navigate to Pokemon Center
+    print("Navigating to Pokemon Center...")
+    browser.driver.get("https://www.pokemoncenter.com")
+
+    time.sleep(3)
+
+    print()
+    print("=" * 60)
+    print("WARMUP INSTRUCTIONS")
+    print("=" * 60)
+    print("""
+1. If you see a CAPTCHA or "Pardon Our Interruption":
+   - Solve it in the browser window
+   - Wait for the page to load
+
+2. Browse naturally for 2-3 minutes:
+   - Click on some products
+   - Look at different categories
+   - Add something to cart (you don't have to buy)
+   - This builds a legitimate browsing profile
+
+3. Navigate to the TCG section:
+   - https://www.pokemoncenter.com/category/tcg-cards
+
+4. Once you're browsing normally without issues:
+   - Come back here
+   - Press ENTER to save the session
+""")
+    print("=" * 60)
+    print()
+
+    input("Press ENTER when you're done browsing to save the session...")
+
+    print()
+    print("Saving session cookies...")
+    browser._save_cookies()
+
+    # Check how many cookies we got
+    cookies = browser.driver.get_cookies()
+    print(f"[OK] Saved {len(cookies)} cookies")
+
+    # Get current URL for reference
+    current_url = browser.driver.current_url
+    print(f"[OK] Final URL: {current_url}")
+
+    print()
+    print("=" * 60)
+    print("SESSION SAVED!")
+    print("=" * 60)
+    print("""
+Your session has been saved. The monitor will now use these
+cookies when checking Pokemon Center.
+
+Tips for best results:
+- Run the monitor with 3-5 minute check intervals
+- Keep USE_STEALTH_BROWSER = True in config.py
+- If you get blocked again, run this warmup again
+
+Session file: data/sessions/pokemoncenter_cookies.pkl
+""")
+
+except KeyboardInterrupt:
+    print("\nInterrupted by user")
+except Exception as e:
+    print(f"Error: {e}")
+    import traceback
+    traceback.print_exc()
+finally:
+    print("Closing browser...")
+    browser.stop()
+    print("Done!")