Pokemon Stock Monitor - Initial commit
Chrome extension for PokemonCenter monitoring with Discord notifications. Includes Python scripts for Target monitoring. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
+20
@@ -0,0 +1,20 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.pyo
|
||||
.env
|
||||
venv/
|
||||
.venv/
|
||||
|
||||
# Logs and data
|
||||
*.log
|
||||
products.json
|
||||
debug_*.png
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
+286
@@ -0,0 +1,286 @@
|
||||
# Pokemon Stock Monitor - Setup Guide
|
||||
|
||||
## Overview
|
||||
This monitor watches retail sites for Pokemon TCG restocks and new drops, sending Discord notifications when products become available.
|
||||
|
||||
**Supported Sites:**
|
||||
- Target (working)
|
||||
- PokemonCenter (requires special setup - see below)
|
||||
- Walmart (planned)
|
||||
- BestBuy (planned)
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.10+
|
||||
- Chrome/Chromium browser
|
||||
- Discord server with webhook access
|
||||
- ~500MB disk space (for browser)
|
||||
|
||||
---
|
||||
|
||||
## Quick Start (5 minutes)
|
||||
|
||||
### 1. Clone/Copy the Project
|
||||
```bash
|
||||
# Copy the pokemon-stock-monitor folder to your server
|
||||
cd pokemon-stock-monitor
|
||||
```
|
||||
|
||||
### 2. Install Dependencies
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
playwright install chromium
|
||||
```
|
||||
|
||||
### 3. Configure Discord Webhook
|
||||
|
||||
1. Open Discord → Your Server → Server Settings → Integrations → Webhooks
|
||||
2. Click "New Webhook"
|
||||
3. Name it "Pokemon Stock Monitor"
|
||||
4. Copy the Webhook URL
|
||||
5. Edit `config.py`:
|
||||
```python
|
||||
DISCORD_WEBHOOK_URL = "https://discord.com/api/webhooks/YOUR_WEBHOOK_URL_HERE"
|
||||
```
|
||||
|
||||
### 4. Configure Sites to Monitor
|
||||
|
||||
Edit `config.py`:
|
||||
```python
|
||||
# Enable the sites you want
|
||||
SITES_ENABLED = ["target"] # Add "pokemoncenter" after special setup
|
||||
|
||||
# Customize check interval (seconds)
|
||||
CHECK_INTERVAL_SECONDS = 60
|
||||
|
||||
# For headless server, set to True
|
||||
HEADLESS = True
|
||||
```
|
||||
|
||||
### 5. Run the Monitor
|
||||
```bash
|
||||
python main.py
|
||||
```
|
||||
|
||||
You should see:
|
||||
```
|
||||
==================================================
|
||||
Pokemon Stock Monitor
|
||||
==================================================
|
||||
Monitoring: target
|
||||
Check interval: 60 seconds
|
||||
==================================================
|
||||
|
||||
Running initial check...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Running as a Background Service
|
||||
|
||||
### Option A: Using Screen (Linux)
|
||||
```bash
|
||||
# Start a screen session
|
||||
screen -S pokemon-monitor
|
||||
|
||||
# Run the monitor
|
||||
python main.py
|
||||
|
||||
# Detach: Press Ctrl+A, then D
|
||||
# Reattach later: screen -r pokemon-monitor
|
||||
```
|
||||
|
||||
### Option B: Using systemd (Linux)
|
||||
|
||||
Create `/etc/systemd/system/pokemon-monitor.service`:
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Pokemon Stock Monitor
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=YOUR_USERNAME
|
||||
WorkingDirectory=/path/to/pokemon-stock-monitor
|
||||
ExecStart=/usr/bin/python3 main.py
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Then:
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable pokemon-monitor
|
||||
sudo systemctl start pokemon-monitor
|
||||
|
||||
# Check status
|
||||
sudo systemctl status pokemon-monitor
|
||||
|
||||
# View logs
|
||||
journalctl -u pokemon-monitor -f
|
||||
```
|
||||
|
||||
### Option C: Using Task Scheduler (Windows)
|
||||
|
||||
1. Open Task Scheduler
|
||||
2. Create Basic Task → Name: "Pokemon Monitor"
|
||||
3. Trigger: "When the computer starts"
|
||||
4. Action: Start a program
|
||||
- Program: `python`
|
||||
- Arguments: `main.py`
|
||||
- Start in: `C:\path\to\pokemon-stock-monitor`
|
||||
5. Check "Run whether user is logged on or not"
|
||||
|
||||
---
|
||||
|
||||
## PokemonCenter Setup (Special - Requires GUI)
|
||||
|
||||
PokemonCenter has strong bot protection (Imperva). To monitor it:
|
||||
|
||||
### Method 1: Use Real Chrome Profile (Recommended)
|
||||
|
||||
This uses your actual Chrome browser with all its cookies/history, making it appear human.
|
||||
|
||||
1. **Find your Chrome profile path:**
|
||||
- Windows: `C:\Users\USERNAME\AppData\Local\Google\Chrome\User Data`
|
||||
- Linux: `~/.config/google-chrome`
|
||||
- Mac: `~/Library/Application Support/Google/Chrome`
|
||||
|
||||
2. **Edit `config.py`:**
|
||||
```python
|
||||
# Add this line
|
||||
CHROME_USER_DATA_DIR = "C:\\Users\\USERNAME\\AppData\\Local\\Google\\Chrome\\User Data"
|
||||
|
||||
# Enable PokemonCenter
|
||||
SITES_ENABLED = ["target", "pokemoncenter"]
|
||||
|
||||
# Must be False to use Chrome profile
|
||||
HEADLESS = False
|
||||
```
|
||||
|
||||
3. **Important:** Close Chrome before running the monitor (can't use same profile twice)
|
||||
|
||||
4. **First run:** Manually solve any CAPTCHA that appears, then the session should stay valid
|
||||
|
||||
### Method 2: VM with Desktop Environment
|
||||
|
||||
If running on a headless server, set up a VM with a desktop:
|
||||
|
||||
1. Install a lightweight desktop (XFCE, LXDE)
|
||||
2. Install Chrome and browse PokemonCenter manually once
|
||||
3. Run the monitor with `HEADLESS = False`
|
||||
4. Use VNC/RDP to check on it occasionally
|
||||
|
||||
---
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### config.py Options
|
||||
|
||||
```python
|
||||
# Discord webhook URL (required)
|
||||
DISCORD_WEBHOOK_URL = "https://discord.com/api/webhooks/..."
|
||||
|
||||
# Check interval in seconds (60 = 1 minute)
|
||||
CHECK_INTERVAL_SECONDS = 60
|
||||
|
||||
# Sites to monitor
|
||||
SITES_ENABLED = ["target"] # Options: "target", "pokemoncenter"
|
||||
|
||||
# URLs to monitor per site
|
||||
TARGET_URLS = [
|
||||
"https://www.target.com/s?searchTerm=pokemon+tcg",
|
||||
]
|
||||
|
||||
POKEMON_CENTER_URLS = [
|
||||
"https://www.pokemoncenter.com/category/tcg-cards?sort=relevance",
|
||||
]
|
||||
|
||||
# Keyword filtering (optional)
|
||||
KEYWORD_FILTER_ENABLED = False
|
||||
KEYWORDS = ["chaos rising", "booster", "etb", "elite trainer"]
|
||||
|
||||
# Browser settings
|
||||
HEADLESS = True # True for servers, False for desktop/VM
|
||||
SLOW_MO = 0 # Slow down browser (ms) for debugging
|
||||
|
||||
# Logging level
|
||||
LOG_LEVEL = "INFO" # DEBUG, INFO, WARNING, ERROR
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "No products found"
|
||||
- Check `debug_screenshot.png` or `debug_target.png` in the project folder
|
||||
- The site may have changed its HTML structure
|
||||
- Try increasing wait times in `browser.py`
|
||||
|
||||
### "Discord notification not sending"
|
||||
- Verify webhook URL is correct in `config.py`
|
||||
- Test webhook: `python -c "from discord_notifier import test_webhook; test_webhook()"`
|
||||
|
||||
### "Browser failed to start"
|
||||
```bash
|
||||
# Reinstall Playwright browsers
|
||||
playwright install chromium --force
|
||||
|
||||
# On Linux, install dependencies
|
||||
playwright install-deps chromium
|
||||
```
|
||||
|
||||
### "Access denied" / Bot blocked
|
||||
- PokemonCenter: Use the Chrome profile method above
|
||||
- Target: Should work, try increasing `CHECK_INTERVAL_SECONDS` to 120+
|
||||
- All sites: Don't run too frequently (rate limiting)
|
||||
|
||||
### High CPU/Memory Usage
|
||||
- Set `HEADLESS = True` (uses less resources)
|
||||
- Increase `CHECK_INTERVAL_SECONDS`
|
||||
- The browser stays open between checks (by design, for session persistence)
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
pokemon-stock-monitor/
|
||||
├── main.py # Entry point - run this
|
||||
├── config.py # Configuration - edit this
|
||||
├── browser.py # Browser automation
|
||||
├── discord_notifier.py # Discord webhook
|
||||
├── product_tracker.py # Tracks products for restock detection
|
||||
├── products.json # Auto-generated product database
|
||||
├── monitor.log # Log file
|
||||
├── scrapers/
|
||||
│ ├── base.py # Base scraper class
|
||||
│ ├── pokemoncenter.py # PokemonCenter scraper
|
||||
│ └── target.py # Target scraper
|
||||
└── requirements.txt # Python dependencies
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding More Sites
|
||||
|
||||
The scraper architecture is modular. To add a new site:
|
||||
|
||||
1. Create `scrapers/newsite.py` based on `target.py`
|
||||
2. Add to `scrapers/__init__.py`
|
||||
3. Add URL config to `config.py`
|
||||
4. Add check function to `main.py`
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues:
|
||||
1. Check `monitor.log` for errors
|
||||
2. Check debug screenshots (`debug_*.png`)
|
||||
3. Try running with `LOG_LEVEL = "DEBUG"` for more info
|
||||
+183
@@ -0,0 +1,183 @@
|
||||
"""
|
||||
Browser automation with Playwright and stealth patches
|
||||
Handles bot protection bypass for retail sites
|
||||
"""
|
||||
|
||||
import logging
|
||||
from playwright.sync_api import sync_playwright, Browser, Page, BrowserContext
|
||||
from playwright_stealth import Stealth
|
||||
from config import HEADLESS, SLOW_MO, CHROME_USER_DATA_DIR, USE_REAL_CHROME
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Create stealth instance
|
||||
stealth = Stealth()
|
||||
|
||||
|
||||
class StealthBrowser:
|
||||
"""Manages a stealth browser instance that bypasses bot detection"""
|
||||
|
||||
def __init__(self):
|
||||
self.playwright = None
|
||||
self.browser: Browser = None
|
||||
self.context: BrowserContext = None
|
||||
self.using_persistent = False
|
||||
|
||||
def start(self):
|
||||
"""Initialize the browser"""
|
||||
logger.info("Starting stealth browser...")
|
||||
self.playwright = sync_playwright().start()
|
||||
|
||||
# Check if we should use a persistent Chrome profile
|
||||
if CHROME_USER_DATA_DIR:
|
||||
logger.info(f"Using Chrome profile from: {CHROME_USER_DATA_DIR}")
|
||||
self._start_with_profile()
|
||||
else:
|
||||
self._start_fresh()
|
||||
|
||||
logger.info("Browser started successfully")
|
||||
|
||||
def _start_fresh(self):
|
||||
"""Start with a fresh browser profile"""
|
||||
# Launch browser - use real Chrome if configured
|
||||
launch_options = {
|
||||
"headless": HEADLESS,
|
||||
"slow_mo": SLOW_MO,
|
||||
"args": [
|
||||
"--disable-blink-features=AutomationControlled",
|
||||
"--disable-dev-shm-usage",
|
||||
"--no-sandbox",
|
||||
]
|
||||
}
|
||||
|
||||
if USE_REAL_CHROME:
|
||||
launch_options["channel"] = "chrome"
|
||||
logger.info("Using real Chrome browser")
|
||||
|
||||
self.browser = self.playwright.chromium.launch(**launch_options)
|
||||
|
||||
# Create context with realistic viewport and user agent
|
||||
self.context = self.browser.new_context(
|
||||
viewport={"width": 1920, "height": 1080},
|
||||
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
|
||||
locale="en-US",
|
||||
timezone_id="America/New_York",
|
||||
)
|
||||
|
||||
# Apply stealth to context (affects all pages created from it)
|
||||
stealth.apply_stealth_sync(self.context)
|
||||
|
||||
def _start_with_profile(self):
|
||||
"""Start using an existing Chrome profile (better for bot detection)"""
|
||||
self.using_persistent = True
|
||||
|
||||
# Launch persistent context with user's REAL Chrome (not Playwright's Chromium)
|
||||
# This is needed because Chrome encrypts credentials with its own binary
|
||||
self.context = self.playwright.chromium.launch_persistent_context(
|
||||
user_data_dir=CHROME_USER_DATA_DIR,
|
||||
channel="chrome", # Use real Chrome, not Playwright's Chromium
|
||||
headless=HEADLESS,
|
||||
slow_mo=SLOW_MO,
|
||||
viewport={"width": 1920, "height": 1080},
|
||||
locale="en-US",
|
||||
timezone_id="America/New_York",
|
||||
args=[
|
||||
"--disable-blink-features=AutomationControlled",
|
||||
"--disable-dev-shm-usage",
|
||||
]
|
||||
)
|
||||
|
||||
# Apply stealth
|
||||
stealth.apply_stealth_sync(self.context)
|
||||
|
||||
def new_page(self) -> Page:
|
||||
"""Create a new page with stealth patches applied"""
|
||||
if not self.context:
|
||||
raise RuntimeError("Browser not started. Call start() first.")
|
||||
|
||||
page = self.context.new_page()
|
||||
return page
|
||||
|
||||
def get_page_content(self, url: str, wait_for_selector: str = None, timeout: int = 30000) -> tuple[Page, str]:
|
||||
"""
|
||||
Navigate to URL and return page + HTML content
|
||||
|
||||
Args:
|
||||
url: URL to navigate to
|
||||
wait_for_selector: CSS selector to wait for before returning
|
||||
timeout: Max time to wait in milliseconds
|
||||
|
||||
Returns:
|
||||
Tuple of (page, html_content)
|
||||
"""
|
||||
page = self.new_page()
|
||||
|
||||
try:
|
||||
logger.info(f"Navigating to: {url}")
|
||||
page.goto(url, wait_until="domcontentloaded", timeout=timeout)
|
||||
|
||||
# Wait for specific element if provided
|
||||
if wait_for_selector:
|
||||
logger.debug(f"Waiting for selector: {wait_for_selector}")
|
||||
page.wait_for_selector(wait_for_selector, timeout=timeout)
|
||||
else:
|
||||
# Wait for DOM to be ready, then give extra time for JS rendering
|
||||
try:
|
||||
page.wait_for_load_state("networkidle", timeout=15000)
|
||||
except:
|
||||
# networkidle can timeout on dynamic sites, that's OK
|
||||
logger.debug("networkidle timeout, continuing anyway")
|
||||
|
||||
# Let dynamic content render
|
||||
page.wait_for_timeout(5000)
|
||||
|
||||
# Scroll down to trigger lazy loading
|
||||
page.evaluate("window.scrollTo(0, document.body.scrollHeight / 2)")
|
||||
page.wait_for_timeout(2000)
|
||||
page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
|
||||
page.wait_for_timeout(2000)
|
||||
page.evaluate("window.scrollTo(0, 0)")
|
||||
page.wait_for_timeout(1000)
|
||||
|
||||
content = page.content()
|
||||
logger.info(f"Page loaded, content length: {len(content)}")
|
||||
|
||||
return page, content
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error loading page {url}: {e}")
|
||||
page.close()
|
||||
raise
|
||||
|
||||
def stop(self):
|
||||
"""Clean up browser resources"""
|
||||
logger.info("Stopping browser...")
|
||||
if self.context:
|
||||
self.context.close()
|
||||
if self.browser and not self.using_persistent:
|
||||
# Only close browser if not using persistent context
|
||||
self.browser.close()
|
||||
if self.playwright:
|
||||
self.playwright.stop()
|
||||
logger.info("Browser stopped")
|
||||
|
||||
|
||||
# Global browser instance
|
||||
_browser: StealthBrowser = None
|
||||
|
||||
|
||||
def get_browser() -> StealthBrowser:
|
||||
"""Get or create the global browser instance"""
|
||||
global _browser
|
||||
if _browser is None:
|
||||
_browser = StealthBrowser()
|
||||
_browser.start()
|
||||
return _browser
|
||||
|
||||
|
||||
def shutdown_browser():
|
||||
"""Shutdown the global browser instance"""
|
||||
global _browser
|
||||
if _browser:
|
||||
_browser.stop()
|
||||
_browser = None
|
||||
@@ -0,0 +1,68 @@
|
||||
# Pokemon Stock Monitor - Chrome Extension
|
||||
|
||||
A Chrome extension that monitors PokemonCenter for stock changes and sends Discord notifications.
|
||||
|
||||
## Why a Chrome Extension?
|
||||
|
||||
PokemonCenter uses Imperva bot protection that blocks automated browsers (Playwright, Selenium). This extension runs **inside your real Chrome browser**, making it undetectable - to the website, it looks exactly like you browsing manually.
|
||||
|
||||
## Installation
|
||||
|
||||
1. **Add Icons** (optional but recommended):
|
||||
- Create or download three PNG icons: `icon16.png`, `icon48.png`, `icon128.png`
|
||||
- Place them in this folder
|
||||
- You can use any Pokemon-themed icons or simple colored squares
|
||||
|
||||
2. **Load the Extension**:
|
||||
- Open Chrome and go to `chrome://extensions`
|
||||
- Enable "Developer mode" (toggle in top right)
|
||||
- Click "Load unpacked"
|
||||
- Select this `chrome-extension` folder
|
||||
- The extension icon should appear in your toolbar
|
||||
|
||||
3. **Configure**:
|
||||
- Click the extension icon
|
||||
- Paste your Discord webhook URL
|
||||
- Add URLs to monitor (default is PokemonCenter TCG)
|
||||
- Set check interval (1-60 minutes)
|
||||
- Click "Save Settings"
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Background Service Worker**: Runs continuously in Chrome
|
||||
2. **Periodic Checks**: Uses Chrome's alarm API to check at your interval
|
||||
3. **Product Parsing**: Fetches pages and parses product data from HTML
|
||||
4. **Change Detection**: Compares against known products to detect:
|
||||
- New product listings
|
||||
- Restocks (was out of stock, now in stock)
|
||||
5. **Discord Notifications**: Sends rich embeds with product info and direct links
|
||||
|
||||
## Features
|
||||
|
||||
- **Keyword Filtering**: Only track products matching specific keywords
|
||||
- **New Product Alerts**: Get notified when new items appear
|
||||
- **Restock Alerts**: Get notified when out-of-stock items return
|
||||
- **Browser Notifications**: Local notifications in addition to Discord
|
||||
- **Persistent Storage**: Remembers products across browser restarts
|
||||
|
||||
## Tips
|
||||
|
||||
- **Keep Chrome Running**: The extension only works while Chrome is open
|
||||
- **First Run**: After installing, click "Check Now" to do an initial scan
|
||||
- **Clear History**: Use this if you want to re-detect all products as "new"
|
||||
- **Multiple URLs**: Add one URL per line in the URLs field
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**No notifications sending:**
|
||||
- Check that your Discord webhook URL is correct
|
||||
- Make sure "Monitor Enabled" is toggled on
|
||||
- Check Chrome's console for errors (right-click extension → Inspect popup)
|
||||
|
||||
**Extension not loading:**
|
||||
- Make sure manifest.json is valid JSON
|
||||
- Check for any icon files referenced but missing
|
||||
|
||||
**Products not detected:**
|
||||
- PokemonCenter may have changed their HTML structure
|
||||
- Check the browser console for parsing errors
|
||||
@@ -0,0 +1,317 @@
|
||||
// Pokemon Stock Monitor - Background Service Worker
|
||||
|
||||
const DEFAULT_CONFIG = {
|
||||
discordWebhook: "",
|
||||
checkIntervalMinutes: 1,
|
||||
enabled: true,
|
||||
urls: [
|
||||
"https://www.pokemoncenter.com/category/tcg-cards?sort=relevance"
|
||||
],
|
||||
keywords: [], // Empty = all products, or ["chaos rising", "booster"] etc
|
||||
notifyNewProducts: true,
|
||||
notifyRestocks: true
|
||||
};
|
||||
|
||||
// Store known products
|
||||
let knownProducts = {};
|
||||
let config = DEFAULT_CONFIG;
|
||||
|
||||
// Initialize
|
||||
chrome.runtime.onInstalled.addListener(() => {
|
||||
console.log("Pokemon Stock Monitor installed");
|
||||
loadConfig();
|
||||
loadProducts();
|
||||
setupAlarm();
|
||||
});
|
||||
|
||||
// Load on startup
|
||||
chrome.runtime.onStartup.addListener(() => {
|
||||
loadConfig();
|
||||
loadProducts();
|
||||
setupAlarm();
|
||||
});
|
||||
|
||||
// Handle alarm
|
||||
chrome.alarms.onAlarm.addListener((alarm) => {
|
||||
if (alarm.name === "stockCheck") {
|
||||
runStockCheck();
|
||||
}
|
||||
});
|
||||
|
||||
// Setup periodic alarm
|
||||
function setupAlarm() {
|
||||
chrome.alarms.create("stockCheck", {
|
||||
periodInMinutes: config.checkIntervalMinutes
|
||||
});
|
||||
console.log(`Alarm set for every ${config.checkIntervalMinutes} minute(s)`);
|
||||
}
|
||||
|
||||
// Load config from storage
|
||||
async function loadConfig() {
|
||||
const stored = await chrome.storage.local.get("config");
|
||||
if (stored.config) {
|
||||
config = { ...DEFAULT_CONFIG, ...stored.config };
|
||||
}
|
||||
console.log("Config loaded:", config);
|
||||
}
|
||||
|
||||
// Save config to storage
|
||||
async function saveConfig() {
|
||||
await chrome.storage.local.set({ config });
|
||||
setupAlarm(); // Reset alarm with new interval
|
||||
}
|
||||
|
||||
// Load known products from storage
|
||||
async function loadProducts() {
|
||||
const stored = await chrome.storage.local.get("knownProducts");
|
||||
if (stored.knownProducts) {
|
||||
knownProducts = stored.knownProducts;
|
||||
}
|
||||
console.log(`Loaded ${Object.keys(knownProducts).length} known products`);
|
||||
}
|
||||
|
||||
// Save known products to storage
|
||||
async function saveProducts() {
|
||||
await chrome.storage.local.set({ knownProducts });
|
||||
}
|
||||
|
||||
// Main stock check function
|
||||
async function runStockCheck() {
|
||||
if (!config.enabled) {
|
||||
console.log("Stock check disabled");
|
||||
return;
|
||||
}
|
||||
|
||||
console.log("Running stock check...");
|
||||
|
||||
for (const url of config.urls) {
|
||||
try {
|
||||
const products = await fetchAndParseProducts(url);
|
||||
const { newProducts, restockedProducts } = processProducts(products);
|
||||
|
||||
// Send notifications
|
||||
if (config.notifyNewProducts) {
|
||||
for (const product of newProducts) {
|
||||
if (product.inStock) {
|
||||
await sendDiscordNotification(product, "new_drop");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (config.notifyRestocks) {
|
||||
for (const product of restockedProducts) {
|
||||
await sendDiscordNotification(product, "restock");
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`Check complete: ${products.length} products, ${newProducts.length} new, ${restockedProducts.length} restocks`);
|
||||
|
||||
} catch (error) {
|
||||
console.error(`Error checking ${url}:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
await saveProducts();
|
||||
}
|
||||
|
||||
// Fetch and parse products from a URL using content script
|
||||
async function fetchAndParseProducts(url) {
|
||||
return new Promise(async (resolve, reject) => {
|
||||
try {
|
||||
// Create a tab to load the page
|
||||
const tab = await chrome.tabs.create({ url, active: false });
|
||||
|
||||
console.log(`Created tab ${tab.id} for ${url}`);
|
||||
|
||||
// Wait for tab to finish loading
|
||||
const waitForLoad = () => {
|
||||
return new Promise((res) => {
|
||||
const listener = (tabId, changeInfo) => {
|
||||
if (tabId === tab.id && changeInfo.status === "complete") {
|
||||
chrome.tabs.onUpdated.removeListener(listener);
|
||||
res();
|
||||
}
|
||||
};
|
||||
chrome.tabs.onUpdated.addListener(listener);
|
||||
|
||||
// Timeout after 30 seconds
|
||||
setTimeout(() => {
|
||||
chrome.tabs.onUpdated.removeListener(listener);
|
||||
res();
|
||||
}, 30000);
|
||||
});
|
||||
};
|
||||
|
||||
await waitForLoad();
|
||||
console.log(`Tab ${tab.id} loaded`);
|
||||
|
||||
// Give extra time for dynamic content to load
|
||||
console.log("Waiting for dynamic content...");
|
||||
await new Promise(r => setTimeout(r, 5000));
|
||||
|
||||
// Send message to content script to extract products
|
||||
let products = [];
|
||||
try {
|
||||
const response = await chrome.tabs.sendMessage(tab.id, { type: "extractProducts" });
|
||||
products = response || [];
|
||||
console.log(`Content script returned ${products.length} products`);
|
||||
} catch (e) {
|
||||
console.log("Content script not ready, injecting manually...");
|
||||
// Inject content script if not already there
|
||||
await chrome.scripting.executeScript({
|
||||
target: { tabId: tab.id },
|
||||
files: ["content.js"]
|
||||
});
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
const response = await chrome.tabs.sendMessage(tab.id, { type: "extractProducts" });
|
||||
products = response || [];
|
||||
}
|
||||
|
||||
// Apply keyword filter
|
||||
if (config.keywords && config.keywords.length > 0) {
|
||||
products = products.filter(p => {
|
||||
const nameLower = p.name.toLowerCase();
|
||||
return config.keywords.some(kw => nameLower.includes(kw.toLowerCase()));
|
||||
});
|
||||
console.log(`After keyword filter: ${products.length} products`);
|
||||
}
|
||||
|
||||
// Close the tab
|
||||
await chrome.tabs.remove(tab.id);
|
||||
console.log(`Closed tab ${tab.id}`);
|
||||
|
||||
resolve(products);
|
||||
|
||||
} catch (error) {
|
||||
reject(error);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Note: Product parsing is now done by the content script (content.js)
|
||||
|
||||
// Process products - detect new and restocked
|
||||
function processProducts(products) {
|
||||
const newProducts = [];
|
||||
const restockedProducts = [];
|
||||
const now = new Date().toISOString();
|
||||
|
||||
for (const product of products) {
|
||||
const existing = knownProducts[product.url];
|
||||
|
||||
if (!existing) {
|
||||
// New product
|
||||
newProducts.push(product);
|
||||
knownProducts[product.url] = {
|
||||
...product,
|
||||
firstSeen: now,
|
||||
lastSeen: now,
|
||||
lastInStock: product.inStock ? now : null
|
||||
};
|
||||
} else {
|
||||
// Existing product - check for restock
|
||||
if (product.inStock && !existing.inStock) {
|
||||
restockedProducts.push(product);
|
||||
existing.lastInStock = now;
|
||||
}
|
||||
|
||||
// Update
|
||||
existing.inStock = product.inStock;
|
||||
existing.price = product.price || existing.price;
|
||||
existing.lastSeen = now;
|
||||
}
|
||||
}
|
||||
|
||||
return { newProducts, restockedProducts };
|
||||
}
|
||||
|
||||
// Send Discord notification
|
||||
async function sendDiscordNotification(product, alertType) {
|
||||
if (!config.discordWebhook) {
|
||||
console.log("No Discord webhook configured");
|
||||
return;
|
||||
}
|
||||
|
||||
const color = alertType === "restock" ? 0x00FF00 : 0x0099FF;
|
||||
const title = alertType === "restock" ? "RESTOCK ALERT" : "NEW DROP";
|
||||
|
||||
const embed = {
|
||||
title: title,
|
||||
description: `**${product.name}**`,
|
||||
url: product.url,
|
||||
color: color,
|
||||
fields: [
|
||||
{ name: "Price", value: product.price || "See link", inline: true },
|
||||
{ name: "Store", value: "Pokemon Center", inline: true },
|
||||
{ name: "Link", value: `[BUY NOW](${product.url})`, inline: false }
|
||||
],
|
||||
footer: { text: "Pokemon Stock Monitor (Chrome Extension)" },
|
||||
timestamp: new Date().toISOString()
|
||||
};
|
||||
|
||||
if (product.imageUrl) {
|
||||
embed.thumbnail = { url: product.imageUrl };
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(config.discordWebhook, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
content: "@everyone",
|
||||
embeds: [embed]
|
||||
})
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
console.log(`Discord notification sent for: ${product.name}`);
|
||||
|
||||
// Also show browser notification
|
||||
chrome.notifications.create({
|
||||
type: "basic",
|
||||
iconUrl: "icon128.png",
|
||||
title: title,
|
||||
message: product.name
|
||||
});
|
||||
} else {
|
||||
console.error("Discord notification failed:", response.status);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Error sending Discord notification:", error);
|
||||
}
|
||||
}
|
||||
|
||||
// Listen for messages from popup
|
||||
chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
|
||||
if (message.type === "getConfig") {
|
||||
sendResponse(config);
|
||||
} else if (message.type === "saveConfig") {
|
||||
config = { ...config, ...message.config };
|
||||
saveConfig();
|
||||
sendResponse({ success: true });
|
||||
} else if (message.type === "runCheck") {
|
||||
runStockCheck();
|
||||
sendResponse({ success: true });
|
||||
} else if (message.type === "getStats") {
|
||||
sendResponse({
|
||||
totalProducts: Object.keys(knownProducts).length,
|
||||
enabled: config.enabled
|
||||
});
|
||||
} else if (message.type === "clearProducts") {
|
||||
knownProducts = {};
|
||||
saveProducts();
|
||||
sendResponse({ success: true });
|
||||
}
|
||||
return true;
|
||||
});
|
||||
|
||||
// Run initial check after a short delay
|
||||
setTimeout(() => {
|
||||
loadConfig().then(() => {
|
||||
loadProducts().then(() => {
|
||||
if (config.enabled && config.discordWebhook) {
|
||||
runStockCheck();
|
||||
}
|
||||
});
|
||||
});
|
||||
}, 5000);
|
||||
@@ -0,0 +1,109 @@
|
||||
// Pokemon Stock Monitor - Content Script
|
||||
// Runs on PokemonCenter pages to extract product data
|
||||
|
||||
(function() {
|
||||
console.log("[Pokemon Monitor] Content script loaded on:", window.location.href);
|
||||
|
||||
// Listen for messages from background script
|
||||
chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
|
||||
if (message.type === "extractProducts") {
|
||||
console.log("[Pokemon Monitor] Extracting products...");
|
||||
const products = extractProducts();
|
||||
console.log("[Pokemon Monitor] Found", products.length, "products");
|
||||
sendResponse(products);
|
||||
}
|
||||
return true;
|
||||
});
|
||||
|
||||
function extractProducts() {
|
||||
const products = [];
|
||||
|
||||
// Group all product links by URL
|
||||
const urlToLinks = {};
|
||||
document.querySelectorAll('a[href*="/product/"]').forEach(link => {
|
||||
const url = link.href;
|
||||
if (!urlToLinks[url]) {
|
||||
urlToLinks[url] = [];
|
||||
}
|
||||
urlToLinks[url].push(link);
|
||||
});
|
||||
|
||||
console.log("[Pokemon Monitor] Found", Object.keys(urlToLinks).length, "unique product URLs");
|
||||
|
||||
// Process each unique product URL
|
||||
for (const [url, links] of Object.entries(urlToLinks)) {
|
||||
// Find the best name from all links to this product
|
||||
let bestName = "";
|
||||
let isSoldOut = false;
|
||||
|
||||
for (const link of links) {
|
||||
const text = link.textContent.trim();
|
||||
|
||||
// Check if any link says "SOLD OUT"
|
||||
if (text.toUpperCase().includes("SOLD OUT")) {
|
||||
isSoldOut = true;
|
||||
}
|
||||
|
||||
// Pick the longest non-"SOLD OUT" text as the name
|
||||
if (text.length > bestName.length &&
|
||||
!text.toUpperCase().startsWith("SOLD") &&
|
||||
text.length > 10) {
|
||||
bestName = text;
|
||||
}
|
||||
}
|
||||
|
||||
// Skip if we couldn't find a good name
|
||||
if (!bestName || bestName.length < 10) continue;
|
||||
|
||||
// Clean up name - remove price if embedded
|
||||
bestName = bestName.replace(/\$[\d,.]+/g, "").trim();
|
||||
// Remove "Add to Cart" etc
|
||||
bestName = bestName.replace(/Add to Cart/gi, "").trim();
|
||||
// Clean whitespace
|
||||
bestName = bestName.replace(/\s+/g, " ").trim();
|
||||
|
||||
// Find price from any of the links' containers
|
||||
let price = null;
|
||||
for (const link of links) {
|
||||
let parent = link.parentElement;
|
||||
for (let i = 0; i < 6 && parent && !price; i++) {
|
||||
const priceMatch = parent.textContent.match(/\$[\d,]+\.?\d*/);
|
||||
if (priceMatch) {
|
||||
price = priceMatch[0];
|
||||
break;
|
||||
}
|
||||
parent = parent.parentElement;
|
||||
}
|
||||
if (price) break;
|
||||
}
|
||||
|
||||
// Find image from any of the links
|
||||
let imageUrl = null;
|
||||
for (const link of links) {
|
||||
const img = link.querySelector("img") ||
|
||||
link.closest("[class*='product']")?.querySelector("img");
|
||||
if (img) {
|
||||
imageUrl = img.src || img.dataset.src;
|
||||
if (imageUrl && !imageUrl.startsWith("data:")) break;
|
||||
}
|
||||
}
|
||||
|
||||
// Extract product ID from URL
|
||||
const idMatch = url.match(/\/product\/([^\/]+)/);
|
||||
const productId = idMatch ? idMatch[1] : url;
|
||||
|
||||
products.push({
|
||||
name: bestName,
|
||||
url: url,
|
||||
price: price,
|
||||
inStock: !isSoldOut,
|
||||
imageUrl: imageUrl,
|
||||
productId: productId,
|
||||
site: "pokemoncenter"
|
||||
});
|
||||
}
|
||||
|
||||
console.log("[Pokemon Monitor] Extracted products:", products.map(p => ({name: p.name.slice(0,40), inStock: p.inStock})));
|
||||
return products;
|
||||
}
|
||||
})();
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"manifest_version": 3,
|
||||
"name": "Pokemon Stock Monitor",
|
||||
"version": "1.0.1",
|
||||
"description": "Monitors PokemonCenter for restocks and new drops, sends Discord notifications",
|
||||
"permissions": [
|
||||
"alarms",
|
||||
"storage",
|
||||
"notifications",
|
||||
"tabs",
|
||||
"scripting"
|
||||
],
|
||||
"host_permissions": [
|
||||
"https://www.pokemoncenter.com/*",
|
||||
"https://discord.com/api/*"
|
||||
],
|
||||
"background": {
|
||||
"service_worker": "background.js"
|
||||
},
|
||||
"action": {
|
||||
"default_popup": "popup.html"
|
||||
},
|
||||
"content_scripts": [
|
||||
{
|
||||
"matches": ["https://www.pokemoncenter.com/*"],
|
||||
"js": ["content.js"],
|
||||
"run_at": "document_idle"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,254 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<style>
|
||||
body {
|
||||
width: 320px;
|
||||
padding: 15px;
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
background: #1a1a2e;
|
||||
color: #eee;
|
||||
margin: 0;
|
||||
}
|
||||
h1 {
|
||||
font-size: 16px;
|
||||
margin: 0 0 15px 0;
|
||||
color: #ffcb05;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
}
|
||||
h1 span {
|
||||
font-size: 20px;
|
||||
}
|
||||
.section {
|
||||
margin-bottom: 15px;
|
||||
}
|
||||
label {
|
||||
display: block;
|
||||
font-size: 12px;
|
||||
color: #aaa;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
input[type="text"], input[type="number"], textarea {
|
||||
width: 100%;
|
||||
padding: 8px;
|
||||
border: 1px solid #333;
|
||||
border-radius: 4px;
|
||||
background: #16213e;
|
||||
color: #eee;
|
||||
font-size: 13px;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
input:focus, textarea:focus {
|
||||
outline: none;
|
||||
border-color: #ffcb05;
|
||||
}
|
||||
textarea {
|
||||
height: 60px;
|
||||
resize: vertical;
|
||||
}
|
||||
.row {
|
||||
display: flex;
|
||||
gap: 10px;
|
||||
}
|
||||
.row > div {
|
||||
flex: 1;
|
||||
}
|
||||
.toggle-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 8px 0;
|
||||
}
|
||||
.toggle-row label {
|
||||
margin: 0;
|
||||
color: #eee;
|
||||
font-size: 13px;
|
||||
}
|
||||
.toggle {
|
||||
position: relative;
|
||||
width: 44px;
|
||||
height: 24px;
|
||||
}
|
||||
.toggle input {
|
||||
opacity: 0;
|
||||
width: 0;
|
||||
height: 0;
|
||||
}
|
||||
.slider {
|
||||
position: absolute;
|
||||
cursor: pointer;
|
||||
top: 0;
|
||||
left: 0;
|
||||
right: 0;
|
||||
bottom: 0;
|
||||
background-color: #333;
|
||||
transition: .3s;
|
||||
border-radius: 24px;
|
||||
}
|
||||
.slider:before {
|
||||
position: absolute;
|
||||
content: "";
|
||||
height: 18px;
|
||||
width: 18px;
|
||||
left: 3px;
|
||||
bottom: 3px;
|
||||
background-color: #eee;
|
||||
transition: .3s;
|
||||
border-radius: 50%;
|
||||
}
|
||||
input:checked + .slider {
|
||||
background-color: #00c853;
|
||||
}
|
||||
input:checked + .slider:before {
|
||||
transform: translateX(20px);
|
||||
}
|
||||
button {
|
||||
width: 100%;
|
||||
padding: 10px;
|
||||
border: none;
|
||||
border-radius: 4px;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
cursor: pointer;
|
||||
transition: background 0.2s;
|
||||
}
|
||||
.btn-primary {
|
||||
background: #ffcb05;
|
||||
color: #1a1a2e;
|
||||
}
|
||||
.btn-primary:hover {
|
||||
background: #ffd633;
|
||||
}
|
||||
.btn-secondary {
|
||||
background: #333;
|
||||
color: #eee;
|
||||
margin-top: 8px;
|
||||
}
|
||||
.btn-secondary:hover {
|
||||
background: #444;
|
||||
}
|
||||
.btn-danger {
|
||||
background: #c62828;
|
||||
color: #fff;
|
||||
margin-top: 8px;
|
||||
}
|
||||
.btn-danger:hover {
|
||||
background: #d32f2f;
|
||||
}
|
||||
.stats {
|
||||
background: #16213e;
|
||||
padding: 10px;
|
||||
border-radius: 4px;
|
||||
font-size: 12px;
|
||||
margin-bottom: 15px;
|
||||
}
|
||||
.stats-row {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.stats-row:last-child {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
.status-dot {
|
||||
display: inline-block;
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
border-radius: 50%;
|
||||
margin-right: 6px;
|
||||
}
|
||||
.status-dot.active {
|
||||
background: #00c853;
|
||||
}
|
||||
.status-dot.inactive {
|
||||
background: #c62828;
|
||||
}
|
||||
.saved-msg {
|
||||
text-align: center;
|
||||
color: #00c853;
|
||||
font-size: 12px;
|
||||
margin-top: 8px;
|
||||
opacity: 0;
|
||||
transition: opacity 0.3s;
|
||||
}
|
||||
.saved-msg.show {
|
||||
opacity: 1;
|
||||
}
|
||||
small {
|
||||
color: #666;
|
||||
font-size: 11px;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1><span>⚡</span> Pokemon Stock Monitor</h1>
|
||||
|
||||
<div class="stats" id="stats">
|
||||
<div class="stats-row">
|
||||
<span>Status:</span>
|
||||
<span><span class="status-dot active" id="statusDot"></span><span id="statusText">Active</span></span>
|
||||
</div>
|
||||
<div class="stats-row">
|
||||
<span>Products tracked:</span>
|
||||
<span id="productCount">0</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<label>Discord Webhook URL</label>
|
||||
<input type="text" id="webhook" placeholder="https://discord.com/api/webhooks/...">
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<label>URLs to Monitor (one per line)</label>
|
||||
<textarea id="urls" placeholder="https://www.pokemoncenter.com/category/tcg-cards"></textarea>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<label>Keywords (comma separated, leave empty for all)</label>
|
||||
<input type="text" id="keywords" placeholder="chaos rising, booster, etb">
|
||||
</div>
|
||||
|
||||
<div class="row">
|
||||
<div>
|
||||
<label>Check Interval (min)</label>
|
||||
<input type="number" id="interval" min="1" max="60" value="1">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<div class="toggle-row">
|
||||
<label>Monitor Enabled</label>
|
||||
<div class="toggle">
|
||||
<input type="checkbox" id="enabled" checked>
|
||||
<span class="slider"></span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="toggle-row">
|
||||
<label>Notify New Products</label>
|
||||
<div class="toggle">
|
||||
<input type="checkbox" id="notifyNew" checked>
|
||||
<span class="slider"></span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="toggle-row">
|
||||
<label>Notify Restocks</label>
|
||||
<div class="toggle">
|
||||
<input type="checkbox" id="notifyRestock" checked>
|
||||
<span class="slider"></span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<button class="btn-primary" id="saveBtn">Save Settings</button>
|
||||
<button class="btn-secondary" id="checkNowBtn">Check Now</button>
|
||||
<button class="btn-danger" id="clearBtn">Clear Product History</button>
|
||||
|
||||
<div class="saved-msg" id="savedMsg">Settings saved!</div>
|
||||
|
||||
<script src="popup.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,75 @@
|
||||
// Pokemon Stock Monitor - Popup Script
|
||||
|
||||
document.addEventListener("DOMContentLoaded", async () => {
|
||||
// Load current config
|
||||
const config = await chrome.runtime.sendMessage({ type: "getConfig" });
|
||||
const stats = await chrome.runtime.sendMessage({ type: "getStats" });
|
||||
|
||||
// Populate form
|
||||
document.getElementById("webhook").value = config.discordWebhook || "";
|
||||
document.getElementById("urls").value = (config.urls || []).join("\n");
|
||||
document.getElementById("keywords").value = (config.keywords || []).join(", ");
|
||||
document.getElementById("interval").value = config.checkIntervalMinutes || 1;
|
||||
document.getElementById("enabled").checked = config.enabled !== false;
|
||||
document.getElementById("notifyNew").checked = config.notifyNewProducts !== false;
|
||||
document.getElementById("notifyRestock").checked = config.notifyRestocks !== false;
|
||||
|
||||
// Update stats
|
||||
document.getElementById("productCount").textContent = stats.totalProducts || 0;
|
||||
updateStatus(stats.enabled !== false);
|
||||
|
||||
// Save button
|
||||
document.getElementById("saveBtn").addEventListener("click", async () => {
|
||||
const urlsText = document.getElementById("urls").value;
|
||||
const keywordsText = document.getElementById("keywords").value;
|
||||
|
||||
const newConfig = {
|
||||
discordWebhook: document.getElementById("webhook").value.trim(),
|
||||
urls: urlsText.split("\n").map(u => u.trim()).filter(u => u),
|
||||
keywords: keywordsText.split(",").map(k => k.trim()).filter(k => k),
|
||||
checkIntervalMinutes: parseInt(document.getElementById("interval").value) || 1,
|
||||
enabled: document.getElementById("enabled").checked,
|
||||
notifyNewProducts: document.getElementById("notifyNew").checked,
|
||||
notifyRestocks: document.getElementById("notifyRestock").checked
|
||||
};
|
||||
|
||||
await chrome.runtime.sendMessage({ type: "saveConfig", config: newConfig });
|
||||
updateStatus(newConfig.enabled);
|
||||
showSaved();
|
||||
});
|
||||
|
||||
// Check Now button
|
||||
document.getElementById("checkNowBtn").addEventListener("click", async () => {
|
||||
await chrome.runtime.sendMessage({ type: "runCheck" });
|
||||
showSaved("Check started!");
|
||||
});
|
||||
|
||||
// Clear button
|
||||
document.getElementById("clearBtn").addEventListener("click", async () => {
|
||||
if (confirm("Clear all tracked products? This will treat all products as new on next check.")) {
|
||||
await chrome.runtime.sendMessage({ type: "clearProducts" });
|
||||
document.getElementById("productCount").textContent = "0";
|
||||
showSaved("History cleared!");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
function updateStatus(enabled) {
|
||||
const dot = document.getElementById("statusDot");
|
||||
const text = document.getElementById("statusText");
|
||||
|
||||
if (enabled) {
|
||||
dot.className = "status-dot active";
|
||||
text.textContent = "Active";
|
||||
} else {
|
||||
dot.className = "status-dot inactive";
|
||||
text.textContent = "Disabled";
|
||||
}
|
||||
}
|
||||
|
||||
function showSaved(msg = "Settings saved!") {
|
||||
const el = document.getElementById("savedMsg");
|
||||
el.textContent = msg;
|
||||
el.classList.add("show");
|
||||
setTimeout(() => el.classList.remove("show"), 2000);
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
"""
|
||||
Configuration for Pokemon Stock Monitor
|
||||
Update DISCORD_WEBHOOK_URL with your actual webhook URL
|
||||
"""
|
||||
|
||||
# Discord webhook URL - GET THIS FROM YOUR DISCORD SERVER
|
||||
# Server Settings -> Integrations -> Webhooks -> New Webhook -> Copy Webhook URL
|
||||
DISCORD_WEBHOOK_URL = "YOUR_WEBHOOK_URL_HERE"
|
||||
|
||||
# How often to check each site (in seconds)
|
||||
CHECK_INTERVAL_SECONDS = 60
|
||||
|
||||
# Which sites to monitor
|
||||
# Note: PokemonCenter has strong bot protection (Imperva) - may need manual workarounds
|
||||
SITES_ENABLED = ["target"] # Options: "pokemoncenter", "target", "walmart", "bestbuy"
|
||||
|
||||
# PokemonCenter URLs to monitor
|
||||
POKEMON_CENTER_URLS = [
|
||||
"https://www.pokemoncenter.com/category/tcg-cards?sort=relevance",
|
||||
]
|
||||
|
||||
# Target URLs to monitor
|
||||
TARGET_URLS = [
|
||||
"https://www.target.com/s?searchTerm=pokemon+tcg",
|
||||
]
|
||||
|
||||
# Keyword filtering (disabled by default)
|
||||
KEYWORD_FILTER_ENABLED = False
|
||||
KEYWORDS = [
|
||||
"chaos rising",
|
||||
"booster",
|
||||
"etb",
|
||||
"elite trainer",
|
||||
"booster bundle",
|
||||
]
|
||||
|
||||
# Browser settings
|
||||
HEADLESS = False # Set to False to see the browser (useful for debugging)
|
||||
SLOW_MO = 0 # Milliseconds to slow down browser actions (for debugging)
|
||||
|
||||
# Chrome profile for PokemonCenter (optional - helps bypass bot detection)
|
||||
# Set to your Chrome user data directory to use your real browser profile
|
||||
# Windows: C:\Users\USERNAME\AppData\Local\Google\Chrome\User Data
|
||||
# Linux: ~/.config/google-chrome
|
||||
# Mac: ~/Library/Application Support/Google/Chrome
|
||||
# Leave as None to use a fresh browser profile
|
||||
# Set to None to use a fresh profile, or a path to use an existing profile
|
||||
# Note: Chrome must be closed when using an existing profile
|
||||
CHROME_USER_DATA_DIR = None
|
||||
|
||||
# Use real Chrome instead of Playwright's Chromium (helps with bot detection)
|
||||
USE_REAL_CHROME = True
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL = "INFO"
|
||||
@@ -0,0 +1,183 @@
|
||||
"""
|
||||
Discord webhook notifications for stock alerts
|
||||
Sends rich embeds with product info and direct links
|
||||
"""
|
||||
|
||||
import logging
|
||||
import requests
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
from config import DISCORD_WEBHOOK_URL
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Colors for different notification types
|
||||
COLOR_RESTOCK = 0x00FF00 # Green - item back in stock
|
||||
COLOR_NEW_DROP = 0x0099FF # Blue - new product listing
|
||||
COLOR_PREORDER = 0xFFAA00 # Orange - pre-order available
|
||||
COLOR_ERROR = 0xFF0000 # Red - error notification
|
||||
|
||||
# Site icons/emojis
|
||||
SITE_EMOJIS = {
|
||||
"pokemoncenter": "\U0001F7E1", # Yellow circle
|
||||
"target": "\U0001F534", # Red circle
|
||||
"walmart": "\U0001F535", # Blue circle
|
||||
"bestbuy": "\U0001F7E1", # Yellow circle
|
||||
}
|
||||
|
||||
|
||||
def send_stock_alert(
|
||||
product_name: str,
|
||||
product_url: str,
|
||||
price: str,
|
||||
site: str,
|
||||
alert_type: str = "restock",
|
||||
image_url: Optional[str] = None,
|
||||
):
|
||||
"""
|
||||
Send a Discord notification for a stock alert
|
||||
|
||||
Args:
|
||||
product_name: Name of the product
|
||||
product_url: Direct link to the product
|
||||
price: Price string (e.g., "$49.99")
|
||||
site: Site name (pokemoncenter, target, etc.)
|
||||
alert_type: "restock", "new_drop", or "preorder"
|
||||
image_url: Optional product image URL
|
||||
"""
|
||||
if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
|
||||
logger.error("Discord webhook URL not configured! Update config.py")
|
||||
return False
|
||||
|
||||
# Choose color based on alert type
|
||||
if alert_type == "restock":
|
||||
color = COLOR_RESTOCK
|
||||
title = f"\U0001F6A8 RESTOCK ALERT"
|
||||
elif alert_type == "new_drop":
|
||||
color = COLOR_NEW_DROP
|
||||
title = f"\U0001F195 NEW DROP"
|
||||
elif alert_type == "preorder":
|
||||
color = COLOR_PREORDER
|
||||
title = f"\u23F0 PRE-ORDER AVAILABLE"
|
||||
else:
|
||||
color = COLOR_RESTOCK
|
||||
title = f"\U0001F514 STOCK ALERT"
|
||||
|
||||
site_emoji = SITE_EMOJIS.get(site.lower(), "\U0001F6D2")
|
||||
site_display = site.replace("pokemoncenter", "Pokemon Center").title()
|
||||
|
||||
# Build the embed
|
||||
embed = {
|
||||
"title": title,
|
||||
"description": f"**{product_name}**",
|
||||
"url": product_url,
|
||||
"color": color,
|
||||
"fields": [
|
||||
{"name": "Price", "value": price or "See link", "inline": True},
|
||||
{"name": "Store", "value": f"{site_emoji} {site_display}", "inline": True},
|
||||
{"name": "Link", "value": f"[\U0001F6D2 BUY NOW]({product_url})", "inline": False},
|
||||
],
|
||||
"footer": {"text": "Pokemon Stock Monitor"},
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
}
|
||||
|
||||
if image_url:
|
||||
embed["thumbnail"] = {"url": image_url}
|
||||
|
||||
payload = {
|
||||
"content": "@everyone", # Ping everyone
|
||||
"embeds": [embed],
|
||||
}
|
||||
|
||||
try:
|
||||
response = requests.post(
|
||||
DISCORD_WEBHOOK_URL,
|
||||
json=payload,
|
||||
timeout=10,
|
||||
)
|
||||
response.raise_for_status()
|
||||
logger.info(f"Discord notification sent for: {product_name}")
|
||||
return True
|
||||
except requests.exceptions.RequestException as e:
|
||||
logger.error(f"Failed to send Discord notification: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def send_error_notification(error_message: str, site: str = "Unknown"):
|
||||
"""Send an error notification to Discord"""
|
||||
if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
|
||||
return False
|
||||
|
||||
embed = {
|
||||
"title": "\u26A0\uFE0F Monitor Error",
|
||||
"description": error_message,
|
||||
"color": COLOR_ERROR,
|
||||
"fields": [{"name": "Site", "value": site, "inline": True}],
|
||||
"footer": {"text": "Pokemon Stock Monitor"},
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
}
|
||||
|
||||
payload = {"embeds": [embed]}
|
||||
|
||||
try:
|
||||
response = requests.post(DISCORD_WEBHOOK_URL, json=payload, timeout=10)
|
||||
response.raise_for_status()
|
||||
return True
|
||||
except requests.exceptions.RequestException as e:
|
||||
logger.error(f"Failed to send error notification: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def send_startup_notification():
|
||||
"""Send a notification that the monitor has started"""
|
||||
if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
|
||||
return False
|
||||
|
||||
embed = {
|
||||
"title": "\u2705 Monitor Started",
|
||||
"description": "Pokemon Stock Monitor is now running and watching for restocks!",
|
||||
"color": 0x00FF00,
|
||||
"footer": {"text": "Pokemon Stock Monitor"},
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
}
|
||||
|
||||
payload = {"embeds": [embed]}
|
||||
|
||||
try:
|
||||
response = requests.post(DISCORD_WEBHOOK_URL, json=payload, timeout=10)
|
||||
response.raise_for_status()
|
||||
return True
|
||||
except requests.exceptions.RequestException as e:
|
||||
logger.error(f"Failed to send startup notification: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def test_webhook():
|
||||
"""Test the Discord webhook connection"""
|
||||
print("Testing Discord webhook...")
|
||||
|
||||
if DISCORD_WEBHOOK_URL == "YOUR_WEBHOOK_URL_HERE":
|
||||
print("ERROR: Webhook URL not configured!")
|
||||
print("Edit config.py and set DISCORD_WEBHOOK_URL")
|
||||
return False
|
||||
|
||||
# Send a test notification
|
||||
success = send_stock_alert(
|
||||
product_name="Test Product - Pokemon TCG Booster",
|
||||
product_url="https://www.pokemoncenter.com/test",
|
||||
price="$4.99",
|
||||
site="pokemoncenter",
|
||||
alert_type="restock",
|
||||
)
|
||||
|
||||
if success:
|
||||
print("SUCCESS! Check your Discord channel for the test message.")
|
||||
else:
|
||||
print("FAILED! Check the webhook URL and try again.")
|
||||
|
||||
return success
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Run webhook test
|
||||
test_webhook()
|
||||
@@ -0,0 +1,232 @@
|
||||
"""
|
||||
Pokemon Stock Monitor - Main entry point
|
||||
Monitors retail sites for Pokemon card restocks and new drops
|
||||
"""
|
||||
|
||||
import logging
|
||||
import time
|
||||
import signal
|
||||
import sys
|
||||
from datetime import datetime
|
||||
|
||||
import schedule
|
||||
|
||||
from config import (
|
||||
CHECK_INTERVAL_SECONDS,
|
||||
SITES_ENABLED,
|
||||
POKEMON_CENTER_URLS,
|
||||
TARGET_URLS,
|
||||
KEYWORD_FILTER_ENABLED,
|
||||
KEYWORDS,
|
||||
LOG_LEVEL,
|
||||
)
|
||||
from browser import get_browser, shutdown_browser
|
||||
from product_tracker import ProductTracker
|
||||
from discord_notifier import send_stock_alert, send_startup_notification, send_error_notification
|
||||
from scrapers import PokemonCenterScraper, TargetScraper
|
||||
|
||||
# Setup logging
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, LOG_LEVEL),
|
||||
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
|
||||
handlers=[
|
||||
logging.StreamHandler(),
|
||||
logging.FileHandler("monitor.log", encoding="utf-8"),
|
||||
],
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Global tracker
|
||||
tracker = ProductTracker()
|
||||
|
||||
# Scrapers
|
||||
scrapers = {
|
||||
"pokemoncenter": PokemonCenterScraper(),
|
||||
"target": TargetScraper(),
|
||||
}
|
||||
|
||||
|
||||
def check_pokemoncenter():
|
||||
"""Check PokemonCenter for restocks and new drops"""
|
||||
logger.info("Checking PokemonCenter...")
|
||||
scraper = scrapers["pokemoncenter"]
|
||||
|
||||
try:
|
||||
for url in POKEMON_CENTER_URLS:
|
||||
logger.info(f"Scraping: {url}")
|
||||
products = scraper.scrape_category_page(url)
|
||||
|
||||
# Apply keyword filter if enabled
|
||||
if KEYWORD_FILTER_ENABLED and KEYWORDS:
|
||||
products = scraper.filter_by_keywords(products, KEYWORDS)
|
||||
logger.info(f"After keyword filter: {len(products)} products")
|
||||
|
||||
# Process and detect changes
|
||||
new_products, restocked_products = tracker.process_products(products)
|
||||
|
||||
# Send notifications for new products
|
||||
for product in new_products:
|
||||
if product.in_stock:
|
||||
send_stock_alert(
|
||||
product_name=product.name,
|
||||
product_url=product.url,
|
||||
price=product.price or "See link",
|
||||
site="pokemoncenter",
|
||||
alert_type="new_drop",
|
||||
image_url=product.image_url,
|
||||
)
|
||||
logger.info(f"Sent notification for new product: {product.name}")
|
||||
|
||||
# Send notifications for restocks
|
||||
for product in restocked_products:
|
||||
send_stock_alert(
|
||||
product_name=product.name,
|
||||
product_url=product.url,
|
||||
price=product.price or "See link",
|
||||
site="pokemoncenter",
|
||||
alert_type="restock",
|
||||
image_url=product.image_url,
|
||||
)
|
||||
logger.info(f"Sent notification for restock: {product.name}")
|
||||
|
||||
# Log stats
|
||||
stats = tracker.get_stats()
|
||||
logger.info(
|
||||
f"Stats: {stats['total_products']} total, "
|
||||
f"{stats['in_stock']} in stock, "
|
||||
f"{stats['out_of_stock']} out of stock"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking PokemonCenter: {e}")
|
||||
send_error_notification(f"Error checking PokemonCenter: {str(e)}", "PokemonCenter")
|
||||
|
||||
|
||||
def check_target():
|
||||
"""Check Target for restocks and new drops"""
|
||||
logger.info("Checking Target...")
|
||||
scraper = scrapers["target"]
|
||||
|
||||
try:
|
||||
for url in TARGET_URLS:
|
||||
logger.info(f"Scraping: {url}")
|
||||
products = scraper.scrape_category_page(url)
|
||||
|
||||
# Apply keyword filter if enabled
|
||||
if KEYWORD_FILTER_ENABLED and KEYWORDS:
|
||||
products = scraper.filter_by_keywords(products, KEYWORDS)
|
||||
logger.info(f"After keyword filter: {len(products)} products")
|
||||
|
||||
# Process and detect changes
|
||||
new_products, restocked_products = tracker.process_products(products)
|
||||
|
||||
# Send notifications for new products
|
||||
for product in new_products:
|
||||
if product.in_stock:
|
||||
send_stock_alert(
|
||||
product_name=product.name,
|
||||
product_url=product.url,
|
||||
price=product.price or "See link",
|
||||
site="target",
|
||||
alert_type="new_drop",
|
||||
image_url=product.image_url,
|
||||
)
|
||||
logger.info(f"Sent notification for new product: {product.name}")
|
||||
|
||||
# Send notifications for restocks
|
||||
for product in restocked_products:
|
||||
send_stock_alert(
|
||||
product_name=product.name,
|
||||
product_url=product.url,
|
||||
price=product.price or "See link",
|
||||
site="target",
|
||||
alert_type="restock",
|
||||
image_url=product.image_url,
|
||||
)
|
||||
logger.info(f"Sent notification for restock: {product.name}")
|
||||
|
||||
# Log stats
|
||||
stats = tracker.get_stats()
|
||||
logger.info(
|
||||
f"Stats: {stats['total_products']} total, "
|
||||
f"{stats['in_stock']} in stock, "
|
||||
f"{stats['out_of_stock']} out of stock"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking Target: {e}")
|
||||
send_error_notification(f"Error checking Target: {str(e)}", "Target")
|
||||
|
||||
|
||||
def run_checks():
|
||||
"""Run all enabled site checks"""
|
||||
logger.info(f"Running checks at {datetime.now().strftime('%H:%M:%S')}")
|
||||
|
||||
if "pokemoncenter" in SITES_ENABLED:
|
||||
check_pokemoncenter()
|
||||
|
||||
if "target" in SITES_ENABLED:
|
||||
check_target()
|
||||
|
||||
logger.info("Check cycle complete")
|
||||
|
||||
|
||||
def graceful_shutdown(signum, frame):
|
||||
"""Handle shutdown gracefully"""
|
||||
logger.info("Shutting down...")
|
||||
shutdown_browser()
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point"""
|
||||
print("=" * 50)
|
||||
print("Pokemon Stock Monitor")
|
||||
print("=" * 50)
|
||||
print(f"Monitoring: {', '.join(SITES_ENABLED)}")
|
||||
print(f"Check interval: {CHECK_INTERVAL_SECONDS} seconds")
|
||||
print(f"Keyword filter: {'ON' if KEYWORD_FILTER_ENABLED else 'OFF'}")
|
||||
print("=" * 50)
|
||||
print()
|
||||
|
||||
# Register signal handlers
|
||||
signal.signal(signal.SIGINT, graceful_shutdown)
|
||||
signal.signal(signal.SIGTERM, graceful_shutdown)
|
||||
|
||||
# Initialize browser
|
||||
logger.info("Initializing browser...")
|
||||
try:
|
||||
get_browser()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to initialize browser: {e}")
|
||||
print(f"\nERROR: Failed to start browser. Make sure Playwright is installed:")
|
||||
print(" pip install playwright")
|
||||
print(" playwright install chromium")
|
||||
return
|
||||
|
||||
# Send startup notification
|
||||
send_startup_notification()
|
||||
|
||||
# Run initial check
|
||||
print("Running initial check...")
|
||||
run_checks()
|
||||
|
||||
# Schedule periodic checks
|
||||
schedule.every(CHECK_INTERVAL_SECONDS).seconds.do(run_checks)
|
||||
|
||||
print(f"\nMonitor running! Checking every {CHECK_INTERVAL_SECONDS} seconds.")
|
||||
print("Press Ctrl+C to stop.\n")
|
||||
|
||||
# Main loop
|
||||
try:
|
||||
while True:
|
||||
schedule.run_pending()
|
||||
time.sleep(1)
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
graceful_shutdown(None, None)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,136 @@
|
||||
"""
|
||||
Product tracker - keeps track of known products to detect new drops and restocks
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Set, Optional
|
||||
from dataclasses import asdict
|
||||
from datetime import datetime
|
||||
|
||||
from scrapers.base import Product
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# File to store known products
|
||||
PRODUCTS_FILE = Path(__file__).parent / "products.json"
|
||||
|
||||
|
||||
class ProductTracker:
|
||||
"""Tracks known products to detect new listings and stock changes"""
|
||||
|
||||
def __init__(self, products_file: Path = PRODUCTS_FILE):
|
||||
self.products_file = products_file
|
||||
self.products: Dict[str, dict] = {} # URL -> product data
|
||||
self.load()
|
||||
|
||||
def load(self):
|
||||
"""Load products from file"""
|
||||
if self.products_file.exists():
|
||||
try:
|
||||
with open(self.products_file, "r", encoding="utf-8") as f:
|
||||
self.products = json.load(f)
|
||||
logger.info(f"Loaded {len(self.products)} tracked products")
|
||||
except (json.JSONDecodeError, IOError) as e:
|
||||
logger.error(f"Error loading products file: {e}")
|
||||
self.products = {}
|
||||
else:
|
||||
self.products = {}
|
||||
logger.info("No existing products file, starting fresh")
|
||||
|
||||
def save(self):
|
||||
"""Save products to file"""
|
||||
try:
|
||||
with open(self.products_file, "w", encoding="utf-8") as f:
|
||||
json.dump(self.products, f, indent=2, ensure_ascii=False)
|
||||
logger.debug(f"Saved {len(self.products)} products")
|
||||
except IOError as e:
|
||||
logger.error(f"Error saving products file: {e}")
|
||||
|
||||
def process_products(self, products: List[Product]) -> tuple[List[Product], List[Product]]:
|
||||
"""
|
||||
Process a list of scraped products and detect changes
|
||||
|
||||
Args:
|
||||
products: List of products from scraper
|
||||
|
||||
Returns:
|
||||
Tuple of (new_products, restocked_products)
|
||||
"""
|
||||
new_products = []
|
||||
restocked_products = []
|
||||
|
||||
for product in products:
|
||||
url = product.url
|
||||
|
||||
if url not in self.products:
|
||||
# New product!
|
||||
new_products.append(product)
|
||||
self.products[url] = {
|
||||
"name": product.name,
|
||||
"url": url,
|
||||
"price": product.price,
|
||||
"in_stock": product.in_stock,
|
||||
"image_url": product.image_url,
|
||||
"site": product.site,
|
||||
"product_id": product.product_id,
|
||||
"first_seen": datetime.now().isoformat(),
|
||||
"last_seen": datetime.now().isoformat(),
|
||||
"last_in_stock": datetime.now().isoformat() if product.in_stock else None,
|
||||
}
|
||||
logger.info(f"NEW PRODUCT: {product.name}")
|
||||
|
||||
else:
|
||||
# Existing product - check for restock
|
||||
existing = self.products[url]
|
||||
was_in_stock = existing.get("in_stock", False)
|
||||
|
||||
# Update last seen
|
||||
existing["last_seen"] = datetime.now().isoformat()
|
||||
existing["price"] = product.price or existing.get("price")
|
||||
existing["image_url"] = product.image_url or existing.get("image_url")
|
||||
|
||||
if product.in_stock and not was_in_stock:
|
||||
# RESTOCK!
|
||||
restocked_products.append(product)
|
||||
existing["last_in_stock"] = datetime.now().isoformat()
|
||||
logger.info(f"RESTOCK: {product.name}")
|
||||
|
||||
existing["in_stock"] = product.in_stock
|
||||
self.products[url] = existing
|
||||
|
||||
self.save()
|
||||
return new_products, restocked_products
|
||||
|
||||
def get_known_urls(self) -> Set[str]:
|
||||
"""Get all known product URLs"""
|
||||
return set(self.products.keys())
|
||||
|
||||
def get_product(self, url: str) -> Optional[dict]:
|
||||
"""Get a specific product by URL"""
|
||||
return self.products.get(url)
|
||||
|
||||
def mark_out_of_stock(self, url: str):
|
||||
"""Mark a product as out of stock"""
|
||||
if url in self.products:
|
||||
self.products[url]["in_stock"] = False
|
||||
self.save()
|
||||
|
||||
def clear(self):
|
||||
"""Clear all tracked products"""
|
||||
self.products = {}
|
||||
self.save()
|
||||
logger.info("Cleared all tracked products")
|
||||
|
||||
def get_stats(self) -> dict:
|
||||
"""Get tracking statistics"""
|
||||
total = len(self.products)
|
||||
in_stock = sum(1 for p in self.products.values() if p.get("in_stock", False))
|
||||
out_of_stock = total - in_stock
|
||||
|
||||
return {
|
||||
"total_products": total,
|
||||
"in_stock": in_stock,
|
||||
"out_of_stock": out_of_stock,
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
playwright>=1.45.0
|
||||
playwright-stealth>=1.0.6
|
||||
requests>=2.31.0
|
||||
schedule>=1.2.1
|
||||
beautifulsoup4>=4.12.0
|
||||
@@ -0,0 +1,5 @@
|
||||
# Scrapers package
|
||||
from .pokemoncenter import PokemonCenterScraper
|
||||
from .target import TargetScraper
|
||||
|
||||
__all__ = ["PokemonCenterScraper", "TargetScraper"]
|
||||
@@ -0,0 +1,75 @@
|
||||
"""
|
||||
Base scraper class with common functionality
|
||||
"""
|
||||
|
||||
import logging
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass
|
||||
from typing import List, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Product:
|
||||
"""Represents a product listing"""
|
||||
name: str
|
||||
url: str
|
||||
price: Optional[str]
|
||||
in_stock: bool
|
||||
image_url: Optional[str] = None
|
||||
site: str = ""
|
||||
product_id: Optional[str] = None # Unique identifier for tracking
|
||||
|
||||
def __hash__(self):
|
||||
return hash(self.url)
|
||||
|
||||
def __eq__(self, other):
|
||||
if isinstance(other, Product):
|
||||
return self.url == other.url
|
||||
return False
|
||||
|
||||
|
||||
class BaseScraper(ABC):
|
||||
"""Base class for site-specific scrapers"""
|
||||
|
||||
site_name: str = "unknown"
|
||||
|
||||
@abstractmethod
|
||||
def scrape_category_page(self, url: str) -> List[Product]:
|
||||
"""
|
||||
Scrape a category/search page and return all products found
|
||||
|
||||
Args:
|
||||
url: URL of the category page
|
||||
|
||||
Returns:
|
||||
List of Product objects
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def check_product_stock(self, product_url: str) -> tuple[bool, Optional[str]]:
|
||||
"""
|
||||
Check if a specific product is in stock
|
||||
|
||||
Args:
|
||||
product_url: URL of the product page
|
||||
|
||||
Returns:
|
||||
Tuple of (is_in_stock, price)
|
||||
"""
|
||||
pass
|
||||
|
||||
def filter_by_keywords(self, products: List[Product], keywords: List[str]) -> List[Product]:
|
||||
"""Filter products by keywords in name"""
|
||||
if not keywords:
|
||||
return products
|
||||
|
||||
filtered = []
|
||||
for product in products:
|
||||
name_lower = product.name.lower()
|
||||
if any(kw.lower() in name_lower for kw in keywords):
|
||||
filtered.append(product)
|
||||
|
||||
return filtered
|
||||
@@ -0,0 +1,276 @@
|
||||
"""
|
||||
PokemonCenter.com scraper
|
||||
Handles bot protection with Playwright stealth
|
||||
"""
|
||||
|
||||
import re
|
||||
import logging
|
||||
from typing import List, Optional
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
from .base import BaseScraper, Product
|
||||
from browser import get_browser
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PokemonCenterScraper(BaseScraper):
|
||||
"""Scraper for PokemonCenter.com"""
|
||||
|
||||
site_name = "pokemoncenter"
|
||||
base_url = "https://www.pokemoncenter.com"
|
||||
|
||||
def scrape_category_page(self, url: str) -> List[Product]:
|
||||
"""
|
||||
Scrape a PokemonCenter category page for all products
|
||||
|
||||
Args:
|
||||
url: Category page URL
|
||||
|
||||
Returns:
|
||||
List of Product objects
|
||||
"""
|
||||
browser = get_browser()
|
||||
products = []
|
||||
|
||||
try:
|
||||
# Navigate and wait for page to load (don't wait for specific selector)
|
||||
page, html = browser.get_page_content(
|
||||
url,
|
||||
wait_for_selector=None, # Let it use networkidle instead
|
||||
timeout=60000,
|
||||
)
|
||||
|
||||
# Save screenshot for debugging if needed
|
||||
try:
|
||||
page.screenshot(path="debug_screenshot.png")
|
||||
logger.info("Saved debug screenshot to debug_screenshot.png")
|
||||
except:
|
||||
pass
|
||||
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
|
||||
# Try multiple selectors for product cards
|
||||
# PokemonCenter may use different structures
|
||||
product_cards = (
|
||||
soup.select("[data-testid='product-card']")
|
||||
or soup.select(".product-card")
|
||||
or soup.select(".product-tile")
|
||||
or soup.select("article[data-product-id]")
|
||||
or soup.select(".product-grid-item")
|
||||
or soup.select("[class*='ProductCard']")
|
||||
)
|
||||
|
||||
logger.info(f"Found {len(product_cards)} product cards")
|
||||
|
||||
# If no product cards found, try a more generic approach
|
||||
if not product_cards:
|
||||
# Look for any links that look like product pages
|
||||
product_links = soup.select("a[href*='/product/']")
|
||||
logger.info(f"Fallback: Found {len(product_links)} product links")
|
||||
|
||||
for link in product_links:
|
||||
href = link.get("href", "")
|
||||
if href and "/product/" in href:
|
||||
full_url = href if href.startswith("http") else f"{self.base_url}{href}"
|
||||
|
||||
# Try to get product name from link text or nearby elements
|
||||
name = link.get_text(strip=True)
|
||||
if not name or len(name) < 3:
|
||||
name = link.get("title", "") or link.get("aria-label", "") or "Unknown Product"
|
||||
|
||||
product = Product(
|
||||
name=name,
|
||||
url=full_url,
|
||||
price=None,
|
||||
in_stock=True, # Assume in stock if listed, will verify later
|
||||
image_url=None,
|
||||
site=self.site_name,
|
||||
product_id=self._extract_product_id(full_url),
|
||||
)
|
||||
products.append(product)
|
||||
|
||||
else:
|
||||
for card in product_cards:
|
||||
product = self._parse_product_card(card)
|
||||
if product:
|
||||
products.append(product)
|
||||
|
||||
page.close()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error scraping PokemonCenter category page: {e}")
|
||||
raise
|
||||
|
||||
# Remove duplicates
|
||||
seen_urls = set()
|
||||
unique_products = []
|
||||
for p in products:
|
||||
if p.url not in seen_urls:
|
||||
seen_urls.add(p.url)
|
||||
unique_products.append(p)
|
||||
|
||||
logger.info(f"Scraped {len(unique_products)} unique products from PokemonCenter")
|
||||
return unique_products
|
||||
|
||||
def _parse_product_card(self, card) -> Optional[Product]:
|
||||
"""Parse a product card element into a Product object"""
|
||||
try:
|
||||
# Try to find product link
|
||||
link = card.select_one("a[href*='/product/']") or card.select_one("a")
|
||||
if not link:
|
||||
return None
|
||||
|
||||
href = link.get("href", "")
|
||||
if not href:
|
||||
return None
|
||||
|
||||
url = href if href.startswith("http") else f"{self.base_url}{href}"
|
||||
|
||||
# Get product name
|
||||
name_elem = (
|
||||
card.select_one("[data-testid='product-name']")
|
||||
or card.select_one(".product-name")
|
||||
or card.select_one("h2")
|
||||
or card.select_one("h3")
|
||||
or card.select_one("[class*='title']")
|
||||
or card.select_one("[class*='name']")
|
||||
)
|
||||
name = name_elem.get_text(strip=True) if name_elem else link.get_text(strip=True)
|
||||
|
||||
if not name or len(name) < 3:
|
||||
name = link.get("title", "") or link.get("aria-label", "") or "Unknown Product"
|
||||
|
||||
# Get price
|
||||
price_elem = (
|
||||
card.select_one("[data-testid='product-price']")
|
||||
or card.select_one(".product-price")
|
||||
or card.select_one("[class*='price']")
|
||||
or card.select_one("span:contains('$')")
|
||||
)
|
||||
price = None
|
||||
if price_elem:
|
||||
price_text = price_elem.get_text(strip=True)
|
||||
# Extract price with regex
|
||||
price_match = re.search(r"\$[\d,]+\.?\d*", price_text)
|
||||
if price_match:
|
||||
price = price_match.group()
|
||||
|
||||
# Check stock status
|
||||
in_stock = self._check_card_stock_status(card)
|
||||
|
||||
# Get image URL
|
||||
img = card.select_one("img")
|
||||
image_url = None
|
||||
if img:
|
||||
image_url = img.get("src") or img.get("data-src")
|
||||
if image_url and not image_url.startswith("http"):
|
||||
image_url = f"{self.base_url}{image_url}"
|
||||
|
||||
return Product(
|
||||
name=name,
|
||||
url=url,
|
||||
price=price,
|
||||
in_stock=in_stock,
|
||||
image_url=image_url,
|
||||
site=self.site_name,
|
||||
product_id=self._extract_product_id(url),
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Error parsing product card: {e}")
|
||||
return None
|
||||
|
||||
def _check_card_stock_status(self, card) -> bool:
|
||||
"""Check if a product card indicates in-stock status"""
|
||||
card_text = card.get_text(lower=True) if hasattr(card, "get_text") else str(card).lower()
|
||||
|
||||
# Out of stock indicators
|
||||
out_of_stock_phrases = [
|
||||
"sold out",
|
||||
"out of stock",
|
||||
"unavailable",
|
||||
"coming soon",
|
||||
"notify me",
|
||||
]
|
||||
|
||||
for phrase in out_of_stock_phrases:
|
||||
if phrase in card_text:
|
||||
return False
|
||||
|
||||
# In stock indicators
|
||||
in_stock_phrases = [
|
||||
"add to cart",
|
||||
"add to bag",
|
||||
"buy now",
|
||||
"in stock",
|
||||
"available",
|
||||
]
|
||||
|
||||
for phrase in in_stock_phrases:
|
||||
if phrase in card_text:
|
||||
return True
|
||||
|
||||
# If we can't determine, assume it might be in stock (will verify on product page)
|
||||
return True
|
||||
|
||||
def check_product_stock(self, product_url: str) -> tuple[bool, Optional[str]]:
|
||||
"""
|
||||
Check if a specific product is in stock by visiting its page
|
||||
|
||||
Args:
|
||||
product_url: URL of the product page
|
||||
|
||||
Returns:
|
||||
Tuple of (is_in_stock, price)
|
||||
"""
|
||||
browser = get_browser()
|
||||
|
||||
try:
|
||||
page, html = browser.get_page_content(
|
||||
product_url,
|
||||
wait_for_selector="button, [data-testid]",
|
||||
timeout=30000,
|
||||
)
|
||||
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
page_text = soup.get_text().lower()
|
||||
|
||||
# Check for out of stock indicators
|
||||
out_of_stock = any(
|
||||
phrase in page_text
|
||||
for phrase in ["sold out", "out of stock", "unavailable", "notify me when available"]
|
||||
)
|
||||
|
||||
# Check for add to cart button
|
||||
add_to_cart = soup.select_one(
|
||||
"button:contains('Add to Cart'), button:contains('Add to Bag'), [data-testid='add-to-cart']"
|
||||
)
|
||||
|
||||
in_stock = not out_of_stock and add_to_cart is not None
|
||||
|
||||
# Get price
|
||||
price = None
|
||||
price_elem = soup.select_one("[data-testid='product-price'], .product-price, [class*='price']")
|
||||
if price_elem:
|
||||
price_match = re.search(r"\$[\d,]+\.?\d*", price_elem.get_text())
|
||||
if price_match:
|
||||
price = price_match.group()
|
||||
|
||||
page.close()
|
||||
return in_stock, price
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking product stock: {e}")
|
||||
return False, None
|
||||
|
||||
def _extract_product_id(self, url: str) -> str:
|
||||
"""Extract product ID from URL"""
|
||||
# PokemonCenter URLs typically look like:
|
||||
# https://www.pokemoncenter.com/product/123456/product-name
|
||||
match = re.search(r"/product/(\d+)", url)
|
||||
if match:
|
||||
return match.group(1)
|
||||
|
||||
# Fallback: use URL path as ID
|
||||
return url.split("/")[-1]
|
||||
@@ -0,0 +1,344 @@
|
||||
"""
|
||||
Target.com scraper
|
||||
"""
|
||||
|
||||
import re
|
||||
import json
|
||||
import logging
|
||||
from typing import List, Optional
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
from .base import BaseScraper, Product
|
||||
from browser import get_browser
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class TargetScraper(BaseScraper):
|
||||
"""Scraper for Target.com"""
|
||||
|
||||
site_name = "target"
|
||||
base_url = "https://www.target.com"
|
||||
|
||||
def scrape_category_page(self, url: str) -> List[Product]:
|
||||
"""
|
||||
Scrape a Target search/category page for all products
|
||||
|
||||
Args:
|
||||
url: Search/category page URL
|
||||
|
||||
Returns:
|
||||
List of Product objects
|
||||
"""
|
||||
browser = get_browser()
|
||||
products = []
|
||||
|
||||
try:
|
||||
page, html = browser.get_page_content(
|
||||
url,
|
||||
wait_for_selector=None,
|
||||
timeout=60000,
|
||||
)
|
||||
|
||||
# Save debug screenshot
|
||||
try:
|
||||
page.screenshot(path="debug_target.png")
|
||||
logger.info("Saved debug screenshot to debug_target.png")
|
||||
except:
|
||||
pass
|
||||
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
|
||||
# Try to find product data in page scripts (Target uses React/hydration)
|
||||
scripts = soup.find_all("script", type="application/json")
|
||||
for script in scripts:
|
||||
try:
|
||||
data = json.loads(script.string)
|
||||
# Look for product data in the JSON
|
||||
products.extend(self._extract_products_from_json(data))
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
continue
|
||||
|
||||
# Parse product links from the page
|
||||
product_links = soup.select("a[href*='/p/']")
|
||||
logger.info(f"Found {len(product_links)} product links")
|
||||
|
||||
# Group links by href and pick the best one (with aria-label or text)
|
||||
href_to_links = {}
|
||||
for link in product_links:
|
||||
href = link.get("href", "")
|
||||
if not href:
|
||||
continue
|
||||
if href not in href_to_links:
|
||||
href_to_links[href] = []
|
||||
href_to_links[href].append(link)
|
||||
|
||||
logger.info(f"Found {len(href_to_links)} unique hrefs")
|
||||
|
||||
for href, links in href_to_links.items():
|
||||
# Find the best link (one with aria-label or text content)
|
||||
best_link = None
|
||||
for link in links:
|
||||
aria = link.get("aria-label", "")
|
||||
text = link.get_text(strip=True)
|
||||
if aria or (text and len(text) > 10):
|
||||
best_link = link
|
||||
break
|
||||
if not best_link:
|
||||
best_link = links[0] # Fallback to first link
|
||||
|
||||
product = self._parse_product_link(best_link, soup)
|
||||
if product:
|
||||
products.append(product)
|
||||
|
||||
page.close()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error scraping Target category page: {e}")
|
||||
raise
|
||||
|
||||
# Remove duplicates
|
||||
seen_urls = set()
|
||||
unique_products = []
|
||||
for p in products:
|
||||
if p.url not in seen_urls:
|
||||
seen_urls.add(p.url)
|
||||
unique_products.append(p)
|
||||
|
||||
logger.info(f"Scraped {len(unique_products)} unique products from Target")
|
||||
return unique_products
|
||||
|
||||
def _extract_products_from_json(self, data, depth=0) -> List[Product]:
|
||||
"""Recursively search JSON for product data"""
|
||||
products = []
|
||||
if depth > 10: # Prevent infinite recursion
|
||||
return products
|
||||
|
||||
if isinstance(data, dict):
|
||||
# Check if this looks like a product
|
||||
if "tcin" in data or ("title" in data and "price" in data):
|
||||
product = self._parse_product_json(data)
|
||||
if product:
|
||||
products.append(product)
|
||||
|
||||
# Recurse into nested objects
|
||||
for value in data.values():
|
||||
products.extend(self._extract_products_from_json(value, depth + 1))
|
||||
|
||||
elif isinstance(data, list):
|
||||
for item in data:
|
||||
products.extend(self._extract_products_from_json(item, depth + 1))
|
||||
|
||||
return products
|
||||
|
||||
def _parse_product_json(self, data: dict) -> Optional[Product]:
|
||||
"""Parse a product from Target's JSON data"""
|
||||
try:
|
||||
name = data.get("title") or data.get("product_description", {}).get("title", "")
|
||||
if not name:
|
||||
return None
|
||||
|
||||
# Build URL
|
||||
tcin = data.get("tcin", "")
|
||||
slug = data.get("url_slug", name.lower().replace(" ", "-"))
|
||||
url = f"{self.base_url}/p/{slug}/-/A-{tcin}" if tcin else ""
|
||||
if not url:
|
||||
return None
|
||||
|
||||
# Get price
|
||||
price_data = data.get("price", {})
|
||||
price = None
|
||||
if isinstance(price_data, dict):
|
||||
price = price_data.get("formatted_current_price") or price_data.get("current_retail")
|
||||
elif isinstance(price_data, (int, float)):
|
||||
price = f"${price_data:.2f}"
|
||||
|
||||
# Check availability
|
||||
availability = data.get("availability_status", "")
|
||||
fulfillment = data.get("fulfillment", {})
|
||||
in_stock = availability not in ["OUT_OF_STOCK", "UNAVAILABLE"]
|
||||
if fulfillment:
|
||||
in_stock = fulfillment.get("is_out_of_stock_in_all_store_locations", True) is False
|
||||
|
||||
# Get image
|
||||
images = data.get("images", [])
|
||||
image_url = images[0].get("base_url") if images else None
|
||||
|
||||
return Product(
|
||||
name=name,
|
||||
url=url,
|
||||
price=str(price) if price else None,
|
||||
in_stock=in_stock,
|
||||
image_url=image_url,
|
||||
site=self.site_name,
|
||||
product_id=tcin,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Error parsing Target product JSON: {e}")
|
||||
return None
|
||||
|
||||
def _parse_product_link(self, link, soup) -> Optional[Product]:
|
||||
"""Parse a product from a product link element"""
|
||||
try:
|
||||
href = link.get("href", "")
|
||||
if not href:
|
||||
return None
|
||||
|
||||
url = href if href.startswith("http") else f"{self.base_url}{href}"
|
||||
|
||||
# Get name - prefer aria-label as it's usually clean
|
||||
name = link.get("aria-label", "")
|
||||
|
||||
# If no aria-label, try link text
|
||||
if not name or len(name) < 5:
|
||||
name = link.get_text(strip=True)
|
||||
|
||||
# Clean up the name
|
||||
if name:
|
||||
# Remove rating text patterns
|
||||
name = re.sub(r'\d+\.?\d*\s*out of \d+ stars.*$', '', name, flags=re.IGNORECASE)
|
||||
name = re.sub(r'\s*\d+ ratings?.*$', '', name, flags=re.IGNORECASE)
|
||||
name = re.sub(r'\s*\d+ reviews?.*$', '', name, flags=re.IGNORECASE)
|
||||
name = name.strip()
|
||||
|
||||
# Skip empty/short names
|
||||
if not name or len(name) < 10:
|
||||
return None
|
||||
|
||||
# Skip if name looks like navigation/rating text only
|
||||
lower_name = name.lower()
|
||||
if any(lower_name.startswith(skip) for skip in ['rating', 'stars', 'review', 'filter', 'sort']):
|
||||
return None
|
||||
|
||||
# Extract product ID from URL (A-12345678)
|
||||
product_id = ""
|
||||
match = re.search(r"/A-(\d+)", url)
|
||||
if match:
|
||||
product_id = match.group(1)
|
||||
|
||||
# Try to find price near this link
|
||||
parent = link.find_parent()
|
||||
price = None
|
||||
for _ in range(5): # Go up 5 levels max
|
||||
if parent:
|
||||
price_text = parent.get_text()
|
||||
price_match = re.search(r"\$[\d,]+\.?\d*", price_text)
|
||||
if price_match:
|
||||
price = price_match.group()
|
||||
break
|
||||
parent = parent.find_parent()
|
||||
|
||||
# Try to find image
|
||||
image_url = None
|
||||
img = link.select_one("img")
|
||||
if img:
|
||||
image_url = img.get("src") or img.get("data-src")
|
||||
|
||||
# Check stock (assume in stock unless we see otherwise)
|
||||
in_stock = True
|
||||
parent = link.find_parent()
|
||||
for _ in range(5):
|
||||
if parent:
|
||||
text = parent.get_text().lower()
|
||||
if "out of stock" in text or "unavailable" in text:
|
||||
in_stock = False
|
||||
break
|
||||
parent = parent.find_parent()
|
||||
|
||||
return Product(
|
||||
name=name,
|
||||
url=url,
|
||||
price=price,
|
||||
in_stock=in_stock,
|
||||
image_url=image_url,
|
||||
site=self.site_name,
|
||||
product_id=product_id,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Error parsing Target product link: {e}")
|
||||
return None
|
||||
|
||||
def _parse_product_card(self, card) -> Optional[Product]:
|
||||
"""Parse a product card element"""
|
||||
try:
|
||||
# Find link
|
||||
link = card.select_one("a[href*='/p/']")
|
||||
if not link:
|
||||
return None
|
||||
|
||||
href = link.get("href", "")
|
||||
url = href if href.startswith("http") else f"{self.base_url}{href}"
|
||||
|
||||
# Get name
|
||||
name_elem = card.select_one("[data-test='product-title']") or card.select_one("a")
|
||||
name = name_elem.get_text(strip=True) if name_elem else "Unknown"
|
||||
|
||||
# Get price
|
||||
price_elem = card.select_one("[data-test='current-price']") or card.select_one("[class*='price']")
|
||||
price = None
|
||||
if price_elem:
|
||||
price_match = re.search(r"\$[\d,]+\.?\d*", price_elem.get_text())
|
||||
if price_match:
|
||||
price = price_match.group()
|
||||
|
||||
# Check stock
|
||||
card_text = card.get_text().lower()
|
||||
out_of_stock = "out of stock" in card_text or "unavailable" in card_text
|
||||
in_stock = not out_of_stock
|
||||
|
||||
# Get image
|
||||
img = card.select_one("img")
|
||||
image_url = img.get("src") if img else None
|
||||
|
||||
# Extract product ID from URL
|
||||
product_id = ""
|
||||
match = re.search(r"/A-(\d+)", url)
|
||||
if match:
|
||||
product_id = match.group(1)
|
||||
|
||||
return Product(
|
||||
name=name,
|
||||
url=url,
|
||||
price=price,
|
||||
in_stock=in_stock,
|
||||
image_url=image_url,
|
||||
site=self.site_name,
|
||||
product_id=product_id,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Error parsing Target product card: {e}")
|
||||
return None
|
||||
|
||||
def check_product_stock(self, product_url: str) -> tuple[bool, Optional[str]]:
|
||||
"""Check if a specific product is in stock"""
|
||||
browser = get_browser()
|
||||
|
||||
try:
|
||||
page, html = browser.get_page_content(product_url, timeout=30000)
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
page_text = soup.get_text().lower()
|
||||
|
||||
out_of_stock = any(
|
||||
phrase in page_text
|
||||
for phrase in ["out of stock", "unavailable", "not available"]
|
||||
)
|
||||
|
||||
in_stock = not out_of_stock
|
||||
|
||||
# Get price
|
||||
price = None
|
||||
price_elem = soup.select_one("[data-test='product-price']")
|
||||
if price_elem:
|
||||
price_match = re.search(r"\$[\d,]+\.?\d*", price_elem.get_text())
|
||||
if price_match:
|
||||
price = price_match.group()
|
||||
|
||||
page.close()
|
||||
return in_stock, price
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking Target product stock: {e}")
|
||||
return False, None
|
||||
Reference in New Issue
Block a user