Add product name cleanup and URL normalization across backend, scrapers, and frontend; introduce product/event deduplication. Key changes:
- Clean up SVG/CSS garbage, trailing "X Reviews", mojibake and extra spaces in product names (api, scrapers, JS).
- Normalize URLs (lowercase host, strip tracking/query params, trim trailing slashes) in scrapers and Database and use normalized URL for comparisons/storage.
- Database: use normalized URLs when creating/getting products, prevent duplicate stock events within 1 hour, adjust recent events query to avoid duplicate products.
- Implement deduplication routines in Database: deduplicate_products (by product_id and normalized/clean name) and deduplicate_events (remove duplicate events within same hour).
- API: add POST endpoints /products/deduplicate and /events/deduplicate to trigger deduplication and return stats.
- Frontend: add UI buttons and handlers to call deduplication endpoints and display results.
These changes reduce duplicate product records/events caused by minor URL/name variations and stray CSS/svg artifacts.
Add local Claude permissions file; make background.js call syncToDashboard() after each run when config.syncToDashboard is enabled; improve content extraction in content.js by cloning link nodes, removing svg/style elements and stripping review-related text/counts before trimming; update config.py defaults to enable target and gamestop (remove bestbuy), set HEADLESS=True and STEALTH_HEADLESS=True to favor headless environments.
- Added Walmart scraper to scrape product data from Walmart.com, including category pages and product details.
- Introduced a stealth browser module to handle bot protection and improve scraping reliability.
- Created a SQLite database for tracking product history, price changes, stock events, and user favorites.
- Developed a Discord bot for user interaction, allowing location setting and stock checking at local stores.
- Implemented a favorites system to manage priority products and categories with custom notification settings.
- Added news aggregation module to fetch and analyze Pokemon TCG news from various sources.
- Created tools for API discovery and monitoring, including a backend monitor for detecting new products.
- Added unit tests for database operations, product filtering, and API endpoints to ensure functionality.
- Enhanced existing modules with improved error handling and logging for better maintainability.
Changed toggle wrapper from div to label so clicking
the slider visual actually toggles the checkbox.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
PokemonCenter's Imperva bot protection bans IPs that hit
too frequently. 10 seconds was too aggressive.
- New minimum: 60 seconds
- New default: 90 seconds
- Updated popup UI to reflect safer limits
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Limit new product notifications to 5 per check
- Add 1 second delay between Discord notifications
- Restocks still get full priority (always notified)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Switch from Chrome alarms (30s min) to setTimeout loops
- Config now uses checkIntervalSeconds instead of minutes
- Default: 15 seconds, minimum: 10 seconds
- Prevents overlapping checks with isChecking flag
- Updated popup UI to show seconds
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Keep tab open between checks (no create/close overhead)
- Poll for products every 500ms instead of fixed 10s wait
- Exit early as soon as products are found
- Max wait reduced to 8 seconds
- Should reduce check time from ~15-20s to ~3-8s
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Send Discord ping when 0 products found (likely bot protection)
- Allow 0.5 minute (30 second) check intervals
- Increase page wait from 5s to 10s for slow loads
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Chrome extension for PokemonCenter monitoring with Discord notifications.
Includes Python scripts for Target monitoring.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
mmcghen