Commit Graph

16 Commits

Author SHA1 Message Date
mmcghen 74c2a45d50 Add multi-channel Discord webhook routing
Route notifications to both site-specific and type-specific channels simultaneously. Includes site name normalization for matching config keys.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-01 22:16:50 -04:00
mmcghen d33639fdc7 Add product pagination and enable BestBuy
Implement client-side pagination for the products grid: add PRODUCTS_PER_PAGE, currentProductPage, offset in /products requests, renderPagination(), and changeProductPage() (with smooth scroll). Reset page when filters or event filters change and render empty-state pagination. Add pagination container to the template and styles for pagination buttons and info. Also enable "bestbuy" in SITES_ENABLED in config.py and add a taskkill line to .claude/settings.local.json for closing stray python.exe processes.
2026-03-28 16:57:03 -04:00
mmcghen f01bde54a5 Add deduplication and normalize names/URLs
Add product name cleanup and URL normalization across backend, scrapers, and frontend; introduce product/event deduplication. Key changes:
- Clean up SVG/CSS garbage, trailing "X Reviews", mojibake and extra spaces in product names (api, scrapers, JS).
- Normalize URLs (lowercase host, strip tracking/query params, trim trailing slashes) in scrapers and Database and use normalized URL for comparisons/storage.
- Database: use normalized URLs when creating/getting products, prevent duplicate stock events within 1 hour, adjust recent events query to avoid duplicate products.
- Implement deduplication routines in Database: deduplicate_products (by product_id and normalized/clean name) and deduplicate_events (remove duplicate events within same hour).
- API: add POST endpoints /products/deduplicate and /events/deduplicate to trigger deduplication and return stats.
- Frontend: add UI buttons and handlers to call deduplication endpoints and display results.

These changes reduce duplicate product records/events caused by minor URL/name variations and stray CSS/svg artifacts.
2026-03-28 13:31:30 -04:00
mmcghen ad5d179ba3 feat: Add clear filters functionality and improve product filter options in dashboard 2026-03-28 13:02:13 -04:00
mmcghen 25cfa80f20 feat: Enhance Best Buy scraper to support new product card formats and improve data extraction methods 2026-03-28 12:45:18 -04:00
mmcghen 297cb9b146 Sync/dashboard, sanitize content, update config
Add local Claude permissions file; make background.js call syncToDashboard() after each run when config.syncToDashboard is enabled; improve content extraction in content.js by cloning link nodes, removing svg/style elements and stripping review-related text/counts before trimming; update config.py defaults to enable target and gamestop (remove bestbuy), set HEADLESS=True and STEALTH_HEADLESS=True to favor headless environments.
2026-03-28 10:24:25 -04:00
mmcghen 8d382e723f feat: Implement Walmart scraper and integrate with existing architecture
- Added Walmart scraper to scrape product data from Walmart.com, including category pages and product details.
- Introduced a stealth browser module to handle bot protection and improve scraping reliability.
- Created a SQLite database for tracking product history, price changes, stock events, and user favorites.
- Developed a Discord bot for user interaction, allowing location setting and stock checking at local stores.
- Implemented a favorites system to manage priority products and categories with custom notification settings.
- Added news aggregation module to fetch and analyze Pokemon TCG news from various sources.
- Created tools for API discovery and monitoring, including a backend monitor for detecting new products.
- Added unit tests for database operations, product filtering, and API endpoints to ensure functionality.
- Enhanced existing modules with improved error handling and logging for better maintainability.
2026-03-27 23:08:09 -04:00
mmcghen cddae24e34 Fix toggle switches not being clickable
Changed toggle wrapper from div to label so clicking
the slider visual actually toggles the checkbox.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-24 18:42:19 -04:00
mmcghen ef2ff0c58c Increase minimum check interval to 60 seconds
PokemonCenter's Imperva bot protection bans IPs that hit
too frequently. 10 seconds was too aggressive.

- New minimum: 60 seconds
- New default: 90 seconds
- Updated popup UI to reflect safer limits

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-24 18:39:08 -04:00
mmcghen 072bc246fa Add Discord rate limiting to prevent 429 errors
- Limit new product notifications to 5 per check
- Add 1 second delay between Discord notifications
- Restocks still get full priority (always notified)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-24 18:36:59 -04:00
mmcghen d03f963e7b Allow check intervals as low as 10 seconds
- Switch from Chrome alarms (30s min) to setTimeout loops
- Config now uses checkIntervalSeconds instead of minutes
- Default: 15 seconds, minimum: 10 seconds
- Prevents overlapping checks with isChecking flag
- Updated popup UI to show seconds

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-24 18:24:38 -04:00
mmcghen 8f526976bc Optimize: reuse tab and smart polling for faster checks
- Keep tab open between checks (no create/close overhead)
- Poll for products every 500ms instead of fixed 10s wait
- Exit early as soon as products are found
- Max wait reduced to 8 seconds
- Should reduce check time from ~15-20s to ~3-8s

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-24 18:22:00 -04:00
mmcghen 4fa50809dd Add bot protection warning, 30-sec intervals, longer page wait
- Send Discord ping when 0 products found (likely bot protection)
- Allow 0.5 minute (30 second) check intervals
- Increase page wait from 5s to 10s for slow loads

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-24 18:11:27 -04:00
mmcghen 4864aa3af4 Revert debug logging - extension working after human verification
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-24 18:07:55 -04:00
mmcghen 2da4e1856f Add debug logging to content script to diagnose 0 products issue
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-24 18:06:00 -04:00
mmcghen 9d49a99916 Pokemon Stock Monitor - Initial commit
Chrome extension for PokemonCenter monitoring with Discord notifications.
Includes Python scripts for Target monitoring.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-24 12:39:09 -04:00