feat: Implement Walmart scraper and integrate with existing architecture
- Added Walmart scraper to scrape product data from Walmart.com, including category pages and product details. - Introduced a stealth browser module to handle bot protection and improve scraping reliability. - Created a SQLite database for tracking product history, price changes, stock events, and user favorites. - Developed a Discord bot for user interaction, allowing location setting and stock checking at local stores. - Implemented a favorites system to manage priority products and categories with custom notification settings. - Added news aggregation module to fetch and analyze Pokemon TCG news from various sources. - Created tools for API discovery and monitoring, including a backend monitor for detecting new products. - Added unit tests for database operations, product filtering, and API endpoints to ensure functionality. - Enhanced existing modules with improved error handling and logging for better maintainability.
This commit is contained in:
+11
-1
@@ -1,5 +1,15 @@
|
||||
# Scrapers package
|
||||
from .pokemoncenter import PokemonCenterScraper
|
||||
from .target import TargetScraper
|
||||
from .gamestop import GameStopScraper, warmup_gamestop
|
||||
from .bestbuy import BestBuyScraper
|
||||
from .walmart import WalmartScraper
|
||||
|
||||
__all__ = ["PokemonCenterScraper", "TargetScraper"]
|
||||
__all__ = [
|
||||
"PokemonCenterScraper",
|
||||
"TargetScraper",
|
||||
"GameStopScraper",
|
||||
"BestBuyScraper",
|
||||
"WalmartScraper",
|
||||
"warmup_gamestop",
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user