feat: Implement Walmart scraper and integrate with existing architecture
- Added Walmart scraper to scrape product data from Walmart.com, including category pages and product details. - Introduced a stealth browser module to handle bot protection and improve scraping reliability. - Created a SQLite database for tracking product history, price changes, stock events, and user favorites. - Developed a Discord bot for user interaction, allowing location setting and stock checking at local stores. - Implemented a favorites system to manage priority products and categories with custom notification settings. - Added news aggregation module to fetch and analyze Pokemon TCG news from various sources. - Created tools for API discovery and monitoring, including a backend monitor for detecting new products. - Added unit tests for database operations, product filtering, and API endpoints to ensure functionality. - Enhanced existing modules with improved error handling and logging for better maintainability.
This commit is contained in:
@@ -3,3 +3,13 @@ playwright-stealth>=1.0.6
|
||||
requests>=2.31.0
|
||||
schedule>=1.2.1
|
||||
beautifulsoup4>=4.12.0
|
||||
flask>=3.0.0
|
||||
flask-cors>=4.0.0
|
||||
psutil>=5.9.0
|
||||
undetected-chromedriver>=3.5.0
|
||||
selenium>=4.15.0
|
||||
setuptools>=70.0.0 # Required for Python 3.13+ (distutils compatibility)
|
||||
discord.py>=2.3.0 # Discord bot for location-based store search
|
||||
nltk>=3.8.0 # Sentiment analysis using VADER
|
||||
pytest>=8.0.0 # Testing framework
|
||||
pytest-cov>=4.0.0 # Coverage reporting
|
||||
|
||||
Reference in New Issue
Block a user