feat: Implement Walmart scraper and integrate with existing architecture
- Added Walmart scraper to scrape product data from Walmart.com, including category pages and product details. - Introduced a stealth browser module to handle bot protection and improve scraping reliability. - Created a SQLite database for tracking product history, price changes, stock events, and user favorites. - Developed a Discord bot for user interaction, allowing location setting and stock checking at local stores. - Implemented a favorites system to manage priority products and categories with custom notification settings. - Added news aggregation module to fetch and analyze Pokemon TCG news from various sources. - Created tools for API discovery and monitoring, including a backend monitor for detecting new products. - Added unit tests for database operations, product filtering, and API endpoints to ensure functionality. - Enhanced existing modules with improved error handling and logging for better maintainability.
This commit is contained in:
+16
-1
@@ -6,11 +6,26 @@ __pycache__/
|
||||
venv/
|
||||
.venv/
|
||||
|
||||
# Logs and data
|
||||
# Logs
|
||||
*.log
|
||||
|
||||
# Data files (runtime/generated)
|
||||
products.json
|
||||
data/sessions/
|
||||
data/*.json
|
||||
data/*.pkl
|
||||
*.pkl
|
||||
|
||||
# Debug artifacts
|
||||
debug_*.png
|
||||
|
||||
# Large analysis files
|
||||
*.har
|
||||
*_analysis.json
|
||||
|
||||
# Database
|
||||
*.db
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
|
||||
Reference in New Issue
Block a user