Add product name cleanup and URL normalization across backend, scrapers, and frontend; introduce product/event deduplication. Key changes:
- Clean up SVG/CSS garbage, trailing "X Reviews", mojibake and extra spaces in product names (api, scrapers, JS).
- Normalize URLs (lowercase host, strip tracking/query params, trim trailing slashes) in scrapers and Database and use normalized URL for comparisons/storage.
- Database: use normalized URLs when creating/getting products, prevent duplicate stock events within 1 hour, adjust recent events query to avoid duplicate products.
- Implement deduplication routines in Database: deduplicate_products (by product_id and normalized/clean name) and deduplicate_events (remove duplicate events within same hour).
- API: add POST endpoints /products/deduplicate and /events/deduplicate to trigger deduplication and return stats.
- Frontend: add UI buttons and handlers to call deduplication endpoints and display results.
These changes reduce duplicate product records/events caused by minor URL/name variations and stray CSS/svg artifacts.
mmcghen
f01bde54a5