mmcghen
f01bde54a5
Add product name cleanup and URL normalization across backend, scrapers, and frontend; introduce product/event deduplication. Key changes: - Clean up SVG/CSS garbage, trailing "X Reviews", mojibake and extra spaces in product names (api, scrapers, JS). - Normalize URLs (lowercase host, strip tracking/query params, trim trailing slashes) in scrapers and Database and use normalized URL for comparisons/storage. - Database: use normalized URLs when creating/getting products, prevent duplicate stock events within 1 hour, adjust recent events query to avoid duplicate products. - Implement deduplication routines in Database: deduplicate_products (by product_id and normalized/clean name) and deduplicate_events (remove duplicate events within same hour). - API: add POST endpoints /products/deduplicate and /events/deduplicate to trigger deduplication and return stats. - Frontend: add UI buttons and handlers to call deduplication endpoints and display results. These changes reduce duplicate product records/events caused by minor URL/name variations and stray CSS/svg artifacts.