Issue:

Previously, the system incorrectly identified URLs with paths as new domains. For example, if "youtube.com" already existed in the database and a user submitted an issue with "youtube.com/home", the system treated "youtube.com/home" as a new domain, creating a duplicate domain object.

Solution:

• Domain Cleaning: Implemented a function to extract the base domain from URLs, ensuring that "youtube.com" and "youtube.com/home" are recognized as the same domain.
• Database Check: Updated the domain checking mechanism to prevent the creation of duplicate domain objects for URLs with the same base domain.

Changes Made:

• Added a clean_domain function to extract the base domain using urlparse.
• Updated the domain creation logic to check for existing domains using the cleaned domain value.

Example:

• Before: "youtube.com" and "youtube.com/home" were treated as separate domains.
• After: Both URLs are recognized as "youtube.com", preventing duplicate entries.

Request for Review:

@DonnieBLT, could you please review these changes for merging?