Store the blocklist in a tree structure.
This should reduce the big memory overhead since we would not store the duplicate segments in the hostnames any more. Implement something along the lines of this:
type Tree struct {
Head string
Tail map[string]Tree
}Then, compile the hostnames in reverse segments, to end up with something like this:
This tree should be equivalent to:
apps.facebook.com
twitter.com
example.net
www.example.net
Since there are a ton of subdomains in the hosts file, and many common TLDs, this could potentially reduce the memory footprint, and could also make the lookups faster (or slower due to the overhead, we should check).
Store the blocklist in a tree structure.
This should reduce the big memory overhead since we would not store the duplicate segments in the hostnames any more. Implement something along the lines of this:
Then, compile the hostnames in reverse segments, to end up with something like this:
"com""facebook""apps""twitter""net""example""""www"This tree should be equivalent to:
Since there are a ton of subdomains in the hosts file, and many common TLDs, this could potentially reduce the memory footprint, and could also make the lookups faster (or slower due to the overhead, we should check).