Structured Data and the such

[jessedp]

Hello again. I'm guessing you've not had time to think about how/if you'd like to publish the lists with extended data like we were talking about.

I bring it up again, here this time, because a user asked a question (jessedp/pihole5-list-tool#18) that would be partially solved by having more structure data.

We hadn't talked exactly about how things were going to be implemented, but something like the following would do it. The 2 important things here are:

1. The extra data we talked about (category, list)
2. Still distributing them like you have been since that works well enough (ie, "ticked' => "ticked", "nocross" => "ticked + nocross")

ListsV2 endpoint:

https://v.firebog.net/hosts/listsV2.php

Ticked List:

https://v.firebog.net/hosts/listsV2.php?type=ticked

"suspicious","tick","https://github.com/PolishFiltersTeam/KADhosts","PolishFiltersTeam KADhosts","https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts_without_controversies.txt"
"suspicious","tick","https://github.com/FadeMind/hosts.extras/","Fademind's Spammers","https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Spam/hosts"
...

Non-crossed List:

https://v.firebog.net/hosts/listsV2.php?type=nocross

"suspicious","tick","https://github.com/PolishFiltersTeam/KADhosts","PolishFiltersTeam KADhosts","https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts_without_controversies.txt"
"suspicious","tick","https://github.com/FadeMind/hosts.extras/","Fademind's Spammers","https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Spam/hosts"
"suspicious","cross","https://hostsfile.mine.nu/","Andy Short","https://hostsfile.mine.nu/hosts0.txt"
"suspicious","cross","http://www.sa-blacklist.stearns.org/sa-blacklist/","Bill Stearn","https://v.firebog.net/hosts/BillStearns.txt"
...

I also saw you mention VPS and code rewrite issues elsewhere - I'm happy to help you generally get a more comfortable setup in place if you'd like.

[WaLLy3K]

I've definitely not had any time to look into this stuff further, due to a nasty injury and simultaneously, my workplace needing far more attention than it normally needs.

Distributing the lists in the old method made sense, because you could just wget everything into a text file and you were done. With the CSV, one is already putting in a bit of work to have to parse that correctly, and I don't see it as much effort for someone parsing the file to throw in an extra line that is the equivalent of grep ',"tick",' to get the list they need. I'm happy to be convinced of why separate lists would be useful, I just don't see the need yet! :)

VPS list parsing will have to come later, and isn't required or is it very high on my list of priorities right now.

[WaLLy3K]

@jessedp @fritterhoff I've just made a change where the CSV now exposes the adult blocklists. Just providing a heads up to both of you directly in case those huge lists end up breaking something in your implementation. 🙂