I was trying to block matrix I would just need to query the url of the text file
Ok this raises a question for me. How do you find a url like this which wouldn't be like, "linked on their site" or something? I know it must be possible to like dump a URL list for a site to a textfile, I'm just wondering how.
Like say I want to find all the super secret pages on www.subgenius.com, they link some but say www.subgenius.com/pam1/pamphlet.html wasn't directly linked (it is, but pretend lol) but could be accessed by the URL, how would I find that URL? Can you just run like someprogram -a www.subgenius.com -o subgenius.txt because that would be cool.