Archiveteam/grab-site
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Clarify ignore sets

Browse Source
This commit is contained in:
Ivan Kozik 2015-07-27 13:28:49 +00:00
parent 41f7683d98
commit dbe1deb9f0
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@ -100,7 +100,8 @@ Options can come before or after the URL.
* `--igsets=blogs,forums`: use ignore sets `blogs` and `forums`.
Ignore sets are used to exclude a set of junk URLs using a pre-made list of regular expressions.
Ignore sets are used to avoid requesting junk URLs using a pre-made set of
regular expressions.
`forums` and `blogs` are some frequently-used ignore sets.
See [the full list of available ignore sets](https://github.com/ArchiveTeam/ArchiveBot/tree/master/db/ignore_patterns).