This page lists the changes to the redundancy checker, if said changes had an effect on the behaviour of the check
+
+
May 23, 2015
+
+
Fixwhitelist filters that start with a protocol no longer imply $document in the next version of ABP (example: @@|http://x.y/)
+
+
+
+
May 17, 2015
+
+
Modifiedthe future options $generichide and $genericblock are still ignored, but no longer trigger error messages
+
UIminor changes for mobile browsers
+
+
+
March 9, 2015
+
+
Newdomain check tool: add option to export the current results, which can also be used to resume the check at a later moment
+
Fixdomain check tool: spaces were removed from filters
+
+
+
March 8, 2015
+
+
Newtool to detect dead, redirected or parked first party domains (requires Chrome)
+
Fixsometimes multiple warnings were shown per rule, instead of only the most important warning ($domain=a.|b|a.)
+
Modifiedtightened the triggering conditions for the warning about a blocking rule being a broken hiding exclusion rule
+
+
+
January 5, 2015
+
+
Newwarning if the value of a blocking option contains a $ which should probably be a , ($domain=abc$image)
+
Improvedwarning messages when invalid domain separators are found ($domain=a||b)
+
Improvedwarning messages related to incorrectly used blocking options ($image=foo or $~domain=foo)
+
Improvedwarning messages related to overruled blocking options ($image,~image,script)
+
Fix$document rules did not take into account the trailing dot domain variant for blocking rules with $domain=... (@@||abc.def^$document and $domain=abc.def)
+
Fixmultiple warnings with shared rules should only be filtered out if all rules are identical (@@||a^$elemhide should warn against both c,a##.adv and a,b##.ads at the same time)
+
Fixwarnings for a mutually redundant, equal length rule were dependent on the position of the other rule in the filters (##.A.\30 .B and ##.A.B.\30 )
+
Fix:not(...) selectors were not checked for conflicts when 'not' contained capital letters (##div:NOT(div))
+
Fixwhitelisted rules containing the $collapse option did not make blocking rules without $collapse redundant
+
Fix$match-case,~match-case was treated as $match-case
+
Fixwhitelist rules that imply $document were made redundant by rules that do not imply $document if no options were present (@@|http:// was made redundant by @@http)
+
Modifiedipv6 addresses are now also accepted after a ||, although I don't know how ABP parses them (||[1234:123:1:12::123]^)
+
Modifiedknown options are now shown uniformly in warning messages (lowercased, without spaces and using underscores)
+
+
+
July 7, 2014:
+
+
Newwarn about rules with contradicting :nth-* selectors as they cannot match anything (##:nth-child(3):nth-child(4))
+
Newwarn about selectors with different values for attribute selectors with the operators ^= and |=, as they cannot match anything (##[attr^="cow"][attr|="dog"])
+
Newwarn about rules containing :not(...) pseudo-classes that cannot coexist with other selector types in the same rule (##:not([class="ads"]).adv)
+
Newdetect redundancies between rules containing :not(...) pseudo-classes and rules containing other selector types (##:not([class="ads"]) versus ##.adv)
+
Newrules containing :nth-* pseudo-classes are compared numerically, instead of literally (##:nth-child(3) and ##:nth-child(2n+1))
+
Newrules containing :not(...) pseudo-classes are no longer only literally compared against other :not(...) selectors (##:not([href*="a"]) and ##:not([href="ads"]))
+
Improvedthe warning messages that tell you about conflicts within a rule are more specific (##[id="a"][id^="b"])
+
Improvedthe warning messages that tell you about redundancies within a rule are more specific (###ads,div#ads)
+
Fixattribute selectors with |= made attribute selectors with ^= redundant if the attribute values were identical (##a[abc^="abc"] and ##[abc|="abc"])
+
Fixcombined hiding rules with internal redundancies could not be made redundant by non-combined rules (###ads,#ads and ###ads)
+
Fix:not(*) shouldn't make rules with different namespaces redundant (##ns|*:not(*))
+
Fix:nth-*(An+B) selectors with A or B very large numbers still suffered from rounding errors
+
Fixinternally ignored :not(...) selectors acted as the universal selector * (##:not(:nth-child(9999999999999999)) made ##div redundant)
+
Modifiedattribute selector combinations that can't coexist do not match other impossible combinations anymore (###a#b and ##[id="a"][id^="b"])
+
Modifiedrewrite the way hiding rules are parsed
+
+
+
January 19, 2014:
+
+
Newwarn about redundant excluded domains in case the least restrictive rule is site-specific (~site.com in site.com##x and ~site.com##x.y)
+
Improvedup to 30% increase of speed
+
Removedthe tab with ignored rules is removed as it was incomplete and didn't distinguish between fully and partially ignored rules
+
Fix:nth-child(An+B)/:nth-last-child(An+B)/:nth-of-type(An+B)/:nth-last-of-type(An+B) selectors with A or B very large numbers suffered from rounding errors
+
UIadd documentation about the limitations of the redundancy checker
+
UIfix some typos
+
+
+
January 13, 2014:
+
+
Newwarn if ,$ is used to separate rules (example: $script,$domain=com)
+
Newdetect redundancies in hiding rules if a #@# rule excluded part of the rule (example: nl#@##ads and ###ads together with com##div#ads)
+
Improvedall warnings for redundant domains now clearly state which domains can be removed
+
Removedwarnings about domain redundancies with non-redundant subdomains (example: ~b.a in b.a,~c.b.a#@#ads and a,~b.a##ads)
+
Fix/\|\|x/$document and similar regex rules with $document or $elemhide no longer make $domain=x or x##y redundant
+
UIin case a rule could be made redundant by two rules, and those rules are no redundancies of each other, the shortest rule is now displayed
+
UItell the user to first check for redundancies again before using the tools, if they modify the contents of the text box
+
UIsome changes in the warning messages of simplified syntax hiding rules
+
+
+
November 16, 2013:
+
+
Newwarning about $image,~image,popup, where $image,~image does nothing. ABP now drops unknown options, so those constructions are unneeded
+
Newwarning about $~match-case, since it's the default to be case insensitive (/ads.$~match-case)
+
Newwarn when pseudo-element selectors are present, since they cannot be used for hiding content (##::before)
+
NewCSS selector identifiers containing escaped Unicode at the end of the identifier may have a trailing space (##\32 >x is equal to ##\32>x)
+
Improvedwarning about redundant type options if the first type option is an excluded type ($~script,image,~image)
+
Improvedspecify the location of an unnecessary universal CSS selector in warning messages (only the first * in ##*.class > *)
+
Improvedmore (all?) redundant sequences of complex tree selectors can now be detected (##a > div div > x and ##a > div div div > x)
+
Fixif A is made redundant by B and B is made redundant by a $document or $elemhide rule C, A must be redundant of C instead of B
+
Fixmore warning messages in which the options were shown as lowercased and with all - replaced by _
+
Fixincorrect warnings for $~elemhide and $~document (@@/ads.$~elemhide,document)
+
Fixcrash of the redundancy checker if a hiding rule matched nothing and also contained an internal duplicate (##:not(*):not(*))
+
Fixrules with duplicates of :not(...) selectors and another comma-separated selector, acted as if :not(...) was * (##:not(a):not(a ),x)
+
FixCSS selectors with :lang(x-y) would skip the selector behind the :lang(...) selector (##:lang(en-gb)>ads)
+
Modifiedrewrite the way the options of blocking rules are parsed
+
Modifiedrewrite the way how rules containing tree selectors ('', >, +, ~) are matched
+
UIfix typo in warning message for :not(...) selectors that match every element (##:not(:nth-child(-4)))
+
+
+
September 20, 2013:
+
+
Improved'same rule, different domains' tool is faster and finds more combinable rules
+
Improvedbetter warning message for rules of the form @@domain##rule
+
Improvedbetter error message for incorrectly excluded domains (abc$~domain=site.com)
+
Fixin the error message unknown option 'X', any - in the option was replaced by _, and it was in lowercase (example: abc$First-Party)
+
+
+
July 18, 2013:
+
+
Fix'same rule, different domains' tool suggested combining a##bar,baz and b##bar+baz
+
Fixrare case where the Check for redundant rules button didn't get enabled
+
+
+
June 12, 2013:
+
+
FixPunycode-converted domains are considered valid now (xn----etbbchqbn2afauadx.xn--p1ai##.ads)
+
+
+
May 31, 2013:
+
+
Fixtool to find whitelists shouldn't report other whitelists that it matches (@@x/xml and @@/xml$image)
+
Fixtool to convert hiding into blocking reports wrong results for \= in the attribute name (##[src\=='hello'])
+
Fixtool to use a less strict matching method matched two characters that would both be matched by ^ (&adType= and ?adType=)
+
Improvedcode improvements
+
+
+
May 20, 2013:
+
+
Newtool to find hiding rules that could potentially become blocking rules
+
Fixincorrect warning message for rules that start with *@@: they should keep their preceding wildcard
+
Improvedmatching of the :lang(...) selector
+
+
+
May 6, 2013:
+
+
Improvedmatching of first/last/nth/only-child and first/last/nth/only-of-type selectors
+
+
+
April 28, 2013:
+
+
Fixincorrect warning message: selectors that do not match anything, actually match everything when in a :not() selector (example: ##:not(:nth-child(-4)))
+
Fixpseudo-element CSS selectors are not allowed in :not() selectors in CSS3 (##:not(::before))
+
Fixcombinations of CSS selectors are not allowed in :not() selectors in CSS3 (##:not(##div#ads))
+
Fixnamespaces in CSS3 are only allowed in front of nodename selectors (##abc|#def)
+
Fixin an internal rule redundancy, the most restrictive part is the active part (##[a^="b"][a^="bcd"] and ##[a^="bc"])
+
Fixincorrect warning message for comma-separated hiding rules for which one part can't match anything (##div,:not(*))
+
Fixdo not automatically assume case insensitivity for browser-specific pseudo selectors (##:moz-whatever(x) versus ##:moz-whatever(X))
+
Modifiedrules that match nothing (example: $image,~image) won't be made redundant by every other rule anymore; a warning is sufficient
+
Modifiedprefer the message that donottrack is deprecated over one that the rule doesn't match anything
+
Modifiedrewrite the way hiding rules are parsed. This method should be more fail-proof.
+
UIfix typo in warning message
+
+
+
April 14, 2013:
+
+
Modifiedreworked the warnings 'management' system so that it always returns the most important warning instead of the last found warning per rule
+
Improvedwarnings about redundancies within a rule (example: ##[a="bcd"][a^="bc"])
+
Improvedup to 5% increase of speed
+
Fixincorrect warnings about redundancies within a rule if namespaces in attribute selectors were present (##[ns1|abc="def"][ns2|abc="def"])
+
Fixdisable warnings about selectors that cannot co-exist when a namespace selector is present (##a|#b[*|id^='b']), because I'm unsure about the validity of that warning
+
Fixincorrect broken selector warning if a rule contains Unicode characters 00A0 to 0177
+
UIfix bad rule highlighting when a filter with a warning contained ' and ' (example: ##*[ and $=" and "])
+
UIwarn when a browser lacks some functionality
+
+
+
March 28, 2013:
+
+
Improvedup to 20% increase of speed
+
+
+
March 26, 2013:
+
+
Fixtreat every line containing *[adblock*]* as comment
+
Fixhiding rules containing ##tag-or-class-or-id\::known-pseudoselector are valid CSS (example: ###mysite\::first-child)
+
Fixduplicate warning if an excluded domain in a hiding rule was also an included domain in a hiding whitelist rule (example: a,b,~b.a##x and a#@#x)
+
Fixhiding rules excluded by a #@# rule do not make other rules redundant anymore (###ads and foo.com#@##ads and foo.com###ads.banner)
+
Fixrules with included domains, excluded subdomains and included sub-subdomains weren't shown as redundant for the last subdomain (a,~b.a,c.b.a##x versus c.b.a##x.y)
+
Fixsimplified hiding rules suggest the #id and .class syntax, instead of [id="the_id"] and [class~="the_class"]
+
Fixwarnings involving the same rule twice were removed (##[id="x"] and ##[id="x"])
+
Newwarn if you have an rule with only excluded domains and a rule with partially redundant included domains (~b.a##x vs z,c.b.a##x)
+
Newif a hiding rule matches a parent element of another hiding rule, the latter is redundant (###foo versus ###foo > #bar)
+
Newhiding rules may skip a tree depth if their selector allows so (###foo > bar > baz versus ###foo baz)
+
Newwarn if you have selectors that can't co-exist (###id1#id2, ##[a="b"][a="c"], ...)
+
Newwarn if comma-separated hiding rules make each other redundant (###ads,.xyz#ads)
+
Improvedwarn if you have multiple included domains and some of them match (example: foo,a.bar##a versus bar,baz##a)
+
Improvedsupport for comma-separated hiding rules (domain.com###ads,#adv)
+
UIfor errors reported (for broken rules, like ##rule]), an ! was shown in front of the message
+
UIreplace contact link with information link
+
UIdisable spellchecking of the filter list input area
+
+
+
January 31, 2013:
+
+
Newadd several tools:
+
+
A similar rules finder tool
+
Tools to ignore domains and blocking options (replaces the checkbox)
+
A tool to use a less strict matching algorithm
+
A tool to find rules which have the same rule, but a different domain
+
A tool to find the rules that make whitelisting rules necessary, which also displays if no rules could be found for a whitelisting rule
+
+
+
New||x will now also be matched if both /x and .x are present
+
Modifieddeprecate $donottrack, since it's removed from ABP too
+
Fix// isn't a regex
+
Fix' !x' (with a whitespace in front of it) is a comment, not a blocking rule
+
Fix*!x and similar will no longer trigger unnecessary preceding wildcard found warnings, since it would become a comment without *
+
Fix/\|\|x/ and similar regex rules no longer make ||x redundant
+
+
+
No changelog available before 2013.
+
+
\ No newline at end of file
diff --git a/famlam-redundancychecker/changelog.txt b/famlam-redundancychecker/changelog.txt
new file mode 100644
index 000000000..301a9685f
--- /dev/null
+++ b/famlam-redundancychecker/changelog.txt
@@ -0,0 +1,128 @@
+January 13, 2014:
+-New: warn if ',$' is used to separate rules (example: $script,$domain=com)
+-New: detect redundancies in hiding rules if a #@# rule excluded part of the rule (example: nl#@##ads and ###ads together with com##div#ads)
+-Improved: all warnings for redundant domains now clearly state which domains can be removed
+-Removed: warnings about domain redundancies with non-redundant subdomains (example: ~b.a in b.a,~c.b.a#@#ads and a,~b.a##ads)
+-Fix: '/\|\|x/$document' and similar regex rules with $document or $elemhide no longer make '$domain=x' or 'x##y' redundant
+-UI: in case a rule could be made redundant by two rules, and those rules are no redundancies of each other, the shortest rule is now displayed
+-UI: tell the user to first check for redundancies again before using the tools, if they modify the contents of the text box
+-UI: some changes in the warning messages of simplified syntax hiding rules
+
+
+November 16, 2013:
+-New: warning about $image,~image,popup, where $image,~image does nothing. ABP now drops unknown options, so those constructions are unneeded
+-New: warning about $~match-case, since it's the default to be case insensitive (/ads.$~match-case)
+-New: warn when pseudo-element selectors are present, since they cannot be used for hiding content (##::before)
+-New: CSS selector identifiers containing escaped unicode at the end of the identifier may have a trailing space (##\32 >x is equal to ##\32>x)
+-Improved: warning about redundant type options if the first type option is an excluded type ($~script,image,~image)
+-Improved: specify the location of an unnecessary universal CSS selector in warning messages (only the first * in ##*.class > * )
+-Improved: more (all?) redundant sequences of complex tree selectors can now be detected (##a > div div > x and ##a > div div div > x)
+-Fix: if A is made redundant by B and B is made redundant by a $document or $elemhide rule C, A must be redundant of C instead of B
+-Fix: more warning messages in which the options were shown as lowercased and with all - replaced by _
+-Fix: incorrect warnings for $~elemhide and $~document (@@/ads.$~elemhide,document)
+-Fix: crash of the redundancy checker if a hiding rule matched nothing and also contained an internal duplicate (##:not(*):not(*) )
+-Fix: rules with duplicates of :not(...) selectors and another comma-separated selector, acted as if ':not(...)' was '*' (##:not(a):not(a ),x )
+-Fix: CSS selectors with :lang(x-y) would skip the selector behind the :lang(...) selector (##:lang(en-gb)>ads)
+-Modified: rewrite the way the the options of blocking rules are parsed
+-Modified: rewrite the way how rules containing tree selectors (' ', >, +, ~) are matched
+-UI: fix typo in warning message for :not(...) selectors that match every element (##:not(:nth-child(-4)) )
+
+
+September 20, 2013:
+-Improved: 'same rule, different domains' tool is faster and finds more combinable rules
+-Improved: better warning message for rules of the form '@@domain##rule'
+-Improved: better error message for incorrectly excluded domains (abc$~domain=site.com)
+-Fix: in the error message "unknown option 'X'", any '-' in the option was replaced by '_', and it was in lowercase (example: abc$First-Party)
+
+
+July 18, 2013:
+-Fix: 'same rule, different domains' tool suggested combining a##bar,baz and b##bar+baz
+-Fix: rare case where the 'Check for redundant rules' button didn't get enabled
+
+
+June 12, 2013:
+-Fix: punycode-converted domains are considered valid now (xn----etbbchqbn2afauadx.xn--p1ai##.ads)
+
+
+May 31, 2013:
+-Fix: tool to find whitelists shouldn't report other whitelists that it matches (@@x/xml and @@/xml$image)
+-Fix: tool to convert hiding into blocking reports wrong results for \= in the attribute name ( ##[src\=='hello'] )
+-Fix: tool to use a less strict matching method matched two characters that would both be matched by ^ ( &adType= and ?adType= )
+-Improved: code improvements
+
+
+May 20, 2013:
+-New: tool to find hiding rules that could potentially become blocking rules
+-Fix: incorrect warning message for rules that start with '*@@': they should keep their preceeding wildcard
+-Improved: matching of the :lang(...) selector
+
+
+May 6, 2013:
+-Improved: matching of first/last/nth/only-child and first/last/nth/only-of-type selectors
+
+
+April 28, 2013:
+-Fix: incorrect warning message: selectors that do not match anything, actually match everything when in a :not() selector (example: ##:not(:nth-child(-4)) )
+-Fix: pseudo-element CSS selectors are not allowed in :not() selectors in CSS3 (##:not(::before) )
+-Fix: combinations of CSS selectors are not allowed in :not() selectors in CSS3 (##:not(##div#ads) )
+-Fix: namespaces in CSS3 are only allowed in front of nodename selectors (##abc|#def)
+-Fix: in an internal rule redundancy, the most restrictive part is the active part (##[a^="b"][a^="bcd"] and ##[a^="bc"] )
+-Fix: incorrect warning message for comma-separated hiding rules for which one part can't match anything (##div,:not(*) )
+-Fix: do not automatically assume case insensitivity for browser-specific pseudo selectors (##:moz-whatever(x) versus ##:moz-whatever(X) )
+-Modified: rules that match nothing (example: $image,~image) won't be made redundant by every other rule anymore; a warning is sufficient
+-Modified: prefer the message that donottrack is depricated over one that the rule doesn't match anything
+-Modified: rewrite the way hiding rules are parsed. This method should be more fail-proof.
+-UI: fix typo in warning message
+
+
+April 14, 2013:
+- Modified: reworked the warnings 'management' system so that it always returns the most important warning instead of the last found warning per rule
+- Improved: warnings about redundancies within a rule (example: ##[a="bcd"][a^="bc"])
+- Improved: up to 5% increase of speed
+- Fix: incorrect warnings about redundancies within a rule if namespaces in attribute selectors were present (##[ns1|abc="def"][ns2|abc="def"])
+- Fix: disable warnings about selectors that cannot co-exist when a namespace selector is present (##a|#b[*|id^='b']), because I'm unsure about the validity of that warning
+- Fix: incorrect 'broken selector' warning if a rule contains unicode characters 00A0 to 0177
+- UI: fix bad rule highlighting when a filter with a warning contained " and " (##*[ and $=" and "])
+- UI: warn when a browser lacks some functionality
+
+
+March 28, 2013:
+- Improved: up to 20% increase of speed
+
+
+March 26, 2013:
+- Fix: treat every line containing *[adblock*]* as comment
+- Fix: hiding rules containing ##tag-or-class-or-id\::known-pseudoselector are valid CSS (example: ###mysite\::first-child)
+- Fix: duplicate warning if an excluded domain in a hiding rule was also an included domain in a hiding whitelist rule (example: a,b,~b.a##x and a#@#x)
+- Fix: hiding rules excluded by a #@# rule do not make other rules redundant anymore (###ads and foo.com#@##ads and foo.com###ads.banner)
+- Fix: rules with included domains, excluded subdomains and included sub-subdomains weren't shown as redundant for the last subdomain (a,~b.a,c.b.a##x versus c.b.a##x.y)
+- Fix: simplified hiding rules suggest the #id and .class syntax, instead of [id="the_id"] and [class~="the_class"]
+- Fix: warnings involving the same rule twice were removed (##[id="x"] and ##[id="x"])
+- New: warn if you have an rule with only excluded domains and a rule with partially redundant included domains (~b.a##x vs z,c.b.a##x)
+- New: if a hiding rule matches a parent element of another hiding rule, the latter is redundant (###foo versus ###foo > #bar)
+- New: hiding rules may skip a tree depth if their selector allows so (###foo > bar > baz versus ###foo baz)
+- New: warn if you have selectors that can't co-exist (###id1#id2, ##[a="b"][a="c"], ...)
+- New: warn if comma-separated hiding rules make each other redundant (###ads,.xyz#ads)
+- Improved: warn if you have multiple included domains and some of them match (example: foo,a.bar##a versus bar,baz##a)
+- Improved: support for comma-separated hiding rules (domain.com###ads,#adv)
+- UI: for errors reported (for broken rules, like "##rule]" ), an "!" was shown in front of the message
+- UI: replace contact link with information link
+- UI: disable spellchecking of the filter list input area
+
+
+January 31, 2013:
+- New: add several tools:
+ * A similar rules finder tool
+ * Tools to ignore domains and blocking options (replaces the checkbox)
+ * A tool to use a less strict matching algorithm
+ * A tool to find rules which have the same rule, but a different domain
+ * A tool to find the rules that make whitelisting rules necessary, which also displays if no rules could be found for a whitelisting rule
+- New: ||x will now also be matched if both /x and .x are present
+- Modified: depricate $donottrack, since it's removed from ABP too
+- Fix: '//' isn't a regex
+- Fix: ' !x' (with a whitespace in front of it) is a comment, not a blocking rule
+- Fix: '*!x' and similar will no longer trigger 'unnecessary preceeding wildcard found' warnings, since it would become a comment without *
+- Fix: '/\|\|x/' and similar regex rules no longer make ||x redundant
+
+
+No changelog available before 2013.
\ No newline at end of file
diff --git a/famlam-redundancychecker/famlam.tar.gz b/famlam-redundancychecker/famlam.tar.gz
new file mode 100644
index 000000000..6d2beeb46
Binary files /dev/null and b/famlam-redundancychecker/famlam.tar.gz differ
diff --git a/famlam-redundancychecker/findWhitelistRules.js b/famlam-redundancychecker/findWhitelistRules.js
new file mode 100644
index 000000000..f4aea6a36
--- /dev/null
+++ b/famlam-redundancychecker/findWhitelistRules.js
@@ -0,0 +1,67 @@
+/*!
+ This script searches for active whitelisting rules written in the Adblock Plus syntax,
+ documentated here: http://adblockplus.org/en/filters, and reports them.
+ Author: Famlam (fam.lam [at] live.nl)
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+*/
+"use strict";
+var findWhitelistRules = function(lines) {
+ importScripts("redundant.js");
+
+ var ELEMHIDE = /^([^\/\*\|\@\"\!]*?)\#\s*(\@)?\s*\#([^\{\}]+)$/, /**/
+ BLOCKING = /^(@@)?(.*?)(\$~?[\w\-]+(?:=[^,\s]+)?(?:,~?[\w\-]+(?:=[^,\s]+)?)*)?$/, /**/
+
+ i, j,
+ startWorkerResults = startWorker({filters: lines, modifiers: {matchWhitelist: true, loosely: true}}, false, true),
+ resultByWhitelist = {},
+ resultByFilter = {};
+
+ // Show all whitelist rules, including the non-matching ones
+ lines = lines.split("\n");
+ for (i=0; i=0; j--) {
+ if (splitted[j].match(BLOCKING)[1]) {
+ splitted.splice(i, 1);
+ }
+ }
+ }
+ resultByWhitelist[i] = resultByWhitelist[i].concat(splitted);
+ for (j=0; j foundRule.length) {
+ foundRule = previousFoundRule;
+ }
+ if (B_REGEX.test(foundRule)) {
+ foundRule = foundRule + "*";
+ } else if (foundRule[0] === "!" || (foundRule[0] === "@" && foundRule[1] === "@")) {
+ foundRule = "*" + foundRule;
+ }
+ if (H_RELATIVEPATH.test(foundRule)) {
+ foundRule = foundRule.replace(H_RELATIVEPATH, "/");
+ if (foundRule === "/") {
+ foundRule = previousFoundRule || "";
+ }
+ }
+ } else {
+ priority = 6;
+ }
+ }
+ if (foundRule) {
+ result[line] = {
+ priority: priority,
+ newRule: (foundRule + (options.length ? "$" + options.sort().join(",") : "")).replace(WHITESPACE_G, "")
+ };
+ }
+ };
+
+ lines = lines.split("\n");
+ for (i=0; i
+
+
+
+
+
+ Redundancy check - limitations
+
+
+
+
Limitations
+
This page lists the limitations of this redundancy checker, as far as known by the author
+
Known false positives
+
+ A false positive is a false alarm: a rule is reported as being redundant, while it actually isn't redundant.
+
+
There are currently no false positives known.
+
+
+
+
Known false negatives
+
+ A false negative is a rule that is redundant (and can be detected as such with the given input), but isn't reported
+
+
+ The options $third-party and $domain=... are not linked
+ As a result, the redundancy checker won't find matches between rules of which one contains $third-party (or $~third-party) and the other one contains a domain that does match the third- or first-party requirement. Examples:
+
+
||site.com/ads.$~third-party should have been made redundant by /ads.$domain=site.com as both only apply to domain site.com
+
||site.com/ads.$domain=anothersite.com should have been made redundant by /ads.$third-party as both apply on domain anothersite.com
+
||site.com^$domain=~site.com should have been made redundant by ||site.com^$third-party as both apply everywhere except for domain site.com
+
+ The cause is that it is not trivial to extract the top level domain. For example: in the case of subdomain.domain.com this would be com, but in case of subdomain.domain.co.uk this would be co.uk.
+
+
+ Complex regular expression rules are only checked one-way
+ As a result, you will not be notified when a regular expression has been made redundant by another rule. Examples:
+
+
/advert(isement)?/ should have been made redundant by /ad(v|s)/
+
/-advert-images?-/ should have been made redundant by -advert-
+
+ The cause is that it is almost impossible to rewrite complex expressions in such a form that it can be checked against all possibilities. For example, the simple regex /\w{3}/ matches every possible filter of at least three successive alphanumeric characters. The regex /ad(s+|v)?/ matches ad, adv, ads, adss, adsss, and so on. It is close to impossible to check this against a normal filter or another regex within the duration of a day computing time and without the need for very complex code.
+
+
+
+
+
Ignored rules
+
+ Ignored rules are rules that the redundancy checker skips before even trying to find matches. Say, false negatives without the need for a second filter.
+
+
+ Rules with syntax errors
+ Rules for which it cannot be determined how to interpret them, are ignored. When such a rule is found, an error will be shown. Examples:
+
+
###.ads
+
/advert(ising/
+
+
+
+ Simplified hiding rules
+ The old, deprecated format of hiding rules is not supported by the tool. When such a rule is found, a warning will be shown telling you how to convert this type of rule to the CSS3 format, if possible. Examples:
+
+
#div
+
#div(ads)
+
#div(class=ads)
+
+
+
+ CSS4 hiding rules
+ CSS4 is still work-in-progress and is therefore subject to changes. Furthermore, at this moment most browsers don't fully support all functionality. For that reason, CSS4 selectors are not supported yet. Examples:
+
+
##[name*="ads" i]
+
##div! > #ads
+
+
+
+ Rules with unknown options
+ Rules with unknown options are usually the result of typos. However, it could also be an option that has been removed from or has recently been added to Adblock Plus. As it is unknown what the default status of those rules is ($third-party defaults to third- and first-party, while $popup defaults to disabled), those rules are skipped to prevent incorrect matches. A warning will be shown. Examples:
+
+
$first-party
+
$donottrack
+
$~domain=site.com
+
$domain=site.com,anothersite.com
+
+
+
+ Rules containing the $sitekey=... option
+ Rules containing the $sitekey= option are silently ignored. The behaviour of those rules is poorly documented, making it hard to determine the behaviour when this rule is combined with other options. Furthermore, there is only one filter list that uses them. Example:
+
+
@@$sitekey=abcdefghijklmnopqrstuvwxyz
+
+
+
+ Rules containing the $generichide or $genericblock options
+ Rules containing the $generichide or $genericblock options are silently ignored. These rules are not yet officially released and their behaviour may still change. Example:
+
+
@@$generichide
+
+
+
+ Rules with problematic domains
+ A rule will also be ignored if the domains of a rule are invalid or if a domain is included and excluded at the same time. Warnings are shown in those cases. Examples:
+
+
$domain=site.com|~site.com
+
$domain=site..com
+
$domain=site.com,domain=anothersite.com
+
site.com,,anothersite.com###ads
+
site.com/###ads
+
*.com###ads
+
+
+
+ Rules containing JavaScript object properties
+ Rules that contain JavaScript object properties may interfere with the code, thereby changing its behaviour. For that reason, those rules are silently ignored. Examples:
+
+
constructor##abc
+
$domain=~con.constructor
+
__proto__
+
+
+
+ Rules containing :nth-child(An+B), :nth-last-child(An+B), :nth-of-type(An+B) or :nth-last-of-type(An+B) with A or B very large
+ Due to rounding errors in JavaScript with large numbers, the values of A and B may differ from the values in the filter. You'll probably never use them anyway, so they are silently ignored. Examples:
+
+
##:nth-child(590295810358705700001)
+
##:nth-of-type(10000000000000000000000000n+4)
+
+
+
+ Rules containing [Adblock] or [Adblock *]
+ Filter lists have to start with a line containing [Adblock] or similar in order to be a valid filter list. As this redundancy checker should also be able to operate with multiple filter lists as input, it is not possible to only filter out the first line, as the second occurrence of this line can indicate the next filter list. Therefore, all rules of this type are silently ignored. Example:
+
+
[Adblock Plus 2.0]
+
+
+
+
+ Rules that cannot match anything
+ If it can be determined on beforehand that a rule cannot match anything, then that rule will be ignored. Technically, those rules can be made redundant by every filter, but that would only be confusing, so a warning is thrown instead. Examples:
+
This script checks if there are any redundant rules in your Adblock Plus filter list. A rule is redundant if the content it blocks, hides or excludes is also blocked, hidden or excluded by another rule. For example, ||somesite.com/ads/ has been made redundant by /ads/*. All calculations are performed on your computer, no data is send to the server.
Copyright (C) 2013-2015 Famlam
+ This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+ This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+ You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.
+
This tool aims to help keep the quality of the filter list high, by removing unnecessary filters and warning about potential issues. When using this tool, one should keep in mind that the results of this tool may not be the best results for the purpose of the filter list, and one should thus always proceed with caution and carefully evaluate the suggestions made.
+
No data you enter on this webpage will be send to any server; all calculations are performed on your local computer
+
If you find an issue or want to suggest an improvement, please report it via one of the contact addresses. Don't forget to include which browser you use, what filters you were trying to validate, any error messages that occured and any steps necessary to reproduce the issue. A list of known limitations is available here.
+
+
+
+
+
\ No newline at end of file
diff --git a/famlam-redundancychecker/redundantRuleChecker.js b/famlam-redundancychecker/redundantRuleChecker.js
new file mode 100644
index 000000000..4d3b73b44
--- /dev/null
+++ b/famlam-redundancychecker/redundantRuleChecker.js
@@ -0,0 +1,797 @@
+"use strict";
+var toolsFile;
+var createText = function(text, appendTo) {
+ var el = document.createTextNode(text.replace(/\ {2,}|\ $|^\ /g, function(m) {
+ return new Array(m.length + 1).join("\u00A0");
+ }));
+ if (appendTo) {
+ appendTo.appendChild(el);
+ }
+ return el;
+};
+
+var createTag = function(tagname, attributes, properties, appendTo) {
+ var i, el = document.createElement(tagname);
+ if (attributes) {
+ for (i in attributes) {
+ el.setAttribute(i, attributes[i]);
+ }
+ }
+ if (properties) {
+ for (i in properties) {
+ if (i === "textContent") {
+ createText(properties[i], el);
+ } else {
+ el[i] = properties[i];
+ }
+ }
+ }
+ if (appendTo) {
+ appendTo.appendChild(el);
+ }
+ return el;
+};
+
+var switchTab = function(e) {
+ var i, currentActive = document.querySelectorAll(".activeTab");
+ for (i=0; i index) {
+ tabtitlebar.insertBefore(title, tabtitlebar.children[index]);
+ } else {
+ tabtitlebar.appendChild(title);
+ }
+ content.id = "tab_" + id;
+ document.getElementById("tab_contents").appendChild(content);
+};
+
+var removeTabs = function(id) {
+ var i, tabIds = id ? [document.getElementById("tabtitle_" + id)] : document.querySelectorAll("[id^='tabtitle_']");
+ for (i=0; i results[b].priority ? -1 : 1;
+ }
+ return a > b ? -1 : 1;
+ });
+ for (i=0; i b ? 1 : -1;
+ }
+ return results[a].length > results[b].length ? -1 : 1;
+ });
+ for (i=0; i -1;
+};
+var chromeListener = function(msg) {
+ var el, i, tr = document.querySelector("tr.pendingDomainCheck[data-domain='" + msg.domain + "']"), top;
+ if (msg.installed) {
+ el = document.querySelector(".extensionMissing");
+ if (msg.incognito) {
+ el.parentElement.removeChild(el);
+ } else {
+ el.textContent = "You need to provide the extension access to incognito pages for it to function correctly. The extension uses incognito pages to determine whether an URL is live, so that no unwanted websites will end up in your search history or leave cookies behind. Furthermore, this way it can fully disable plugins on the websites it opens to prevent security issues, without comprimising your normal browsing on regular tabs. Please visit about://extensions and check the box to allow the extension access to " + (isOpera() ? "private" : "incognito") + " mode. ";
+ var a = createTag("a", {href: "#", title: "Retry!"}, {textContent: "Retry!"}, el);
+ a.addEventListener("click", function(e) {
+ e.preventDefault();
+ chromeConnect(chromeConnect.maxIndent, true);
+ }, false);
+ }
+ } else if (msg.cancelled) {
+ tr = document.querySelectorAll("tr.pendingDomainCheck");
+ for (i=0; i 0) {
+ els = document.querySelectorAll("tr.pendingDomainCheck > td[data-indent='" + maxIndent + "']");
+ } else if (maxIndent === 0) {
+ els = document.querySelectorAll("tr.pendingDomainCheck > td:first-child");
+ } else {
+ document.getElementById("domainCheckProgress").classList.add("hidden");
+ return;
+ }
+ for (j=0; j 0) {
+ tline = createTag("tr", {"data-domain": subdomains[0], "data-parent": results[subdomains[0]].parentDomain, "class": "pendingDomainCheck"}, {});
+ var parentRow = ttype.querySelector("tr[data-domain='" + results[subdomains[0]].parentDomain + "']");
+ parentRow.parentNode.insertBefore(tline, parentRow.nextSibling);
+ subdomains = subdomains.concat(results[subdomains[0]].subdomains.reverse());
+ createTag("td", {"data-indent": Number(parentRow.firstChild.dataset.indent || 0) + 1}, {textContent: subdomains[0]}, tline);
+ maxIndent = Math.max(Number(parentRow.firstChild.dataset.indent || 0) + 1, maxIndent);
+ createTag("td", {}, {textContent: "pending..."}, tline);
+ createTag("td", {"class": "filterCell"}, {textContent: results[subdomains[0]].filters.join("\n")}, tline);
+ subdomains.shift();
+ }
+
+ var trs = ttype.querySelectorAll("tr"), topDomain;
+ for (i=0; i " + errors[i];
+ }
+ output += "\n\n\n! Involved filters:\n\n" + filters.sort().filter(function(elem, i, arr) {return elem !== arr[i+1]}).join("\n");
+ } else {
+ output = document.querySelector("#tab_Domaincheck .toolsResults .emptynotifier").textContent;
+ }
+ window.open("data:text/plain;charset=utf-8," + encodeURIComponent("! Redundancy check domain check results:\n" + output).replace(/\'/g, "%27"));
+ }, false);
+ });
+};
+
+
+var startToolWhitelists = function() {
+ if (document.getElementById("tab_WhitelistsPerRule")) {
+ switchTab("WhitelistsPerRule");
+ return;
+ }
+ startWorker("findWhitelistRules.js", {filters: toolsFile}, function(e) {
+ var i, results = e.resultByFilter;
+ var tabTitle = createTag("span", {"class": "toolsTab"}, {textContent: "whitelists per rule"});
+ var tabContent = createTag("div");
+ createTag("p", {}, {textContent: "The following rules have one or more matching whitelists"}, tabContent);
+ createTag("em", {}, {textContent: "Please note: this is only an indication! If the rule isn't litterally included in the whitelist, it won't be found!"}, tabContent);
+ createTag("br", {}, {}, tabContent);
+ var div = createTag("div", {"class": "toolsResults"}, {}, tabContent);
+ var b, j, pre;
+ var clickhandler = function() {
+ this.nextElementSibling.classList.toggle("hidden");
+ };
+ var keys = Object.keys(results);
+ keys.sort(function(a, b) {
+ if (results[a].length === results[b].length) {
+ return a > b ? 1 : -1;
+ }
+ return results[a].length > results[b].length ? -1 : 1;
+ });
+ for (i=0; i b ? 1 : -1;
+ }
+ return results[a].length > results[b].length ? -1 : 1;
+ });
+ for (i=0; i 1) {
+ p = createTag("p", {}, {}, tabContent);
+ createText("The following " + warnings.length + " ", p);
+ createTag("span", {"class": "majorwarning"}, {"textContent": "errors"}, p);
+ createText(", warnings or ", p);
+ createTag("span", {"class": "minorwarning"}, {"textContent": "optimalizations"}, p);
+ createText(" were encountered while checking the rules:", p);
+ } else if (warnings.length === 1) {
+ p = createTag("p", {}, {}, tabContent);
+ createText("The following ", p);
+ createTag("span", {"class": "majorwarning"}, {"textContent": "error"}, p);
+ createText(", warning or ", p);
+ createTag("span", {"class": "minorwarning"}, {"textContent": "optimalization"}, p);
+ createText(" was encountered while checking the rules:", p);
+ } else {
+ createTag("p", {}, {textContent: "No errors or warnings were encountered while checking the rules"}, tabContent);
+ }
+ major = createTag("span", {"class": "majorwarning"}, {}, tabContent);
+ minor = createTag("span", {"class": "minorwarning"}, {});
+ for (i=0; i\+\~\,]\s*|./gi,
+ WHITESPACE_G = /\s+/g, /**/
+ B_REGEX = /^\/.+\/$/, /**/
+ B_BLOCKINGKEYS = /^\|\||\%[0-9a-f]{2}|[a-z]+|[0-9]+|[^\*\^]|\^\*/g,
+
+ cat_elemhide = {},
+ cat_block = {},
+
+ currentChecks = 0,
+ maxChecks = 0,
+ nextReport = 0,
+ reportProgress = function(n) {
+ currentChecks += n;
+ nextReport -= n;
+ if (nextReport < 1) {
+ nextReport = maxChecks;
+ self.postMessage({progress: Math.round(currentChecks / maxChecks) / 2});
+ }
+ };
+
+ var isEmptyObject = function(obj) {
+ var i;
+ for (i in obj) {
+ return false;
+ }
+ return true;
+ };
+
+ var get_keys_block = function(line) {
+ var newLine, i, j,
+ match = line.replace(WHITESPACE_G, "").match(BLOCKING),
+ keys = [],
+ singlekeys = [];
+ if (B_REGEX.test(match[2])) {
+ match[2] = match[2].substring(1, match[2].length - 1);
+ } else {
+ match[2] = match[2]
+ .replace(MANYSTARS_G, "*")
+ .replace(B_USELESSFILTEREND, "*")
+ .replace(B_USELESSFILTERSTART, "*");
+ match[2] = match[2]
+ .replace(B_STARTWILDCARDPIPE_G, "**|")
+ .replace(B_ENDWILDCARDPIPE_G, "|**")
+ .replace(B_USELESSSTAR_G, "");
+ }
+ newLine = (match[1] || "") + (match[2] || "") + (match[3] ? "$" : "");
+ if (match[1]) {
+ keys.push("@@");
+ }
+ if (match[2]) {
+ singlekeys = match[2].toLowerCase().match(B_BLOCKINGKEYS);
+ for (i=0; ii; j--) {
+ keys.push(singlekeys.slice(i, j).join(""));
+ }
+ }
+ }
+ if (match[3]) {
+ keys.push("$");
+ singlekeys = match[3].substring(1).split(",");
+ for (i=0; i b ? 1 : -1;
+ }
+ return b.length-a.length < 0 ? -1 : 1;
+ });
+ };
+ var get_keys_hide = function(line) {
+ var singlekeys, i, j,
+ match = line.match(ELEMHIDE),
+ keys = [];
+ match[1] = match[1].replace(WHITESPACE_G, "").toLowerCase();
+ match[2] = match[2] ? "#@#" : "##";
+ if (match[1]) {
+ keys.push(match[1]);
+ }
+ keys.push(match[2]);
+ singlekeys = match[3].trim().match(H_HIDINGKEYGROUPS);
+
+ for (i=0; ii; j--) {
+ keys.push(singlekeys.slice(i, j).join(""));
+ }
+ }
+ return keys.sort(function(a, b) {
+ if (a.length === b.length) {
+ return a > b ? 1 : -1;
+ }
+ return b.length-a.length < 0 ? -1 : 1;
+ });
+ };
+
+ var tresholdcache = [9, 9, 9, 9, 9, 4, 5, 5, 6, 6];
+ var get_treshold = function(length) {
+ if (tresholdcache.length > length) {
+ return tresholdcache[length];
+ }
+ var i, sqrt;
+ for (i=tresholdcache.length; i 0 && rule[i] === rule[i-1]) {
+ equal += prevScore; // prevScore <= 0
+ continue;
+ }
+ prevScore = 0;
+ if (!similarRule.contains(rule[i])) {
+ equal -= 1;
+ prevScore = -1;
+ if (equal < treshold) {
+ return 0;
+ }
+ }
+ }
+ equal = equal/rule.length;
+ return equal >= 0.6 ? equal : 0;
+ };
+
+ for (i=0; i
+
+
+
+Ohai!
+
+
+