View Source MopURL.Rule (MopURL v0.1.0)
Transformation rule specification for removing URL elements
Follows https://docs.clearurls.xyz/1.23.0/specs/rules/
Example rule format:
{
"urlPattern": "^https?:\/\/(?:[a-z0-9-]+\.)*?doubleclick(?:\.[a-z]{2,}){1,}",
"completeProvider": false,
"rules": [],
"referralMarketing": [],
"rawRules": [],
"exceptions": [],
"redirections": [
"^https?:\/\/(?:[a-z0-9-]+\.)*?doubleclick(?:\.[a-z]{2,}){1,}\/.*?tag_for_child_directed_treatment=;%3F([^&]*)"
],
"forceRedirection": false
}
urlpattern
urlPattern
The urlPattern is a regular expression, that must match every URL that should be affected by the specified rules, exceptions, or redirections of the provider. The typical structure is
<protocol><subdomain><domain name><TLD><directorie>?<fields>
In most cases the following expression covers all the needs, you only have to substitute the remaining <>
fields: ^https?://(?:[a-z0-9-]+\.)*?<domain name>\.<TLD>
. If you want to match with every TLD,
you can substitute the TLD field with (?:[a-z]{2,}){1,}
.
completeprovider
completeProvider
The completeProvider is a boolean, that determines if every URL that matches the urlPattern will be blocked.
If you want to specify rules, exceptions, and/or redirections, the value of completeProvider must be false
.
rules
rules
The rules property is a JSON array. Every element in this array will be automatically rewritten to
(?:&|[/?#&])(?:{==<field name>==}=[^&]*)
to match the field name
.
The field name
can still be a regular expression, but don't have to.
If ClearURLs found a field that matches a rule in a given URL the field will be deleted.
Only URLs that match the url Pattern will be checked for matching fields.
rawrules
rawRules
Because other elements in a URL that should be filtered, besides "well-formed fields" aka normal rules, there is also the rawRules property. Regular expressions formulated in this property can refer to the entire URL and not just to individual fields. They are therefore applied directly to the URL.
referralmarketing
referralMarketing
Since version 1.9.0, you can allow referral marketing in ClearURLs. Previously these fields were always removed. Since some people want to support other people through referral marketing, referral marketing can be allowed in the settings. If referral marketing is allowed, the regular expressions in this array are no longer filtered.
Referral Marketing is disabled by default.
exceptions
exceptions
The exceptions property is also a JSON array. Every element in this array is a regular expression, that matches a URL. If ClearURLs found a URL, that matches an exception, no further processing on this URL is done.
redirections
redirections
The redirections property is a JSON array. Every element in this array is a regular expression, that matches a URL and
must follow a specification.
The first regular expression group must be the URL that should be redirected.
If ClearURLs find a URL that matches the redirection, the first matching group will be decoded with will decodeURIComponent()
and
replaces the old URL.
forceredirection
forceRedirection
Some websites, such as Google, have manipulated URLs (<a>
tags) in such a way that the URL is no
longer called normally, rather via a built-in script. The result is that a simple redirect of the URL does no longer work.
Thus, to still forward the URL by ClearURLs, there is the property forceRedirection
which writes the URL into the main_frame
object of the browser.
Link to this section Summary
Link to this section Types
@type t() :: %{ :urlPattern => Regex.t(), :completeProvider => boolean(), optional(:rules) => [Regex.t()], optional(:rawRules) => [Regex.t()], optional(:referralMarketing) => [Regex.t()], optional(:exceptions) => [Regex.t()], optional(:redirections) => [Regex.t()], optional(:forceRedirection) => boolean() }
Simple url editing rule
Link to this section Functions
Every provider has the following properties:
urlPattern
(required) - match patterncompleteProvider
(required) - blacklistrules
(optional) - transformation rulerawRules
(optional) - transformation rulereferralMarketing
(optional) - transformation ruleexceptions
(optional) - match pattern exceptionredirections
(optional) - redirection bypass transformation ruleforceRedirection
(optional) - are tracking elements inserted with js?
Apply a rule.
Note: rule might return a malformed url.
Use MopURL.ValidateURL.cast/1
and MopURL.ValidateURL.validate_hostname/1
to make sure that result is still an URL.