View Source MopURL.Rule (MopURL v0.1.0)

Transformation rule specification for removing URL elements

Follows https://docs.clearurls.xyz/1.23.0/specs/rules/

Example rule format:

{
    "urlPattern": "^https?:\/\/(?:[a-z0-9-]+\.)*?doubleclick(?:\.[a-z]{2,}){1,}",
    "completeProvider": false,
    "rules": [],
    "referralMarketing": [],
    "rawRules": [],
    "exceptions": [],
    "redirections": [
        "^https?:\/\/(?:[a-z0-9-]+\.)*?doubleclick(?:\.[a-z]{2,}){1,}\/.*?tag_for_child_directed_treatment=;%3F([^&]*)"
    ],
    "forceRedirection": false
}

urlpattern

urlPattern

The urlPattern is a regular expression, that must match every URL that should be affected by the specified rules, exceptions, or redirections of the provider. The typical structure is

<protocol><subdomain><domain name><TLD><directorie>?<fields>

In most cases the following expression covers all the needs, you only have to substitute the remaining <> fields: ^https?://(?:[a-z0-9-]+\.)*?<domain name>\.<TLD>. If you want to match with every TLD, you can substitute the TLD field with (?:[a-z]{2,}){1,}.

completeprovider

completeProvider

The completeProvider is a boolean, that determines if every URL that matches the urlPattern will be blocked. If you want to specify rules, exceptions, and/or redirections, the value of completeProvider must be false.

rules

rules

The rules property is a JSON array. Every element in this array will be automatically rewritten to (?:&amp;|[/?#&])(?:{==<field name>==}=[^&]*) to match the field name. The field name can still be a regular expression, but don't have to. If ClearURLs found a field that matches a rule in a given URL the field will be deleted. Only URLs that match the url Pattern will be checked for matching fields.

rawrules

rawRules

Because other elements in a URL that should be filtered, besides "well-formed fields" aka normal rules, there is also the rawRules property. Regular expressions formulated in this property can refer to the entire URL and not just to individual fields. They are therefore applied directly to the URL.

referralmarketing

referralMarketing

Since version 1.9.0, you can allow referral marketing in ClearURLs. Previously these fields were always removed. Since some people want to support other people through referral marketing, referral marketing can be allowed in the settings. If referral marketing is allowed, the regular expressions in this array are no longer filtered.

Referral Marketing is disabled by default.

exceptions

exceptions

The exceptions property is also a JSON array. Every element in this array is a regular expression, that matches a URL. If ClearURLs found a URL, that matches an exception, no further processing on this URL is done.

redirections

redirections

The redirections property is a JSON array. Every element in this array is a regular expression, that matches a URL and must follow a specification. The first regular expression group must be the URL that should be redirected. If ClearURLs find a URL that matches the redirection, the first matching group will be decoded with will decodeURIComponent() and replaces the old URL.

forceredirection

forceRedirection

Some websites, such as Google, have manipulated URLs (<a> tags) in such a way that the URL is no longer called normally, rather via a built-in script. The result is that a simple redirect of the URL does no longer work. Thus, to still forward the URL by ClearURLs, there is the property forceRedirection which writes the URL into the main_frame object of the browser.

Link to this section Summary

Types

t()

Simple url editing rule

Functions

Every provider has the following properties

Apply a rule.

Link to this section Types

@type t() :: %{
  :urlPattern => Regex.t(),
  :completeProvider => boolean(),
  optional(:rules) => [Regex.t()],
  optional(:rawRules) => [Regex.t()],
  optional(:referralMarketing) => [Regex.t()],
  optional(:exceptions) => [Regex.t()],
  optional(:redirections) => [Regex.t()],
  optional(:forceRedirection) => boolean()
}

Simple url editing rule

Link to this section Functions

Every provider has the following properties:

  • urlPattern (required) - match pattern
  • completeProvider (required) - blacklist
  • rules (optional) - transformation rule
  • rawRules (optional) - transformation rule
  • referralMarketing (optional) - transformation rule
  • exceptions (optional) - match pattern exception
  • redirections (optional) - redirection bypass transformation rule
  • forceRedirection (optional) - are tracking elements inserted with js?

Apply a rule.

Note: rule might return a malformed url. Use MopURL.ValidateURL.cast/1 and MopURL.ValidateURL.validate_hostname/1 to make sure that result is still an URL.