Syntax Reference

Rules

Rules are defined under a rules object.

Field	Required?	Description
`rules`	Yes	The list of rule definitions. Each rule embeds one or more `rule`.

tip

Clone https://github.com/prequel-dev/preq to follow along, copy/paste, and try the examples referenced below in the repo to learn the simple Prequel rules syntax. All of the examples below are located here.

Rule

A rule describes a network of conditions that must be true across one or more abstract data source. A rule is either a collection or an ordered list of conditions. Collections are called a set and ordered lists are called a sequence.

note

Conditions are literal strings, jq queries (for structured data), or regular expressions.

Field	Required?	Description
`sequence`	Yes	An ordered collection of two or more positive conditions and/or zero or more negative conditions where order matters and repetitions (duplicates) are allowed, e.g. A followed by B followed by and C. Positive conditions must be described in an `order` object.
`set`	Yes	A collection of one or more distinct positive and/or negative conditions forming a group, e.g. A, B, and C in any order with no duplicates. Positive conditions must be described in a `match` object.

Rules Document

examples/00-rules-document-example.yaml
rules:
  - cre:
      id: CRE-2025-0000
    metadata:
      id: mC5rnfG5qz4TyHNscXKuJL
      hash: cBsS3QQY1fwPVFUfYkKtHQ
    rule:
      set:
        window: 5s
        event:
          source: cre.log.kafka
        match:
          - commonExpression1
          - "this is another match"
  - cre:
      id: CRE-2025-0001
    metadata:
      id: nC5rnfG5qz4TyHNscXKuJL
      hash: dBsS3QQY1fwPVFUfYkKtHQ
    rule:
      sequence:
        window: 10s
        event:
          source: cre.log.kafka
        order:
          - regex: "foo(.+)bar"
          - regex: "bo(.+)r"
          - value: "some other match"

Add the rule document on the command line or in the configuration file to use the new rules.

examples/00-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:38 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to baaz
2019/02/05 12:07:41 [emerg] 1655#1655: bind() foo bar
2019/02/05 12:07:42 [emerg] 1655#1655: bind() boooor
2019/02/05 12:07:43 [emerg] 1655#1655: bind() some other match
2019/02/05 12:07:44 [emerg] 1655#1655: bind() this is another match
2019/02/05 12:07:45 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:46 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:47 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:48 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat examples/00-example.log | preq -r ./examples/00-rules-document-example.yaml -d

Example output:

Parsing rules           done! [2 rules in 1ms; 2.47K rules/s]
Problems detected       done! [2 in 1ms; 2.43K/s]
Matching lines          done! [12 lines in 1ms; 14.42K lines/s]
Reading stdin           done! [906B in 0s; 2.91MB/s]
cre-2025-0           critical [1 hits @ 2019-02-05T06:07:37-06:00]
cre-2025-1           critical [1 hits @ 2019-02-05T06:07:38-06:00]

Wrote report to preq-report-1743512495.json

preq-report-1743512495.json
[
  {
    "cre": {
      "id": "CRE-2025-0001"
    },
    "hits": [
      {
        "timestamp": "2019-02-05T06:07:41-06:00",
        "entry": "2019/02/05 12:07:41 [emerg] 1655#1655: bind() foo bar"
      },
      {
        "timestamp": "2019-02-05T06:07:42-06:00",
        "entry": "2019/02/05 12:07:42 [emerg] 1655#1655: bind() boooor"
      },
      {
        "timestamp": "2019-02-05T06:07:43-06:00",
        "entry": "2019/02/05 12:07:43 [emerg] 1655#1655: bind() some other match"
      }
    ],
    "id": "CRE-2025-0001",
    "rule_hash": "dBsS3QQY1fwPVFUfYkKtHQ",
    "rule_id": "nC5rnfG5qz4TyHNscXKuJL",
    "timestamp": "2019-02-05T06:07:41-06:00"
  },
  {
    "cre": {
      "id": "CRE-2025-0000"
    },
    "hits": [
      {
        "timestamp": "2019-02-05T06:07:42-06:00",
        "entry": "2019/02/05 12:07:42 [emerg] 1655#1655: bind() boooor"
      },
      {
        "timestamp": "2019-02-05T06:07:44-06:00",
        "entry": "2019/02/05 12:07:44 [emerg] 1655#1655: bind() this is another match"
      }
    ],
    "id": "CRE-2025-0000",
    "rule_hash": "cBsS3QQY1fwPVFUfYkKtHQ",
    "rule_id": "mC5rnfG5qz4TyHNscXKuJL",
    "timestamp": "2019-02-05T06:07:42-06:00"
  }
]

The -d option disables community CREs while testing a new rule to contribute to the problem detection community. See Getting Started for more information.

tip

Try out this example in the playground!

Set

A set is a collection of one or more distinct (unique) positive and (optionally) negative conditions that form a group of matches that can occur in any order.

warning

Duplicate conditions—conditions that match the exact same thing—are not allowed in a set. Conditions in a set must be unique. Use a sequence for rules that require repetition.

Positive conditions must be described in a match object. Negative conditions use a negate object.

One positive condition

Use match to define conditions in a set. A set is used for rules that have only one positive condition.

examples/01-set-single-example.yaml
rules:
  - cre:
      id: set-example
    metadata:
      id: ZRFiu1mDd8eCruq2ZUH9hx
      hash: TzUzLggVQLvReC1mivmkrK
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - regex: "foo(.+)bar"

This set rule will match the following data.

examples/01-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/01-example.log | preq -r ./examples/01-set-single-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.26K rules/s]
Problems detected       done! [1 in 0s; 2.15K/s]
Reading stdin           done! [822B in 0s; 3.10MB/s]
Matching lines          done! [10 lines in 0s; 36.23K lines/s]
set-example          critical [1 hits @ 2019-02-05T06:07:38-06:00]

Multiple positive conditions

More than one positive condition can be used in a set.

examples/02-set-multiple-example-good-window.yaml
rules:
  - cre:
      id: set-example-2
    metadata:
      id: 2BeLA3siEd5Kd2zvbdH2Jq
      hash: vse1eFMii3UcjqQsq1cfdF
    rule:
      set:
        window: 10s
        event:
          source: cre.log.kafka
        match:
          - value: "test"
          - regex: "foo(.+)bar"
          - regex: "b(.+)az"

This set rule will match the following data.

examples/02-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/02-example.log | preq -r ./examples/02-set-multiple-example-good-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.41K rules/s]
Problems detected       done! [1 in 0s; 2.33K/s]
Reading stdin           done! [822B in 0s; 4.11MB/s]
Matching lines          done! [10 lines in 0s; 50.00K lines/s]
set-example-2        critical [1 hits @ 2019-02-05T06:07:38-06:00]

The same rule will also match this data even though the conditions are in a different order:

examples/02-example-out-of-order.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat examples/02-example-out-of-order.log | preq -r ./examples/02-set-multiple-example-good-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.41K rules/s]
Problems detected       done! [1 in 0s; 2.33K/s]
Reading stdin           done! [822B in 0s; 4.11MB/s]
Matching lines          done! [10 lines in 0s; 50.00K lines/s]
set-example-2        critical [1 hits @ 2019-02-05T06:07:38-06:00]

Match `window`

Two or more positive conditions require a match window. A match window describes the maximum amount of time that is allowed to pass between matches of the first and last positive conditions. If one or more conditions matches outside the window, the rule will not fire.

examples/02-set-multiple-example-bad-window.yaml
rules:
  - cre:
      id: set-example-2
    metadata:
      id: EnaMNTt5czTj5kXERzjnCA
      hash: vFLA3MfPRhzKLLDCyUm7HM
    rule:
      set:
        window: 1s
        event:
          source: cre.log.kafka
        match:
          - value: "test"
          - regex: "foo(.+)bar"
          - regex: "b(.+)az"

This set rule will not match the same data as above because the third positive condition occurs outside the rule's new 1s window.

examples/02-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/02-example-out-of-order.log | preq -r ./examples/02-set-multiple-example-bad-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.41K rules/s]
Problems detected       done! [1 in 0s; 2.33K/s]
Reading stdin           done! [822B in 0s; 4.11MB/s]
Matching lines          done! [10 lines in 0s; 50.00K lines/s]

danger

A match window can only be used if there are multiple positive conditions. In order to add a window around a positive and negative condition, use the negate window described below.

Negative conditions

A set may use one or more negative condition to reduce false positive detections (FPs).

examples/03-set-negative-example.yaml
rules:
  - cre:
      id: set-negative
    metadata:
      id: 5zwKegrhCpf86okDM7qVYc
      hash: awacKg539rEvgkEngk679x
    rule:
      set:
        window: 10s
        event:
          source: cre.log.kafka
        match:
          - value: "test"
          - regex: "foo(.+)bar"
          - regex: "b(.+)az"
        negate:
          - already in use

This set rule will not match the following data because of the negative condition highlighted below.

examples/03-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:38 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:41 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/03-example.log | preq -r ./examples/03-set-negative-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.01K rules/s]
Problems detected       done! [0 in 1ms; 0/s]
Reading stdin           done! [725B in 0s; 4.01MB/s]
Matching lines          done! [9 lines in 0s; 49.72K lines/s]

tip

By default, negative conditions cannot occur between the first and last positive condition. This behavior can be modified with negate options.

One positive and one negative condition may be used in a set. However, without additional negate options, the negative condition must occur at the exact same time to fire.

examples/04-set-1x1-example.yaml
rules:
  - cre:
      id: set-1x1
    metadata:
      id: NcHwCP9Kg6qigh3udUQfLK
      hash: 8C6ZmFNfpnHhUoC1jmiC3S
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - value: "test"
        negate:
          - already in use

This set rule will not match the following data because of the negative condition occurs at the same time as the positive condition.

examples/04-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:41 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/04-example.log | preq -r ./examples/04-set-1x1-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.32K rules/s]
Problems detected       done! [0 in 1ms; 0/s]
Reading stdin           done! [822B in 0s; 5.11MB/s]
Matching lines          done! [10 lines in 0s; 60.98K lines/s]

If the negative condition occurs at a different time, the rule will fire.

Common pitfalls

An `order` is not allowed in a `set`

danger

Use match instead. The order term is reserved for describing positive conditions in a sequence.

examples/05-bad-set-example.yaml
rules:
  - cre:
      id: bad-order
    metadata:
      id: UoddCfP6YFxLPpX9BdmYdh
      hash: BhzLAwg1SxZpX3mYUacw2g
    rule:
      set:
        event:
          source: cre.log.kafka
        order: # Use match instead
          - bar

Example:

cat ./examples/*.log | preq -r ./examples/05-bad-set-example.yaml -d

Example output:

Rules error: err="'set' missing 'match'", line=8, col=9, cre_id=bad-order, rule_id=rule-id

One negative not allowed

danger

A single negative condition alone is not allowed in either a set or a sequence.

examples/06-bad-set-example.yaml
rules:
  - cre:
      id: bad-negate
    metadata:
      id: 5QiBd6c1NhVhmfD43TQTUp
      hash: kkqfjvotUX2gPnTuv34rNU
    rule:
      set:
        event:
          source: cre.log.kafka
        negate: # Negates may not be used without positive conditions
          - "foo(.+)bar"

Example:

cat ./examples/*.log | preq -r ./examples/06-bad-set-example.yaml -d

Example output:

Rules error: err="'set' missing 'match'", line=8, col=9, cre_id=bad-negate, rule_id=rule-id

Two positive terms without a `window` not allowed

danger

A window is required for more than one positive term.

examples/07-bad-set-window.yaml
rules:
  - cre:
      id: bad-window
    metadata:
      id: JYRdT16mrXf6WmesK6LzUc
      hash: gWxiVWqJtRNNbaapTrzbxU
    rule:
      set:
        # missing window
        event:
          source: cre.log.kafka
        match:
          - foo
          - bar

Example:

cat ./examples/*.log | preq -r ./examples/07-bad-set-window.yaml -d

Example output:

Rules error: err="invalid window", line=9, col=9, cre_id=bad-window, rule_id=rule-id, file=./examples/07-bad-set-window.yaml

Sequence

A sequence defines two or more positive conditions that must occur in the same order as defined in the order object. A sequence with only one positive condition is not allowed. Use a set instead for a single positive condition.

warning

A single negative condition is not allowed without two or more positive conditions in a sequence.

Multiple positive conditions

examples/08-sequence-example-good-window.yaml
rules:
  - cre:
      id: seq-example-1
    metadata:
      id: YCCUmV8SMuMCaQvTnjXXwm
      hash: BQ8ouGjLv8mPxvFHd2myeA
    rule:
      sequence:
        event:
          source: cre.log.kafka
        window: 10s
        order:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"

This rule matches the following data:

examples/08-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/08-example.log | preq -r ./examples/08-sequence-example-good-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 1.08K rules/s]
Problems detected       done! [1 in 1ms; 1.06K/s]
Reading stdin           done! [736B in 0s; 1.09MB/s]
Matching lines          done! [9 lines in 0s; 13.16K lines/s]
seq-example-1        critical [1 hits @ 2019-02-05T06:07:38-06:00]

Order `window`

Two or more positive conditions in a sequence require an order window. An order window describes the maximum amount of time that is allowed to pass between matches of the first and last positive conditions. If one or more conditions match outside the window, the rule will not fire.

examples/08-sequence-example-bad-window.yaml
rules:
  - cre:
      id: seq-example-2
    metadata:
      id: VPnEyus1uAudzNL8nMaqm2
      hash: MHXNJnG7cYEr8FLrpndsLZ
    rule:
      sequence:
        event:
          source: cre.log.kafka
        window: 1s
        order:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"

This rule does not match the following data because the third positive conditions occurs outside the rule's new 1s window:

examples/08-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/08-example.log | preq -r ./examples/08-sequence-example-bad-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.28K rules/s]
Problems detected       done! [0 in 1ms; 0/s]
Reading stdin           done! [822B in 0s; 5.34MB/s]
Matching lines          done! [10 lines in 0s; 62.89K lines/s]

Negative conditions

examples/09-sequence-negate-example.yaml
rules:
  - cre:
      id: seq-negate
    metadata:
      id: W2wbe3TXRvvpzNMznsmATh
      hash: G2C1EKqxkX6JsD8xNBthMr
    rule:
      sequence:
        event:
          source: cre.log.kafka
        window: 10s
        order:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - already in use

This rule will not fire on the following data because of the highlighted negative conditions that occurs after the first and before the last positive condition.

exmaples/09-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:41 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/09-example.log | preq -r ./examples/09-sequence-negate-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.19K rules/s]
Problems detected       done! [0 in 1ms; 0/s]
Reading stdin           done! [736B in 0s; 5.41MB/s]
Matching lines          done! [9 lines in 0s; 66.67K lines/s]

Common pitfalls

`match` conditions not allowed

danger

A match cannot be used with a sequence. Use order instead.

examples/10-bad-sequence-match-example.yaml
rules:
  - cre:
      id: bad-seq-match
    metadata:
      id: 7DEQGUdv2MDNsPLDABDBq5
      hash: 1LBvYS5ZTGdAY3ApjE3TR6
    rule:
      sequence:
        event:
          source: cre.log.kafka
        window: 10s
        match: # use order instead
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"

Example:

cat ./examples/*.log | preq -r ./examples/10-bad-sequence-match-example.yaml -d

Example output:

Rules error: err="'sequence' missing 'order'", line=8, col=9, cre_id=bad-seq-match, rule_id=rule-id

One positive condition not allowed

danger

A sequence with only one condition is not supported.

examples/11-bad-sequence-one-condition.yaml
rules:
  - cre:
      id: bad-seq-one-condition
    metadata:
      id: 9fgGD7QxhnvFsMH4coDtET
      hash: tNyxbKu9cDyqDyvkcS8nv8
    rule:
      sequence:
        event:
          source: cre.log.kafka
        order:
          - regex: "foo(.+)bar" # one condition not allowed

Example:

cat ./examples/*.log | preq -r ./examples/11-bad-sequence-one-condition.yaml -d

Example output:

Rules error: err="sequences require two or more positive conditions", line=8, col=9, cre_id=bad-seq-one-condition, rule_id=rule-id, file=./examples/11-bad-sequence-one-condition.yaml

One negative condition not allowed

danger

A negative condition by itself is not allowed. It always requires relative positive conditions.

examples/12-bad-sequence-one-negate.yaml
rules:
  - cre:
      id: bad-seq-one-negate-condition
    metadata:
      id: AbyQH7oFWG5CoWZr8mpKph
      hash: DmHG6sGA9yarH25oxVFDEZ
    rule:
      sequence:
        event:
          source: cre.log.kafka
        negate:
          - regex: "foo(.+)bar" # one negate condition not allowed

Example:

cat ./examples/*.log | preq -r ./examples/12-bad-sequence-one-negate.yaml -d

Example output:

Rules error: err="'sequence' missing 'order'", line=8, col=9, cre_id=bad-seq-one-negate-condition, rule_id=rule-id

Conditions

Conditions describe how to match patterns that exist in any data source with a timestamp. Positive (or "matching") and negative (or "inverse" or "absent") conditions on data sources can be expressed in one of three ways:

Field	Type	Description
value	`string`	A UTF-8 encoded string that supports unicode characters.
regex	`string`	RE2-style regular expressions
jq	`string`	A `jq` formatted query string

String Values

Strings can be added in quotes using the value field to match this string in the example data below.

examples/13-string-example.yaml
rules:
  - cre:
      id: string-example-1
    metadata:
      id: yGbWBUFtXu7R2hhuNJnJ4k
      hash: r7AM3gA9zeaG3sA7ykCi72
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - value: "[emerg] 1655#1655: still could not bind()"

This will match the highlighted line below.

examples/13-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:443 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to [::]:443 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: still could not bind()
2019/02/05 12:07:41 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/13-example.log | preq -r ./examples/13-string-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.15K rules/s]
Problems detected       done! [1 in 1ms; 1.13K/s]
Reading stdin           done! [807B in 0s; 3.62MB/s]
string-example-1     critical [1 hits @ 2019-02-05T06:07:38-06:00]
Matching lines          done! [9 lines in 0s; 40.18K lines/s]

String values may also be expressed without the value field.

examples/14-string-example.yaml
rules:
  - cre:
      id: string-example-2
    metadata:
      id: pe77KnYc8daXGDnP2nTN2k
      hash: kHuH898oQS8UyroaXb4bf4
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - "[emerg] 1655#1655: still could not bind()"

Example:

cat ./examples/14-example.log | preq -r ./examples/14-string-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 1.70K rules/s]
Problems detected       done! [1 in 0s; 1.64K/s]
Reading stdin           done! [807B in 0s; 2.97MB/s]
string-example-2     critical [1 hits @ 2019-02-05T06:07:38-06:00]
Matching lines          done! [9 lines in 0s; 32.03K lines/s]

Escape Sequences

Use a backslash \ to escape the following special characters in a string value:

\r carriage return CR
\n new line LF
\t tab
\' single quotes '
\" double quotes "
\\ backslash \
\u for 4-digit hexadecimal code points (e.g., \u00E9 for é)
\U for 8-digit hexadecimal code points (e.g., \U0001F600 for 😀)

Regular Expressions

Regular expressions can be added in quotes using the regex field. Regular expressions ignore parts of a condition that may change frequently, like the 1655#1655 process and thread IDs in the example.

examples/15-regex-example.yaml
rules:
  - cre:
      id: regex-example-1
    metadata:
      id: N8YSvSpsMDeuVPoFERDJFT
      hash: iwJNaJ8PzbRf8xwCzkedsR
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - regex: "emerg(.+)still could not bind()"

This regular expression will match the highlighted line below.

examples/15-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:443 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to [::]:443 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: still could not bind()
2019/02/05 12:07:41 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/15-example.log | preq -r ./examples/15-regex-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.89K rules/s]
Problems detected       done! [1 in 0s; 2.78K/s]
Reading stdin           done! [807B in 0s; 5.24MB/s]
regex-example-1      critical [1 hits @ 2019-02-05T06:07:38-06:00]
Matching lines          done! [9 lines in 0s; 55.90K lines/s]

The following regular expression special characters must be escaped using a backslash \:

Regular expression special characters
\.+*?()|[]{}^$`

warning

An escape backslash \ must also be escaped in the Yaml string. This means two backslashes are needed. One for the Yaml string and the other for the regular expression.

16-regex-example.yaml
rules:
  - cre:
      id: regex-example-2
    metadata:
      id: zLbu8UZvrsdhRymhrFpAGr
      hash: tfXp5cmbDHp1fYiHKj5MF1
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - regex: "\\[emerg] (.+) still could not bind()"

This will match the highlighted line below.

16-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:443 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to [::]:443 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: still could not bind()
2019/02/05 12:07:41 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/16-example.log | preq -r ./examples/16-regex-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.10K rules/s]
Problems detected       done! [1 in 1ms; 1.08K/s]
Reading stdin           done! [807B in 0s; 3.77MB/s]
Matching lines          done! [9 lines in 0s; 43.69K lines/s]
regex-example-2      critical [1 hits @ 2019-02-05T06:07:38-06:00]

`jq` Expressions

jq expressions may be used on structured JSON or Yaml data sources.

examples/17-jq-example.yaml
rules:
  - cre:
      id: jq-example-1
    metadata:
      id: Zb8k3w7yYLWxKdy6yYcj8z
      hash: k7SDcit3z8pMY5rqVUs3AC
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - jq: ".event == \"worker_process_crash\""

This jq expression will match the highlighted line below.

examples/17-example.log
{"timestamp":"2025-03-26T14:01:02Z","level":"info","service":"nginx-ingress","namespace":"default","pod":"nginx-ingress-abc123","request_id":"req001","method":"GET","path":"/","status":200,"latency_ms":5,"upstream":"web-service:80","client_ip":"192.168.1.10"}
{"timestamp":"2025-03-26T14:01:03Z","level":"info","service":"nginx-ingress","namespace":"default","pod":"nginx-ingress-abc123","request_id":"req002","method":"GET","path":"/api/status","status":200,"latency_ms":12,"upstream":"api-service:8080","client_ip":"192.168.1.11"}
{"timestamp":"2025-03-26T14:01:04Z","level":"error","event":"worker_process_crash","message":"worker process 4123 exited on signal 9","service":"nginx-ingress","namespace":"default","pod":"nginx-ingress-abc123","worker_pid":4123,"exit_signal":9}
{"timestamp":"2025-03-26T14:01:10Z","level":"warn","service":"nginx-ingress","namespace":"default","pod":"nginx-ingress-abc123","request_id":"req003","method":"POST","path":"/api/login","status":401,"latency_ms":15,"upstream":"auth-service:9000","client_ip":"192.168.1.12"}
{"timestamp":"2025-03-26T14:01:11Z","level":"info","service":"nginx-ingress","namespace":"default","pod":"nginx-ingress-abc123","request_id":"req004","method":"GET","path":"/metrics","status":200,"latency_ms":1,"upstream":"metrics-service:9090","client_ip":"192.168.1.15"}
{"timestamp":"2025-03-26T14:01:12Z","level":"info","service":"nginx-ingress","namespace":"default","pod":"nginx-ingress-abc123","request_id":"req005","method":"GET","path":"/healthz","status":200,"latency_ms":1,"upstream":"health-service:8081","client_ip":"192.168.1.18"}

Example:

cat ./examples/17-example.log | preq -r ./examples/17-jq-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 973 rules/s]
Problems detected       done! [1 in 1ms; 990/s]
Reading stdin           done! [1.60KB in 0s; 7.23MB/s]
jq-example-1         critical [1 hits @ 2025-03-26T09:01:04-05:00]
Matching lines          done! [6 lines in 0s; 26.67K lines/s]

Use more complex jq queries to match on more than one field in the data.

examples/18-jq-example.yaml
rules:
  - cre:
      id: jq-example-2
    metadata:
      id: WUPyNqyy1TEwca5ZnPEqeb
      hash: xM614Hx4jqdZfHbt6U7MJt
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - jq: "select(.event == \"worker_process_crash\" and .level == \"error\")"

Example:

cat ./examples/18-example.log | preq -r ./examples/18-jq-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 850 rules/s]
Problems detected       done! [1 in 1ms; 885/s]
Reading stdin           done! [1.60KB in 0s; 7.23MB/s]
jq-example-2         critical [1 hits @ 2025-03-26T09:01:04-05:00]
Matching lines          done! [6 lines in 0s; 26.09K lines/s]

Counts

Use count to look for more than one occurrence of the same condition within a defined window. This can be used in both sets and sequences.

examples/32-example-condition-count.yaml
rules:
  - cre:
      id: count-example
    metadata:
      id: ZRFiu1mDd8eCruq2ZUH9hx
      hash: TzUzLggVQLvReC1mivmkrK
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - regex: "foo(.+)bar"
            count: 2
        window: 10s

This set rule will match the following data.

examples/01-count-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/32-count-example.log | preq -r ./examples/32-example-condition-count.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.26K rules/s]
Problems detected       done! [1 in 0s; 2.15K/s]
Reading stdin           done! [822B in 0s; 3.10MB/s]
Matching lines          done! [10 lines in 0s; 36.23K lines/s]
count-example          critical [1 hits @ 2019-02-05T06:07:38-06:00]

note

Count may not exceed the maximum limit of 64.

Common pitfalls

Avoid Yaml literal-block scalers `|`

danger

Avoid using Yaml literal-block scalars (e.g. |) in conditions. The literal-block scalar adds a line feed \n at the end of the string.

examples/19-bad-literal-block-example.yaml
rules:
  - cre:
      id: bad-yaml-literal-block
    metadata:
      id: 2o278NtyqfoNacFrGHkm3d
      hash: 59b7HHeeZwvgtyYbe2WaT9
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - regex: |
              "\\[emerg] (.+) still could not bind()"

This will add a \n to the end of the expression. As a result, it will no longer match the example data.

$ cat ./examples/*.log | preq -r ./examples/19-bad-literal-block-example.yaml -d
Parsing rules           done! [1 rules in 0s; 1.88K rules/s]
Problems detected       done! [0 in 0s; 0/s]
Reading stdin           done! [11.95KB in 0s; 47.22MB/s]
Matching lines          done! [105 lines in 0s; 420.00K lines/s]

A regular expression may not be used without using the regex field.

examples/20-bad-regex-example.yaml
rules:
  - cre:
      id: bad-regex-example
    metadata:
      id: A4ZFQZLprRMYH6g9kfiJRe
      hash: B5MsHJocMTimFUzEU1isyP
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - "\\[emerg] (.+) still could not bind()" # use regex: instead

Negative conditions

Negative conditions are used to reduce false positives in a set or sequence. By default, a negative condition cannot occur between the first and last positive condition in a match or an order.

Simple negative condition

examples/21-negative-example.yaml
rules:
  - cre:
      id: negate-example
    metadata:
      id: xv3JdgtRShwMrj8rX5mi6p
      hash: QXooQcifDGLxtZKYYt2Nvb
    rule:
      sequence:
        event:
          source: cre.log.kafka
        window: 10s
        order:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - this is normal

This sequence rule would not match the following data because of the negative condition that occurs between the first and last positive condition in the sequence.

examples/21-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/21-example.log | preq -r ./examples/21-negative-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 948 rules/s]
Problems detected       done! [0 in 1ms; 0/s]
Reading stdin           done! [822B in 0s; 3.01MB/s]
Matching lines          done! [10 lines in 0s; 34.97K lines/s]

The same sequence rule would match the following data because the negative condition happens after the last positive condition.

examples/22-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:45 [emerg] 1655#1655: bind() just kidding false alarm this is normal

Example:

cat ./examples/22-example.log | preq -r ./examples/21-negative-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 1.27K rules/s]
Problems detected       done! [1 in 1ms; 1.20K/s]
Reading stdin           done! [822B in 0s; 1.24MB/s]
Matching lines          done! [10 lines in 0s; 14.99K lines/s]
negate-example       critical [1 hits @ 2019-02-05T06:07:38-06:00]

It would also match this data because the negative condition happens before the first positive condition.

example/23-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:43 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:44 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/23-example.log | preq -r ./examples/21-negative-example.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.22K rules/s]
Problems detected       done! [1 in 1ms; 1.20K/s]
Reading stdin           done! [822B in 0s; 5.04MB/s]
negate-example       critical [1 hits @ 2019-02-05T06:07:38-06:00]
Matching lines          done! [10 lines in 0s; 61.35K lines/s]

To match the last two examples add negate options to the negative conditions.

Multiple negative conditions

More than one negative condition may be used.

examples/24-multiple-negatives.yaml
rules:
  - cre:
      id: multiple-negatives
    metadata:
      id: nNW6fP7yjWafb3DqzWt79t
      hash: zKB3UYeYPjSPophgXf1BhP
    rule:
      sequence:
        event:
          source: cre.log.kafka
        window: 10s
        order:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - FP1
          - FP2
          - FP3

examples/24-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:40 [emerg] FP1
2019/02/05 12:07:40 [emerg] FP2
2019/02/05 12:07:41 [emerg] FP3
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:43 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

This rule should not fire since all three negative conditions exist within the window of positive conditions.

Example:

cat ./examples/24-example.log | preq -r ./examples/24-multiple-negatives.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 1.80K rules/s]
Problems detected       done! [0 in 0s; 0/s]
Reading stdin           done! [918B in 0s; 1.78MB/s]
Matching lines          done! [13 lines in 0s; 24.81K lines/s]

Negate options

Negative conditions provide additional options to match (or reset) the rule. This is helpful in chaotic distributed systems where additional windows of time is needed before, during, or after positive conditions to prevent a rule from firing on a false positive (FP).

Window

The negate window is used to extend or move the amount of time in which a negative condition cannot occur relative to the positive conditions.

The following set rule defines one positive and one negative condition.

examples/25-negate-options-1x1.yaml
rules:
  - cre:
      id: negate-options-1x1
    metadata:
      id: gJeoP8hPeNuGN7XhcJdPKN
      hash: K7z17XgLcbLnpYHtTvaX7X
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - regex: "foo(.+)bar"
        negate:
          - FP1

This rule will not fire on the data below because the negative condition occurs at the same time as the positive condition.

examples/25-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:38 [emerg] FP1
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:43 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/25-example.log | preq -r ./examples/25-negate-options-1x1.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.30K rules/s]
Problems detected       done! [0 in 0s; 0/s]
Reading stdin           done! [918B in 0s; 5.56MB/s]
Matching lines          done! [13 lines in 0s; 77.38K lines/s]

If the negative condition occurs at a different time than the positive condition, the rule will now fire.

examples/25-example-new-negate-time.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] FP1
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:43 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/25-example-new-negate-time.log | preq -r ./examples/25-negate-options-1x1.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.61K rules/s]
Problems detected       done! [1 in 0s; 2.51K/s]
Reading stdin           done! [918B in 0s; 5.67MB/s]
negate-options-1x1   critical [1 hits @ 2019-02-05T06:07:38-06:00]
Matching lines          done! [13 lines in 0s; 78.31K lines/s]

Match Width

The default behavior of a negative condition is it cannot occur between the first and last positive condition. This period of time between the first and last occurrence of a positive condition is referred to as its match width.

tip

The negate option window modifies the default behavior of negative conditions. It extends the time beyond the width of the positive matches. When defined, a negate window adds the specified amount of time to the width of the match that occurred.

examples/26-negate-window.yaml
rules:
  - cre:
      id: negate-window
    metadata:
      id: ymoAXHpdkpSoGgirhVie4d
      hash: Nox2VQXTPAFbuSh5GBVssB
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - regex: "foo(.+)bar"
        negate:
          - value: FP1
            window: 5s

This rule will no longer fire on the data above (26-example.log) because the negative condition cannot happen for up to 5s seconds after the positive condition.

examples/26-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:43 [emerg] FP1
2019/02/05 12:07:46 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/26-example.log | preq -r ./examples/26-negate-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 1.94K rules/s]
Problems detected       done! [0 in 0s; 0/s]
Reading stdin           done! [918B in 0s; 5.16MB/s]
Matching lines          done! [13 lines in 0s; 68.78K lines/s]

This rule no longer matches because:

The positive condition occurs at 12:07:38 on line 5 in 26-example.log
The negative condition occurs 5 seconds later at 12:07:43 on line 8 in 26-example.log

If the negative condition occurs outside of the 5 second negate window, then the rule will fire again.

examples/26-example-moved-negative.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:44 [emerg] FP1 happens 6 seconds later so the rule fires
2019/02/05 12:07:47 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:48 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:49 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)

Example:

cat ./examples/26-example-moved-negative.log | preq -r ./examples/26-negate-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.08K rules/s]
Problems detected       done! [1 in 0s; 2.00K/s]
Reading stdin           done! [960B in 0s; 5.33MB/s]
Matching lines          done! [13 lines in 0s; 70.65K lines/s]
negate-window        critical [1 hits @ 2019-02-05T06:07:38-06:00]

Multiple positive conditions `width`

When more than one positive condition is used with a negate window, the width of the positive matches—the period of time between the first and last match—will be added to the negate window when no other negate options are expressed.

examples/27-negate-window.yaml
rules:
  - cre:
      id: negate-window-2
    metadata:
      id: geap3CjsXBSjoXiEdVuQ3M
      hash: TQETkvbmT8MnSt4X2m1HAa
    rule:
      set:
        event:
          source: cre.log.kafka
        window: 5s
        match:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - value: FP2
            window: 18s

In the example data below:

examples/27-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:43 [emerg] FP1
2019/02/05 12:07:46 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:08:00 [emerg] FP2

The width of these positive conditions is 4s:

foo(.+)bar occurs at 12:07:38
b(.+)az occurs at 12:07:42 4 seconds later

A negate window of 18s means the negative condition cannot occur within 22s after the first positive condition when the width of the positive conditions is 4s and the negate window is an additional 18s.

Because "FP2" occurs 22s after the first positive condition, the rule will not fire.

Example:

cat ./examples/27-example.log | preq -r ./examples/27-negate-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.90K rules/s]
Problems detected       done! [0 in 0s; 0/s]
Reading stdin           done! [886B in 0s; 8.60MB/s]
Matching lines          done! [12 lines in 0s; 108.11K lines/s]

If the width of the positive conditions is smaller, then the rule will fire.

examples/27-example-smaller.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:43 [emerg] FP1
2019/02/05 12:07:46 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:08:00 [emerg] FP2

Because the match width is 3s in the data above, the rule will now fire. The negative condition now cannot occur within 21s of the first positive condition when the match width is 3s and the negate window is an additional 18s. Because "FP2" occurs 22s after the first positive condition, the rule will fire.

Example:

cat ./examples/27-example-smaller.log | preq -r ./examples/27-negate-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.02K rules/s]
Problems detected       done! [1 in 1ms; 1.00K/s]
Reading stdin           done! [886B in 0s; 3.90MB/s]
Matching lines          done! [12 lines in 0s; 52.40K lines/s]
negate-window-2      critical [1 hits @ 2019-02-05T06:07:38-06:00]

Similarly, if the negate window is decreased, then the rule will fire again if the match window stays the same.

examples/27-negate-window-shorter.yaml
rules:
  - cre:
      id: negate-window-shorter
    metadata:
      id: G1VtCxz7hPVwH1LCpJbdZr
      hash: Mp4NrH8nea7oy1VVbMSWvC
    rule:
      set:
        event:
          source: cre.log.kafka
        window: 5s
        match:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - value: FP2
            window: 17s

Example:

cat ./examples/27-example.log | preq -r ./examples/27-negate-window-shorter.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 1.93K rules/s]
Problems detected       done! [1 in 0s; 1.87K/s]
Reading stdin           done! [886B in 0s; 4.57MB/s]
Matching lines          done! [12 lines in 0s; 60.30K lines/s]
negate-window-shorter critical [1 hits @ 2019-02-05T06:07:38-06:00]

This new negate window means the negative condition cannot occur within 21s of the first positive condition when the match width is 4s (in 27-example.log). Because "FP2" occurs 22s after the first positive condition, the rule will fire.

Anchor

The negate option anchor may be used to change the relative positive condition from which a negate window is measured. This is useful when the negative condition has a known dependency on a specific positive term.

Recall the negate window defines the time a negative condition cannot occur as the width of the positive conditions plus the additional negate window. By default, this window of time is relative to the first positive condition (or anchor: 0).

tip

Use anchor to change when the relative negative window begins. The default negate window is relative to the first positive condition (i.e. when anchor: 0).

To change the anchor to start the negate window after the second positive condition:

examples/28-negate-anchor.yaml
rules:
  - cre:
      id: negate-anchor
    metadata:
      id: CQmArNYk7PziNhfsVruuTD
      hash: NN36Tt1K7W8h6s3xKkdVtZ
    rule:
      set:
        event:
          source: cre.log.kafka
        window: 5s
        match:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - value: FP2
            window: 17s
            anchor: 1

examples/28-example.log
2019/02/05 12:07:37 [notice] 1629#1629: signal process started
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:43 [emerg] FP1
2019/02/05 12:07:46 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:08:00 [emerg] FP2

The negative condition cannot occur 21s after the second positive condition (anchor: 1):

The match width of time between the first and last positive condition is 4s
The second positive condition "test" occurs at 12:07:39
4s (match width) + 17s (negate window) = 21s
The negative condition "FP2" occurs 21s later at 12:08:00 (within the new total negate window of 21s)

Because the negative condition occurs within the total negate window anchored on the second positive condition, this rule does not fire.

Example:

cat ./examples/28-example.log | preq -r ./examples/28-negate-anchor.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 945 rules/s]
Problems detected       done! [0 in 1ms; 0/s]
Reading stdin           done! [918B in 0s; 5.46MB/s]
Matching lines          done! [13 lines in 0s; 76.47K lines/s]

Slide

tip

Use the negate option slide to shift the total negate window to the left to define when negative conditions cannot happen before positive conditions.

examples/29-negate-slide.yaml
rules:
  - cre:
      id: negate-slide
    metadata:
      id: 6VYeEGFDDscmfU2ySJFHjj
      hash: CMxGLAXx2J1e44oJxoutvn
    rule:
      set:
        event:
          source: cre.log.kafka
        window: 5s
        match:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - value: FP1
            slide: -8s

This rule moves the negate window to the left of the first positive condition (anchor: 0) by 8s.

examples/29-example.log
2019/02/05 12:07:30 [notice] FP1
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:46 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:08:00 [emerg] FP2

This rule will not fire because the negative condition "FP1" occurs 8s before the first positive condition (within the total negate window that is now shifted to the left by 8s using slide).

Example:

cat ./examples/29-example.log | preq -r ./examples/29-negate-slide.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.44K rules/s]
Problems detected       done! [0 in 0s; 0/s]
Reading stdin           done! [824B in 0s; 5.84MB/s]
Matching lines          done! [11 lines in 0s; 78.57K lines/s]

If anchor is added to slide the negate window on the second positive condition, the rule will fire.

examples/29-negate-slide-anchor-1.yaml
rules:
  - cre:
      id: negate-slide-anchor
    metadata:
      id: 9AMcopuDdzsQJjjxYSBgnt
      hash: mRMBKh5pgxTTpHF7J65rKc
    rule:
      set:
        event:
          source: cre.log.kafka
        window: 5s
        match:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - value: FP1
            slide: -8s
            anchor: 1
            window: 1s

Example:

cat ./examples/29-example.log | preq -r ./examples/29-negate-slide-anchor-1.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.16K rules/s]
Problems detected       done! [1 in 1ms; 1.14K/s]
Reading stdin           done! [824B in 0s; 4.60MB/s]
negate-slide-anchor  critical [1 hits @ 2019-02-05T06:07:38-06:00]
Matching lines          done! [11 lines in 0s; 59.14K lines/s]

The negate option window can be added to increase the total negate window in a slide.

examples/29-negate-slide-anchor-1-window.yaml
rules:
  - cre:
      id: neg-all-opts
    metadata:
      id: XZUx7gkmbkc3BQSadKeSk4
      hash: Vgs91FMjNCPMJF81xUm4UZ
    rule:
      set:
        event:
          source: cre.log.kafka
        window: 5s
        match:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - value: FP1
            slide: -9s
            anchor: 1
            window: 1s

The match width is 4s in 29-example.log
The additional negate window is 1s
The total negate window is 4s + 1s = 5s
A slide of -9s anchored on the second positive condition starts the total negate window of 5s at 12:07:30
A negative condition cannot occur between 12:07:30 and 12:07:35 (+5s) when the second positive condition occurs at 12:07:39

The rule will not fire since the negative condition occurs at 12:07:30.

Example:

cat ./examples/29-example.log | preq -r ./examples/29-negate-slide-anchor-1-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.57K rules/s]
Problems detected       done! [0 in 0s; 0/s]
Reading stdin           done! [824B in 0s; 5.68MB/s]
Matching lines          done! [11 lines in 0s; 74.83K lines/s]

If the negative condition moves to 12:07:36, then the rule will fire because it is now outside of the shifted window before the positive terms.

examples/29-example-fp-moved.log
2019/02/05 12:07:36 [notice] FP1
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:46 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:08:00 [emerg] FP2

Example:

cat ./examples/29-example-fp-moved.log | preq -r ./examples/29-negate-slide-anchor-1-window.yaml -d

Example output:

Parsing rules           done! [1 rules in 1ms; 1.34K rules/s]
Problems detected       done! [1 in 1ms; 1.35K/s]
Reading stdin           done! [824B in 0s; 5.25MB/s]
neg-all-opts         critical [1 hits @ 2019-02-05T06:07:38-06:00]
Matching lines          done! [11 lines in 0s; 69.18K lines/s]

Absolute

tip

Use the negate option absolute to avoid adding the match width to the total negate window and instead use only the negate window specified in negative condition. This is useful when the match width is unpredictable.

examples/30-negate-absolute.yaml
rules:
  - cre:
      id: negate-absolute
    metadata:
      id: Ptu8PAQuvobc3TkWnck1Zo
      hash: fvwHowvd9RPLCvWN6rJH6P
    rule:
      set:
        event:
          source: cre.log.kafka
        window: 5s
        match:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - value: FP2
            window: 20s
            absolute: true

This rule creates absolute window of 20s anchored by default to the first positive term.

examples/30-example.log
2019/02/05 12:07:30 [notice] FP1
2019/02/05 12:07:37 [error] 1629#1629: open() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:37 [emerg] 1655#1655: bind() just kidding false alarm this is normal
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to 0.0.0.0:80 failed (98: Address already in use)
2019/02/05 12:07:38 [emerg] 1655#1655: bind() to foo bar
2019/02/05 12:07:39 [emerg] 1655#1655: bind() to test
2019/02/05 12:07:40 [emerg] 1655#1655: bind() to 0.0.0.0:444 failed (98: Address already in use)
2019/02/05 12:07:41 [emerg] 1655#1655: bind() to [::]:444 failed (98: Address already in use)
2019/02/05 12:07:42 [emerg] 1655#1655: still could not bind() to baaaz
2019/02/05 12:07:46 [alert] 1631#1631: unlink() "/run/nginx.pid" failed (2: No such file or directory)
2019/02/05 12:07:58 [emerg] FP2

Because the negative condition "FP2" occurs 20s after the first positive condition, this rule does not fire.

Example:

cat ./examples/30-example.log | preq -r ./examples/30-negate-absolute.yaml -d

Example output:

Parsing rules           done! [1 rules in 0s; 2.58K rules/s]
Problems detected       done! [0 in 0s; 0/s]
Reading stdin           done! [824B in 0s; 7.85MB/s]
Matching lines          done! [11 lines in 0s; 100.00K lines/s]

Common Pitfalls

Anchor index cannot exceed number of positive conditions

danger

Do not use an anchor index that exceeds the range of positive conditions.

examples/31-bad-negate-anchor.yaml
rules:
  - cre:
      id: negate-bad-anchor
    metadata:
      id: htJPAEEN63dcawBaiMVLG8
      hash: 7CdKXQgLEBTq6WyTs7RXbd
    rule:
      set:
        event:
          source: cre.log.kafka
        window: 5s
        match:
          - regex: "foo(.+)bar"
          - value: "test"
          - regex: "b(.+)az"
        negate:
          - value: FP2
            window: 20s
            anchor: 3 # the range of the three positive anchors is 0, 1, and 2

Example:

cat ./examples/*.log | preq -r ./examples/31-bad-negate-anchor.yaml -d

Example output:

Rules error: err="invalid negate anchor", line=10, col=17, cre_id=negate-bad-anchor, rule_id=rule-id, file=./examples/31-bad-negate-anchor.yaml

Data Sources

A rule defines an abstract data source to target an addressable named data type (e.g. log, metric, trace, or event). The source name can be anything, but by convention CRE data sources are namespaced and begin with cre.. The next name indicates the data type (for preq this is log). Then the target technology, e.g. kafka. Optionally version numbers can be used to target specific versions, e.g. cre.log.kafka.3.*.

examples/13-string-example.yaml
rules:
  - cre:
      id: string-example-1
    metadata:
      id: rule-id
    rule:
      set:
        event:
          source: cre.log.kafka
        match:
          - value: "[emerg] 1655#1655: still could not bind()"

Because the location and address of data sources vary based on the environment (e.g. Kubernetes vs. virtual machines vs. laptops and servers) and distribution (Docker Hub vs. Bitnami vs. Quay.io), CRE rules are created to target a data source abstraction that is translated by the problem detector to the actual source's location in the environment.

The preq problem detector supports targeting log files via stdin (all data sources) or via a data source Yaml that defines the file system location for each abstract data type. Refer to Data Sources to learn how to generate a data source template and customize it for your system.

Example data source Yaml:

./examples/40-sources.yaml
version: 0.0.1
sources:
  - name: example-1
    type: cre.log.rabbitmq
    desc: "Example 1"
    locations:
      - path: ./examples/26-example.log
  - name: example-2
    type: cre.log.django
    desc: "Example 2"
    locations:
      - path: ./examples/25-example.log
  - name: example-3
    type: cre.log.nginx
    desc: "Example 3"
    locations:
      - path: ./examples/28-example.log
  - name: example-4
    type: cre.log.kafka
    desc: "Example 4"
    locations:
      - path: ./examples/23-example.log
      - path: ./examples/29-example.log

Example:

preq -s ./examples/40-sources.yaml -r ./examples/29-negate-slide-anchor-1.yaml

Example output:

Parsing rules           done! [24 rules in 3ms; 4.37K rules/s]
Problems detected       done! [1 in 5ms; 181/s]
Reading example-1       done! [854B in 0s; 359.73KB/s]
Reading example-2       done! [854B in 1ms; 359.43KB/s]
Reading example-3       done! [886B in 0s; 371.80KB/s]
Reading example-4       done! [822B in 2ms; 343.93KB/s]
Matching lines          done! [44 lines in 2ms; 18.34K lines/s]
negate-slide-anchor  critical [1 hits @ 2019-02-05T06:07:38-06:00]

Nested Conditions

One or more nested conditions from another sequence or set may be used as as a condition of another set or sequence. Nested expressions enable complex positive and negative conditions to be created across distributed data sources. It also enables correlations on two or more conditions to test whether they share attributes in common. For example, correlations on conditions by host evaluate to true only when all conditions occur on the same machine.

Nested conditions and correlations are supported in Prequel.

rules:
  - cre:
      id: nested-example
    metadata:
      id: 4t48r67AjGo5QG9yCtgXQq
      hash: RLCmWA91nBLTSdmw5dprkq
    rule:
      sequence:
        window: 30s
        correlations:
          - hostname
        order:
          - sequence:
              window: 10s
              event:
                source: cre.log.rabbitmq
                origin: true
              order:
                - value: Discarding message
                  count: 11
                - Mnesia overloaded
              negate:
                - SIGTERM
          - sequence:
              window: 5s
              correlations:
                - container_id
              order:
                - sequence:
                    window: 1s
                    event:
                      source: cre.log.nginx
                    order:
                      - error message
                      - shutdown
                - set:
                    event:
                      source: cre.log.nginx
                    match:
                      - 90%
                - set:
                    event:
                      source: cre.prequel.k8s
                    match:
                      - field: "reason"
                        value: "Killing"
        negate:
          - set:
              event:
                source: cre.prequel.k8s
              match:
                - field: "reason"
                  value: "NodeShutDown"

The rule above detects the following:

At the top level the rule looks for two positive conditions and resets on one negative condition.
- The first positive condition is a sequence of 11 positive conditions and 1 negative condition in rabbitmq.
- The second positive condition is another sequence of three positive conditions.
  - The first positive condition is a sequence in nginx of two positive conditions, error message and shutdown.
  - The second positive condition is a set in nginx that matches on 90%
  - The third positive condition is another set that matches a Kubernetes Killing event
  - The correlation on these conditions means all three positive conditions must share the same container_id to match
- The top level rule also looks for one negative condition
  - A Kubernetes NodeShutdown event

Origin

When using nested conditions, the rule author must specify which expression on an abstract data source constitutes the origin event that will take the blame for the overall detection. The origin is implied when nested conditions are not used since the rule is only targeting one data source.

Correlations

A correlation is used in nested conditions to add additional criteria to the match. The following fields may be used in a correlation expression:

Field	Description
imageUrl	The container's `image_url`
containerName	The name of the container
containerId	The container's unique container runtime ID
podName	The name of the Kubernetes pod
filepath	The path to the data on the file system
processName	The name of the process
namespace	The Kubernetes `namespace`
machineId	The host's unique machine ID (see https://man7.org/linux/man-pages/man5/machine-id.5.html)
hostname	The fully qualified DNS name of the machine
workload	The name of the Kubernetes service, statefulset, daemonset, or deployment

Rules​

Rule​

Rules Document​

Set​

One positive condition​

Multiple positive conditions​

Match window​

Negative conditions​

Common pitfalls​

An order is not allowed in a set​

One negative not allowed​

Two positive terms without a window not allowed​

Sequence​

Multiple positive conditions​

Order window​

Negative conditions​

Common pitfalls​

match conditions not allowed​

One positive condition not allowed​

One negative condition not allowed​

Conditions​

String Values​

Escape Sequences​

Regular Expressions​

jq Expressions​

Counts​

Common pitfalls​

Avoid Yaml literal-block scalers |​

Negative conditions​

Simple negative condition​

Multiple negative conditions​

Negate options​

Window​

Match Width​

Multiple positive conditions width​

Anchor​

Slide​

Absolute​

Common Pitfalls​

Anchor index cannot exceed number of positive conditions​

Data Sources​

Nested Conditions​

Origin​

Correlations​

Rules

Rule

Rules Document

Set

One positive condition

Multiple positive conditions

Match `window`

Negative conditions

Common pitfalls

An `order` is not allowed in a `set`

One negative not allowed

Two positive terms without a `window` not allowed

Sequence

Multiple positive conditions

Order `window`

Negative conditions

Common pitfalls

`match` conditions not allowed

One positive condition not allowed

One negative condition not allowed

Conditions

String Values

Escape Sequences

Regular Expressions

`jq` Expressions

Counts

Common pitfalls

Avoid Yaml literal-block scalers `|`

Negative conditions

Simple negative condition

Multiple negative conditions

Negate options

Window

Match Width

Multiple positive conditions `width`

Anchor

Slide

Absolute

Common Pitfalls

Anchor index cannot exceed number of positive conditions

Data Sources

Nested Conditions

Origin

Correlations