Policy sources

Policy sources represent third-party collections of security relevant data, including threat intelligence sources and Suricata rulesets. Corelight-update provides a method to collect and provide these data sources to your sensors.

Policy sources differ from Corelight-update Third-party integrations settings, in that a Policy source can be any pre-formatted content that can be downloaded via an unauthenticated, or token-authenticated URL.

The URL can be a local or remote source. For example, in an air-gapped environment, a policy source could point to a local webserver to collect content.

See Using a proxy with Corelight-update for details about using a proxy to download remote sources.

Overview of adding policy sources

  1. Determine the access url and authentication required for the policy source.

  2. Configure the policy source settings under the sources: section of the Corelight-update db-config file.

  3. Upload your changes into Corelight-update.

Policy source settings

The following fields are available for configuring a policy source:

sources:
  - name:
    url:
    source_type:
    global_cache:
    auth_type:
    auth_token:
    auth_token_header:
    username:
    encrypted_pass:
    ignore_tls:
  • The policy source source_type field can be set to either suricata or intel. When using the intel source type, the URL must provide the data in a Zeek or Bro compatible format. For suricata, the URL must provide the data in the Suricata rule format.

  • The global_cache is enabled (true) by default for all sources. If global_cache is disabled, that source will be download once for each policy that uses it.

  • The auth_type field can be set to basic, token, or left empty.

Processing a policy source

When Corelight-update processes a policy source, it:

  1. Checks the global cache for the target filename.

    1. If the file is present, use the file to process the source.

    2. If the file is not present in the global cache:

      1. Check for a policy level cache of the file and generate an If-Modified-Since HTTP header.

      2. Attempt to download the file using the If-Modified-Since HTTP header.

        • If a new file is downloaded, create or update the policy-level cache.

      3. Use the policy-level cache to process the source.

Caution

Matches are made in the global cache using only the filename, not the full URL.

Default policy sources

The pre-configured policy sources are:
  • Corelight Labs Suricata Rules

  • ET/Open ruleset:

You will find the following pre-configured policy source settings in the db-config example file:

sources:
  - name: "Corelight"
    url: "https://feed.corelight.com/corelight.rules"
    source_type: "suricata"
    global_cache: true
  - name: "ET/Open"
    url: "https://rules.emergingthreats.net/open/suricata-6.0/emerging.rules.tar.gz"
    source_type: "suricata"
    global_cache: false

Commonly used Suricata rulesets

Any source that can be downloaded in the standard Suricata ruleset format, and does not require authentication, can be added to the list of sources. Here is a list of common Suricata ruleset sources. Just verify the URL, modify as needed, and add it to your list of sources.

  • Corelight Labs Suricata Rules: https://feed.corelight.com/corelight.rules

  • ET/Open: https://rules.emergingthreats.net/open/suricata-6.0/emerging.rules.tar.gz

  • ET/Pro: https://rules.emergingthreatspro.com/<insert-et-pro-key-here>/suricata-5.0/etpro.rules.tar.gz

  • oisf/trafficid: https://openinfosecfoundation.org/rules/trafficid/trafficid.rules

  • ptresearch/attackdetection: https://raw.githubusercontent.com/ptresearch/AttackDetection/master/pt.rules.tar.gz

  • scwx/enhanced: https://ws.secureworks.com/ti/ruleset/<insert-secret-code-here>/Suricata_suricata-enhanced_latest.tgz

  • scwx/malware: https://ws.secureworks.com/ti/ruleset/<insert-secret-code-here>/Suricata_suricata-malware_latest.tgz

  • scwx/security: https://ws.secureworks.com/ti/ruleset/<insert-secret-code-here>/Suricata_suricata-security_latest.tgz

  • sslbl/ssl-fp-blacklist: https://sslbl.abuse.ch/blacklist/sslblacklist.rules

  • sslbl/js3-fingerprints: https://sslbl.abuse.ch/blacklist/ja3_fingerprints.rules

  • etnetera/aggressive: https://security.etnetera.cz/feeds/etn_aggressive.rules

  • tgreen/hunting: https://raw.githubusercontent.com/travisbgreen/hunting-rules/master/hunting.rules

  • malsilo: https://malsilo.gitlab.io/feeds/dumps/malsilo.rules.tar.gz

Threat intelligence sources

The threat intelligence sources managed with Corelight-update must provide their data in a Zeek or Bro compatible format.

The following example includes settings for the ThreatQ and MISP threat intel sources:

sources:
  - name: ThreatQ
    url: https://string.experience.threatq.online/api/export/c8299290f2d4319923e2eb/?token=aasTjqMXwJ4u
    source_type: intel
    global_cache: false
  - name: MISP
    url: https://misp/attributes/bro/download/all
    source_type: intel
    global_cache: false
    auth_type: token
    auth_token: BVkgNaFh27IGelkIuEAiPBBlDsOp9cjd
    auth_token_header: Authorization
    ignore_tls: true

Note

For more details on these examples, see Third-party configurations.