# Teler | Real-time HTTP Intrusion Detection

**URL:** <https://onehack.st/t/teler-real-time-http-intrusion-detection/163672>\
**Category:** Tools & Scripts\
**Tags:** tools\
**Created:** [November 24, 2020, 12:26pm UTC](https://onehack.st/t/teler-real-time-http-intrusion-detection/163672 "2020-11-24T12:26:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheStrength](https://onehack.st/user_avatar/onehack.st/thestrength/32/46753_2.png) [@TheStrength](https://onehack.st/u/TheStrength)\
**Post date:** [November 24, 2020, 12:26pm UTC](https://onehack.st/t/teler-real-time-http-intrusion-detection/163672/1 "2020-11-24T12:26:35Z")

</div>

[![teler](https://user-images.githubusercontent.com/25837540/97091757-7200d880-1668-11eb-82c4-e5c4971d2bc8.png)](https://github.com/kitabisa/teler#)

### **teler**

[![Kitabisa SecLab](https://camo.githubusercontent.com/40c2c7c4eae7bd8cacd7127123b831a6c25d418c56d4c7754920d345b7f9bcf0/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6b697461626973612d736563757269747925323070726f6a6563742d626c7565)](https://github.com/kitabisa/teler#) [![License](https://camo.githubusercontent.com/aab9931405dd70e348d0263ec02f33960c82d457103d677ea02ac344a5de4676/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4c6963656e73652d417061636865253230322e302d79656c6c6f77677265656e)](https://github.com/kitabisa/teler/blob/development/LICENSE) [![made with Go](https://camo.githubusercontent.com/92f2d39d524fb48dcf06a5a35b231361e84dec165d6becee283b34ce2a423bc5/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6d616465253230776974682d476f2d627269676874677265656e)](http://golang.org/) [![Version](https://camo.githubusercontent.com/b3be00f12c433690058a11ecc2af3d9be31b3b5eb489163301d4961ebd9868b3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f76657273696f6e2d302e302e342d626c756576696f6c6574)](https://github.com/kitabisa/teler/releases) [![Platform](https://camo.githubusercontent.com/df42252b9ee3d72bf201262d5be3dc1c6defa94bbaf501948a856c403e89093a/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f706c6174666f726d2d6f73782532466c696e757825324677696e646f77732d677265656e)](https://github.com/kitabisa/teler#) [![GitHub issues](https://camo.githubusercontent.com/dea5a93f5c239c96e1f84b4a76cf17bc20670ca88e34f095fdb665e93167ac8a/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f6973737565732f6b697461626973612f74656c6572)](https://github.com/kitabisa/teler/issues)

Real-time HTTP Intrusion Detection  
[Contribute](https://github.com/kitabisa/teler/blob/development/.github/CONTRIBUTING.md) · [What’s new](https://github.com/kitabisa/teler/blob/development/CHANGELOG.md) · [Report Bug](https://github.com/kitabisa/teler/issues/new/choose) · [Request Feature](https://github.com/kitabisa/teler/issues/new/choose)

`teler` is an **real-time intrusion detection** and threat alert based on web log that runs in a **terminal** with resources that we collect and provide by the community. ❤

[![teler](https://user-images.githubusercontent.com/25837540/97096468-f8ccaa00-1696-11eb-8830-0d3a7be45a2d.gif)](https://github.com/kitabisa/teler#)

## Table of Contents

- [Table of Contents](https://github.com/kitabisa/teler#table-of-contents)
- [Features](https://github.com/kitabisa/teler#features)
- [Why teler?](https://github.com/kitabisa/teler#why-teler)
- [Demo](https://github.com/kitabisa/teler#demo)
- [Installation](https://github.com/kitabisa/teler#installation)
  - [from Binary](https://github.com/kitabisa/teler#from-binary)
  - [using Docker](https://github.com/kitabisa/teler#using-docker)
  - [from Source](https://github.com/kitabisa/teler#from-source)
  - [from GitHub](https://github.com/kitabisa/teler#from-github)

- [Usage](https://github.com/kitabisa/teler#usage)
  - [Flags](https://github.com/kitabisa/teler#flags)
    - [Config](https://github.com/kitabisa/teler#config)
    - [Input](https://github.com/kitabisa/teler#input)
    - [Concurrency](https://github.com/kitabisa/teler#concurrency)
    - [Output](https://github.com/kitabisa/teler#output)
    - [JSON Format](https://github.com/kitabisa/teler#json-format)
    - [Remove Caches](https://github.com/kitabisa/teler#remove-caches)

- [Configuration](https://github.com/kitabisa/teler#configuration)
  - [Log formats](https://github.com/kitabisa/teler#log-formats)
    - [Apache](https://github.com/kitabisa/teler#apache)
    - [Nginx](https://github.com/kitabisa/teler#nginx)
    - [Nginx Ingress](https://github.com/kitabisa/teler#nginx-ingress)
    - [Amazon S3](https://github.com/kitabisa/teler#amazon-s3)
    - [Elastic LB](https://github.com/kitabisa/teler#elastic-lb)
    - [CloudFront](https://github.com/kitabisa/teler#cloudfront)

  - [Threat rules](https://github.com/kitabisa/teler#threat-rules)
    - [Excludes](https://github.com/kitabisa/teler#excludes)
    - [Whitelists](https://github.com/kitabisa/teler#whitelists)

  - [Notification](https://github.com/kitabisa/teler#notification)
  - [Metrics](https://github.com/kitabisa/teler#metrics)
    - [Prometheus](https://github.com/kitabisa/teler#prometheus)

- [Contributors](https://github.com/kitabisa/teler#contributors)
  - [Resources](https://github.com/kitabisa/teler#resources)

- [Pronunciation](https://github.com/kitabisa/teler#pronunciation)
- [Changes](https://github.com/kitabisa/teler#changes)
- [License](https://github.com/kitabisa/teler#license)

## Features

- **Real-time** : Analyze logs and identify suspicious activity in real-time.
- **Alerting** : teler provides alerting when a threat is detected, push notifications include Slack, Telegram and Discord.
- **Monitoring** : We’ve our own metrics if you want to monitor threats easily, and we use Prometheus for that.
- **Latest resources** : Collections is continuously up-to-date.
- **Minimal configuration** : You can just run it against your log file, write the log format and let teler analyze the log and show you alerts!
- **Flexible log formats** : teler allows any custom log format string! It all depends on how you write the log format in configuration file.
- **Incremental log processing** : Need data persistence rather than [buffer stream](https://linux.die.net/man/1/stdbuf)? teler has the ability to process logs incrementally through the on-disk persistence options.

## Why teler?

teler was designed to be a fast, terminal-based threat analyzer. Its core idea is to quickly analyze and hunt threats in real time!

## Demo

Here is a preview of `teler` with conditions of use as:

| **Buffer-streams** | **Incremental** |
| --- | --- |
| [![teler](https://camo.githubusercontent.com/ae532780a0b820db86f8c6117fff0b72e5f86f94c1a6cb78f313efc5195ba24d/68747470733a2f2f61736369696e656d612e6f72672f612f3336373631362e737667)](https://asciinema.org/a/367616) | [![teler](https://camo.githubusercontent.com/e8584441e0e328565b7c53f2b990e8de64a2a9c125503b3e361b11078aecea2a/68747470733a2f2f61736369696e656d612e6f72672f612f3336373631302e737667)](https://asciinema.org/a/367610) |

## Installation

### from Binary

The installation is easy. You can download a prebuilt binary from [releases page](https://github.com/kitabisa/teler/releases), unpack and run! or run with:

▶ curl -sSfL ‘[https://ktbs.dev/get-teler.sh](https://ktbs.dev/get-teler.sh)’ | sh -s – -b /usr/local/bin

### using Docker

Pull the Docker image by running:

▶ docker pull kitabisa/teler

### from Source

If you have go1.14+ compiler installed and configured:

▶ GO111MODULE=on go get -v -u ktbs.dev/teler/cmd/teler

In order to update the tool, you can use `-u` flag with `go get` command.

### from GitHub

▶ git clone [https://github.com/kitabisa/teler](https://github.com/kitabisa/teler) ▶ cd teler ▶ make build ▶ mv ./bin/teler /usr/local/bin

## Usage

Simply, teler can be run with:

▶ [buffers] | teler -c /path/to/config/teler.yaml # or ▶ teler -i /path/to/access.log -c /path/to/config/teler.yaml

If you’ve built teler with a Docker image:

▶ [buffers] | docker run -i --rm -e TELER\_CONFIG=/path/to/config/teler.yaml teler # or ▶ docker run -i --rm -e TELER\_CONFIG=/path/to/config/teler.yaml teler --input /path/to/access.log

### Flags

▶ teler -h

This will display help for the tool.

[![teler](https://user-images.githubusercontent.com/25837540/97946822-164bf300-1dbe-11eb-8f54-3a25ee0eb80e.png)](https://github.com/kitabisa/teler#)

Here are all the switches it supports.

| Flag | Description | Examples |
| --- | --- | --- |
| -c, | | |
| –config | teler configuration file | kubectl logs nginx |
| -i, | | |
| –input | Analyze logs from data persistence rather than buffer stream | teler -i /var/log/nginx/access.log |
| -x, | | |
| –concurrent | Set the concurrency level to analyze logs | |
| (default: 20) | tail -f /var/log/nginx/access.log | teler -x 50 |
| -o, | | |
| –output | Save detected threats to file | teler -i /var/log/nginx/access.log -o /tmp/threats.log |
| –json | Display threats in the terminal as JSON format | teler -i /var/log/nginx/access.log --json |
| –rm-cache | Remove all cached resources | teler --rm-cache |
| -v, | | |
| –version | Show current teler version | teler -v |

#### Config

The `-c` flag is to specify teler configuration file.

▶ tail -f /var/log/nginx/access.log | teler -c /path/to/config/teler.yaml

This is **required** , but if you have defined `TELER_CONFIG` environment you don’t need to use this flag, e.g.:

▶ export TELER\_CONFIG=“/path/to/config/teler.yaml” ▶ tail -f /var/log/nginx/access.log | teler # or ▶ tail -f /var/log/nginx/access.log | TELER\_CONFIG=“/path/to/config/teler.yaml” teler

#### Input

Need log analysis incrementally? This `-i` flag is useful for that.

▶ teler -i /var/log/nginx/access.log

#### Concurrency

Concurrency is the number of logs analyzed at the same time. Default value teler provide is 20, you can change it by using `-x` flag.

▶ teler -i /var/log/nginx/access.log -x 50

#### Output

You can also save the detected threats into a file with `-o` flag.

▶ teler -i /var/log/nginx/access.log -o threats.log

#### JSON Format

If you want to display the detected threats as JSON format, switch it with `--json` flag.

▶ teler -i /var/log/nginx/access.log --json

Please note this will also apply if you save it to a file with `-o` flag.

#### Remove Caches

It will removes all stored resources in the user-level cache directory, see [cache](https://github.com/kitabisa/teler#cache).

▶ teler --rm-cache

## Configuration

`teler` requires a minimum of configuration to process and/or log analysis, and execute threats and/or alerts. See [teler.example.yaml](https://github.com/kitabisa/teler/blob/development/teler.example.yaml) for an example.

### Log Formats

Because we use `gonx` package to parse the log, you can write any log format. As an example:

#### Apache

log\_format: | $remote\_addr - $remote\_user [$time\_local] “$request\_method $request\_uri $request\_protocol” $status $body\_bytes\_sent

#### Nginx

log\_format: | $remote\_addr $remote\_user - [$time\_local] “$request\_method $request\_uri $request\_protocol” $status $body\_bytes\_sent “$http\_referer” “$http\_user\_agent”

#### Nginx Ingress

log\_format: | $remote\_addr - [$remote\_addr] $remote\_user - [$time\_local] “$request\_method $request\_uri $request\_protocol” $status $body\_bytes\_sent “$http\_referer” “$http\_user\_agent” $request\_length $request\_time [$proxy\_upstream\_name] $upstream\_addr $upstream\_response\_length $upstream\_response\_time $upstream\_status $req\_id

#### Amazon S3

log\_format: | $bucket\_owner $bucket [$time\_local] $remote\_addr $requester $req\_id $operationration $key “$request\_method $request\_uri $request\_protocol” $status $error\_code $body\_bytes\_sent - $total\_time - “$http\_referer” “$http\_user\_agent” $version\_id $host\_id $signature\_version $cipher\_suite $http\_auth\_type $http\_host\_header $tls\_version

#### Elastic LB

log\_format: | $time\_local $elb\_name $remote\_addr $upstream\_addr $request\_processing\_time $upstream\_processing\_time $response\_processing\_time $status $upstream\_status $body\_received\_bytes $body\_bytes\_sent “$request\_method $request\_uri $request\_protocol” “$http\_user\_agent” $cipher\_suite $tls\_version

#### CloudFront

log\_format: | $date $time $edge\_location $body\_bytes\_sent $remote\_addr $request\_method $http\_host\_header $requst\_uri $status $http\_referer $http\_user\_agent $request\_query $http\_cookie $edge\_type $req\_id $http\_host\_header $ssl\_protocol $body\_bytes\_sent $response\_processing\_time $http\_host\_forwarded $tls\_version $cipher\_suite $edge\_result\_type $request\_protocol $fle\_status $fle\_encrypted\_fields $http\_port $time\_first\_byte $edge\_detail\_result\_type $http\_content\_type $request\_length $request\_length\_start $request\_length\_end

### Threat rules

#### Cache

By default, `teler` will fetch external resources every time you run it, but you can switch external resources to be cached or not.

rules: cache: true

If you choose to cache resources, it’s stored under user-level cache directory of cross-platform and will be updated every day, see [resources](https://github.com/kitabisa/teler#resources).

#### Excludes

We include resources for predetermined threats, including:

- Common Web Attack
- Bad IP Address
- Bad Referrer
- Bad Crawler
- Directory Bruteforce

You can disable any type of threat in the `excludes` configuration _(case-sensitive)_.

rules: threat: excludes: - “Bad IP Address”

The above format detects threats that are not included as bad IP address, and will not analyze logs/ send alerts for that type.

#### Whitelists

You can also add whitelists to teler configuration.

rules: threat: whitelists: - “(curl|Go-http-client|okhttp)/\*” - “^/wp-login\.php”

It covers the entire HTTP request and processed as _regExp_, please write it with caution!

### Notification

We provide alert notification options:

- Slack,
- Telegram
- Discord

Configure the notification alerts needed on:

notifications: slack: token: “xoxb-…” color: “#ffd21a” channel: “G30SPKI” telegram: token: “123456:ABC-DEF1234…-…” chat\_id: “-111000” discord: token: “NkWkawkawkawkawka.X0xo.n-kmZwA8aWAA” color: “16312092” channel: “700000000000000…”

You can also choose to disable alerts or want to be sent where the alerts are.

alert: active: true provider: “slack”

### Metrics

`teler` also supports metrics using Prometheus.

#### Prometheus

You can configure the host, port and endpoint to use Prometheus metrics in the configuration file.

prometheus: active: true host: “localhost” port: 9099 endpoint: “/metrics”

Here are all the metrics we collected & categorized.

| Metric | Description |
| --- | --- |
| `teler_threats_count_total` | Total number of detected threats |
| `teler_cwa` | Get lists of Common Web Attacks |
| `teler_badcrawler` | Get lists of Bad Crawler requests |
| `teler_dir_bruteforce` | Get lists of Directories Bruteforced |
| `teler_bad_referrer` | Get lists of Bad Referrer requests |
| `teler_badip_count` | Total number of Bad IP Addresses |

## GitHub:

[https://github.com/kitabisa/teler](https://github.com/kitabisa/teler)