From 7195539a3fb036f44903d3702b9dd982399916da Mon Sep 17 00:00:00 2001 From: Robert Fratto <robert.fratto@grafana.com> Date: Tue, 3 Sep 2019 14:28:39 -0400 Subject: [PATCH] docs: incorporate formatting fixes from #860 --- docs/promtail.md | 40 ++++++++++++++++---------------- docs/promtail/config-examples.md | 4 ++-- 2 files changed, 22 insertions(+), 22 deletions(-) diff --git a/docs/promtail.md b/docs/promtail.md index d11d39d9..a547ea36 100644 --- a/docs/promtail.md +++ b/docs/promtail.md @@ -14,42 +14,42 @@ Promtail is an agent which reads log files and sends streams of log data to the centralised Loki instances along with a set of labels. For example if you are running Promtail in Kubernetes then each container in a single pod will usually yield a single log stream with a set of labels based on that particular pod Kubernetes labels. You can also run Promtail outside Kubernetes, but you would -then need to customise the scrape_configs for your particular use case. +then need to customise the `scrape_configs` for your particular use case. -The way how Promtail finds out the log locations and extracts the set of labels is by using the *scrape_configs* +The way how Promtail finds out the log locations and extracts the set of labels is by using the *`scrape_configs`* section in the Promtail yaml configuration. The syntax is the same what Prometheus uses. -The scrape_configs contains one or more *entries* which are all executed for each container in each new pod running +The `scrape_config`s contains one or more *entries* which are all executed for each container in each new pod running in the instance. If more than one entry matches your logs you will get duplicates as the logs are sent in more than one stream, likely with a slightly different labels. Everything is based on different labels. The term "label" here is used in more than one different way and they can be easily confused. -* Labels starting with __ (two underscores) are internal labels. They are not stored to the loki index and are +* Labels starting with `__` (two underscores) are internal labels. They are not stored to the loki index and are invisible after Promtail. They "magically" appear from different sources. -* Labels starting with \_\_meta_kubernetes_pod_label_* are "meta labels" which are generated based on your kubernetes +* Labels starting with `__meta_kubernetes_pod_label_*` are "meta labels" which are generated based on your kubernetes pod labels. Example: If your kubernetes pod has a label "name" set to "foobar" then the scrape_configs section - will have a label \_\_meta_kubernetes_pod_label_name with value set to "foobar". -* There are other \_\_meta_kubernetes_* labels based on the Kubernetes metadadata, such as the namespace the pod is - running (\_\_meta_kubernetes_namespace) or the name of the container inside the pod (\_\_meta_kubernetes_pod_container_name) -* The label \_\_path\_\_ is a special label which Promtail will read to find out where the log files are to be read in. -* The label `filename` is added for every file found in \_\_path\_\_ to ensure uniqueness of the streams. It contains the absolute path of the file being tailed. - -The most important part of each entry is the *relabel_configs* which are a list of operations which creates, -renames, modifies or alters labels. A single scrape_config can also reject logs by doing an "action: drop" if -a label value matches a specified regex, which means that this particular scrape_config will not forward logs + will have a label `__meta_kubernetes_pod_label_name` with value set to "foobar". +* There are other `__meta_kubernetes_*` labels based on the Kubernetes metadadata, such as the namespace the pod is + running (`__meta_kubernetes_namespace`) or the name of the container inside the pod (`__meta_kubernetes_pod_container_name`) +* The label `__path__` is a special label which Promtail will read to find out where the log files are to be read in. +* The label `filename` is added for every file found in `__path__` to ensure uniqueness of the streams. It contains the absolute path of the file being tailed. + +The most important part of each entry is the `relabel_configs` which are a list of operations which creates, +renames, modifies or alters labels. A single `scrape_config` can also reject logs by doing an `action: drop` if +a label value matches a specified regex, which means that this particular `scrape_config` will not forward logs from a particular log source, but another scrape_config might. -Many of the scrape_configs read labels from \_\_meta_kubernetes_* meta-labels, assign them to intermediate labels -such as \_\_service\_\_ based on a few different logic, possibly drop the processing if the \_\_service\_\_ was empty -and finally set visible labels (such as "job") based on the \_\_service\_\_ label. +Many of the scrape_configs read labels from `__meta_kubernetes_*` meta-labels, assign them to intermediate labels +such as `__service__` based on a few different logic, possibly drop the processing if the `__service__` was empty +and finally set visible labels (such as "job") based on the `__service__` label. In general, all of the default Promtail scrape_configs do the following: * They read pod logs from under /var/log/pods/$1/*.log. - * They set "namespace" label directly from the \_\_meta_kubernetes_namespace. + * They set "namespace" label directly from the `__meta_kubernetes_namespace.` * They expect to see your pod name in the "name" label * They set a "job" label which is roughly "your namespace/your job name" -### Idioms and examples on different relabel_configs: +### Idioms and examples on different `relabel_configs:` * Drop the processing if a label is empty: ```yaml @@ -112,4 +112,4 @@ There are three [Prometheus metric types](https://prometheus.io/docs/concepts/me `Counter` and `Gauge` record metrics for each line parsed by adding the value. While `Histograms` observe sampled values by `buckets`. -See the [pipeline metric docs](./logentry/processing-log-lines.md#metrics) for more info on creating metrics from log content. \ No newline at end of file +See the [pipeline metric docs](./logentry/processing-log-lines.md#metrics) for more info on creating metrics from log content. diff --git a/docs/promtail/config-examples.md b/docs/promtail/config-examples.md index 4bbf7504..52b7c929 100644 --- a/docs/promtail/config-examples.md +++ b/docs/promtail/config-examples.md @@ -48,13 +48,13 @@ scrape_configs: #### Description -Scrape_config section of config.yaml contents contains various jobs for parsing your logs +`scrape_config` section of `config.yaml` contents contains various jobs for parsing your logs `job` and `host` are examples of static labels added to all logs, labels are indexed by Loki and are used to help search logs. `__path__` it is path to directory where stored your logs. -If you run promtail and this config.yaml in Docker container, don't forget use docker volumes for mapping real directories +If you run promtail and this `config.yaml` in Docker container, don't forget use docker volumes for mapping real directories with log to those folders in the container. #### Example Use -- GitLab