Adding `make debug` support to build debug binaries and debug containers which wrap the binary with delve and allow for remote debugging

Instead of polling for new logs, a grpc stream is opened between ingester and querier to get live logs
Querier reconnects to disconnected or newly added ingesters

* Revert "Make sure the default for EnforceMetricName is 

 (#518)"

This reverts commit 199746a1.

* Fix overrides unmarshalling properly

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Move live log tailing behind websocket on server

Live log tailing now requires client to just open a websocket and keep listening to it for getting live logs for matching query

* Fixed linting errors

* Unexported tailQuery querier method

* Added a delay in querying of logs to not miss delayed entries

* Renamed variable for better readability

* Some code refactoring suggested in PR review

* Fixed linter errors

* Code refactoring requested in PR review

* Disabling linting for writeHTTPErrorResponse since same value for an input is being passed from all function calls for now

* Removed a function which already existed

* Improved printing of log entries using cli and some other code refactoring

* Checking no-labels flag before printing log stream while tailing

Signed-off-by: Xiang Dai <764524258@qq.com>

Since kinpin.v2 not support use config like this, remove them.

Signed-off-by: Xiang Dai <764524258@qq.com>

* Add tracing to loki

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Use the op names from common/server

We don't need to the tracing middleware again. Further, the op names are
not sanitised here.

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

Signed-off-by: Xiang Dai <764524258@qq.com>

Signed-off-by: Xiang Dai <764524258@qq.com>

* Adding a sync() function to filetarget which runs on a timer to catch any events missed by fsnotify

* moving where we mark the final position to avoid a race condition exposed in testing, we want to stop the tailer before shutting down our tailer loop

* Fix changes from PR and refactor to remove circular imports and make config consistent

* Adding an "integration" type unit test which tests promtail as a whole

* fixing a bunch of imports I munged
moved tailer to its own file
refactoring filetarget to make synclogic easier to follow and reused as much as possible

alpine:3.4 only contains a docker image for arm64, so building for other
architectures doesn't work. The newest alpine relases, 3.9, is therefore
better.

This allows easier integration of other targets than files for promtail.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Flush idle chunks properly

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Review feedback

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Review feedback.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Check the correct chunk is full before appending a new chunk.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Add the config to helm and ksonnet too.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Update cortex to include optionally-enforce-metric-name

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Configure validation to allow 'metrics' without metric names.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* allow promtail target path to be a glob match

Promtail now expects __path__ to be a glob. All files that match
the glob expression will be tailed.  If new files are created that
match the glob, they will immediately be tailed.

* remove entry from positions file when file is deleted

* fix syncing the current positions to positions file

* handle file renames

When a file is renamed a "rename" and a "create" event are sent.
We need to treat the rename as a remove.  If the new filename matches
the path glob, then we will start tailing it when the "create" event
is reveived.

* lint

Signed-off-by: Xiang Dai <764524258@qq.com>

* Move all the config for various ways of deploying Loki into one place.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Fix reference to consul ksonnet module.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Move getting started docs into the production dir.

Remove the raw docker commands, only use the compose file.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Spell production correctly.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Tweaks

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Tweak docs and docker compose setup; put config in images.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* typo

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: RikiyaFujii <ridai22hanriku@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Rename Tempo to Loki.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Use new build image, don't delete generated files on clean.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Make promtail read all its config from a file.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Use gokit logging everywhere.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Start work to parse normal log lines.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* update common and kuberesolver.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* lint

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Get EntryParser stuff working, so we can tail raw log files.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Make sure filename label works.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* lint, remove interfacer.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Check in generated files, to make getting started easier.

Add gometalinter, and a lint check for changes to the generated files.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Make lint pass?

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>