Signed-off-by: Xiang Dai <764524258@qq.com>

Signed-off-by: Xiang Dai <764524258@qq.com>

* Improvements in live tailing of logs

Instead of polling for new logs, a grpc stream is opened between ingester and querier to get live logs
Querier reconnects to disconnected or newly added ingesters

* Added more comments to code for live log tailing

* Some code refactoring in live log tailing

* handling delayfor in logcli, max delayfor to be 5 seconds

* some changes in tail response in live tailing

* Fixed issue with stopping ingesters gracefully when live tailing is being used

* Added tests for tailer in querier

* Live tailing made logql compatible, some code refactoring suggested in PR

updating the loki dashboards to use the metrics from the new gateway

Removing the pre-allocation of a buffer when serializing blocks, in most of my empirical testing we were using 9-10k out of the 32k allocated leaving about 2/3 of the buffer allocated and unused times the number of blocks per chunk, times the number of chunks kept in memory.  This was adding up to a fair amount of allocated but unused space.

component/integrations: rename fluent plugin

Signed-off-by: Steven Sheehy <ssheehy@firescope.com>

* fix docker plugin ci

* use binary cache

* adds first version of docker driver

* without logrus and fixes some linter issue

* fix the driver and start a build system

* adds swarm label discovery

* Add documentation and more targets

* make the linter happy 

* indent config.json

* with circleci steps for master and branch

* Review Feebacks

* Helm chart integration testing

Signed-off-by: Steven Sheehy <ssheehy@firescope.com>

* Don't check helm version on every push

Signed-off-by: Steven Sheehy <ssheehy@firescope.com>

* Remove chart upgrade testing

Signed-off-by: Steven Sheehy <ssheehy@firescope.com>

Because of the logger being initialized before the configuration file is parsed,
the log_level from the config file is ignored.
To solve this, the logger is reinitialized after the file is parsed, as it is
already being done in promtail.

* fix(promtail/targets): remove dependency on prometheus/relabel

`filetargetmanager.go` had a dependency on prometheus/relabel, a package that
has been removed in favor of prometheus/pkg/relabel.
This converts the code to the new package, to allow the prometheus vendor be
updated to master

* chore(vendor): update prometheus vendor

Updates prometheus vendor to current master, to add support for InitContainers
in kubernetes service discovery

* chore(vendor): do dep's homework

Gives dep hints on how to resolve the vendor so that it work

* fix(promtail/targets): non-nil check

Accidentally checked against the wrong labelSet, this one can never get nil

Signed-off-by: Xiang Dai <764524258@qq.com>

Build promtail on linux and windows, discard artifacts

Updates cortex vendor to get the changes from cortexproject/cortex#1431.
Required for making promtail build on Windows

Signed-off-by: sh0rez <me@shorez.de>

removed helm entry in scrape_config.sh because there doesn't seem to be anyway to make it work with new pipeline config

renaming `metric` stage to `metrics` as it defines multiple metrics, similar to labels stage which is also plural.
Adding a couple unit tests to regex and json stage to act as examples

cleaning up GoDoc
pipeline_name is now optional for matcher stage

refactored and improved some of the tests, each stage type should now have a test using actual YAML
A few TODO's remaining and some comment cleanup

metric test is still failing because of missing metrics when the counter is 0
added a lot of TODO's I need to go back and cleanup

Making the pipeline itself a Stage so that we can use it to better implement the Match stage (and it cleans up the Docker and CRI extensions some too)
Still needs to fix the metrics test

Still needs:  Matcher stage work, pipeline test work