fixed one bug which was probably unrelated to the test failure and broke out the test to avoid some non-deterministic behaviors in querying the ingester

Signed-off-by: Steven Sheehy <ssheehy@firescope.com>

* adding a few more test cases for entry parsing docker format

* adding testify assert and require to clean up tests

* some tweaking to improve readability

Adding a configurable timeout for calls to the loki server.
Updating promtail test to set sane defaults for all configurations and then override for testing.
Removing .idea folder from .gitignore, we should encourage people (and myself) to use global ignore for IDE configs

Signed-off-by: Steven Sheehy <ssheehy@firescope.com>

Signed-off-by: Steven Sheehy <ssheehy@firescope.com>

Signed-off-by: Steven Sheehy <ssheehy@firescope.com>

Signed-off-by: Steven Sheehy <ssheehy@firescope.com>

Signed-off-by: kun <oiooj@qq.com>

Signed-off-by: Xiang Dai <764524258@qq.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

adding third party lib for Glob pattern matching which supports /**/ style subdirectory matching (#331)

* adding third party lib for Glob pattern matching which supports **
refactor promtail_test to be easier to read and include more tests

* Adding a sync() function to filetarget which runs on a timer to catch any events missed by fsnotify

* moving where we mark the final position to avoid a race condition exposed in testing, we want to stop the tailer before shutting down our tailer loop

* Fix changes from PR and refactor to remove circular imports and make config consistent

* Adding an "integration" type unit test which tests promtail as a whole

* fixing a bunch of imports I munged
moved tailer to its own file
refactoring filetarget to make synclogic easier to follow and reused as much as possible

* Include the stream's labels in OOO error responses.

Also, read the body of error responses and log them.  And retries 500s and connection errors.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Log close errors.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Update pkg/promtail/client/client.go

Co-Authored-By: tomwilkie <tomwilkie@users.noreply.github.com>

* Log retries at warn, final errors at error.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* set currLabels when creating new heapIterator

fixes #307

* update unit tests to ensure labels are set correctly

Proves that they can accept multiple samples with the same timestamp.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

This was broken in a422f394.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

I broke this when I updated the vendored Cortex, which includes https://github.com/cortexproject/cortex/commit/ac653065193fc9f9b91d0a3da38886cd20b816e5#diff-42196489d8c6339357699ab3656cb2e1R315

.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

When starting up, start all dependencies first then start the target
module.  When stopping, reverse the order to ensure we dont try and
stop a module while there is still something running that needs it.

fixes #105

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

This allows easier integration of other targets than files for promtail.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* handle symbolic links correctly

When tailing a file that is rotated we expect that a fsnotify.Rename
event will be sent when the old file is renamed and a fsnotif.Create
event will be sent when the replacement file is created. This allows
us to close and re-open the file.  However if the file we are
tailing is a symbolic link (and the target is in another directory)
we wont get these events.  To address this we need to tail the target
file directly.

fixes #274

* handle symlinks being re-linked

When a symbolic link is re-linked we will only get a fsnotify.Create
event. No rename or delete event will have been sent to trigger
the tailer to be stoped.  When this happens we should stop the current
tailer for the path and start a new one. This will allow us to perform
a new Readink on the path to get the updated target filename.

* Flush idle chunks properly

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Review feedback

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Review feedback.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Check the correct chunk is full before appending a new chunk.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

* Add the config to helm and ksonnet too.

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>