diff --git a/docs/aws_storage.md b/docs/aws_storage.md new file mode 100644 index 0000000000000000000000000000000000000000..ef3aa41819ee536b85a53a5564cb3da3dff66a9a --- /dev/null +++ b/docs/aws_storage.md @@ -0,0 +1,41 @@ +# AWS Storage setup instruction for Loki + +## Example config via yaml file +```yaml +schema_config: + configs: + - from: 0 + store: aws + object_store: aws + schema: v9 + index: + prefix: dynamodb_table_name + period: 0 +storage_config: + aws: + s3: s3://access_key:secret_access_key@region/bucket_name + dynamodbconfig: + dynamodb: dynamodb://access_key:secret_access_key@region +``` + +## S3 + +Loki is using S3 as object storage. It stores log within directories based on +[`OrgID`](./operations.md#Multi-tenancy). For example, Logs from org `faker' +will stored in `s3://BUCKET_NAME/faker/`. + +The S3 configuration is setup with url format: `s3://access_key:secret_access_key@region/bucket_name`. + +## Dynamo DB + +Loki uses dynamodb for the index storage. It is used for querying logs, make +sure you adjuest your throughput to your usage. + +DynamoDB access is very similar to S3, however you do not need to specific a +table name in the storage section. You will need to set the table name for +`index.prefix` inside schema config section. + +To setup dynamodb, you can do it manually, or use `table_manager` to create and +maintain the tables for you. You can find out more info about table manager at +[cortex +project](https://github.com/cortexproject/cortex)(https://github.com/cortexproject/cortex). There is an example table manager deployment inside the ksonnet deployment method. You can find it [here](../production/ksonnet/loki/table-manager.libsonnet) diff --git a/docs/operations.md b/docs/operations.md index 72f4d086e00e44c7331b211a964fd5802b6bd631..352fee773855bf9452995069038deea70508499d 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -76,7 +76,11 @@ The chunks are stored under `/tmp/loki/chunks`. ### Cloud storage +#### Google Cloud Storage + Loki has support for Google Cloud storage. Take a look at our [production setup](https://github.com/grafana/loki/blob/a422f394bb4660c98f7d692e16c3cc28747b7abd/production/ksonnet/loki/config.libsonnet#L55) for the relevant configuration fields. -Support for AWS and Azure is also available, but not yet documented. + +#### AWS S3 & DynamoDB +Take a look at our [s3-dynamodb setup](./aws_storage.md)