All the time Prometheus Most of them are Tucao. HA And clustering solutions.

Think about it before. We use top matching machines, But there will be disks, network IO, Query timeout and other issues, Mainly because the amount of data collected is too large, Beyond the range of single machine.

In the past, We have, department, All that can be split into different ones Prometheus Gone, But even so, There are still some problems:

*
Some businesses cannot be split, Data volume is still very large.

*
What data collection and query do is completely independent, But they often compete with each other to occupy resources.

Today we are going to explore new ways to solve this problem, That is to say remote_read Realization Prometheus Cluster scheme of data read write separation .

remote_read brief introduction

As everyone knows, from Prometheus 1.8 start, Added a name remote_read Configuration, Details are as follows:

*
# The URL of the endpoint to query from.

*
url: <string>

*


*
# Timeout for requests to the remote read endpoint.

*
[ remote_timeout: <duration> | default = 30s ]

*


*
# Sets the `Authorization` header on every remote read request with the

*
# configured username and password.

*
basic_auth:

*
 [ username: <string> ]

*
 [ password: <string> ]

*


*
# Sets the `Authorization` header on every remote read request with

*
# the configured bearer token. It is mutually exclusive with
`bearer_token_file`.

*
[ bearer_token: <string> ]

*


*
# Sets the `Authorization` header on every remote read request with the bearer
token

*
# read from the configured file. It is mutually exclusive with `bearer_token`.

*
[ bearer_token_file: /path/to/bearer/token/file ]

*


*
# Configures the remote read request's TLS settings.

*
tls_config:

*
 [ <tls_config> ]

*


*
# Optional proxy URL.

*
[ proxy_url: <string> ]

remote_read Parameters are used to read data remotely, Use http Agreement.

Of course, being read as a data source needs to support its remote storage reader Interface, This is designed to decouple in a new generation of storage architecture, It's easy to do:

*
Prometheus The reading and writing of can be in different Prometheus Server Conduct, That is, one. Prometheus Read other Prometheus Data.

*
Prometheus Can be read and written in more storage engines, That is, you can use InfluxDB Store data as a database.

Deployment framework



Architecture specification:

*
Server A'  Express Server A  Mirror image, Same functions and data(B,C Empathy).

*
Data flow adopts pull mode.

Not hard to find, In architecture Prometheus There are two main categories, For data collection( for example B) and For data query( for example A).

Where data query Prometheus The data is read from the nodes collected to the data, Please note that, It only does real-time queries and memory operations, No data storage.

Through such an architecture, We can easily separate the whole monitoring data collection and query, It's also easier to achieve high availability.

How to configure

The deployment architecture we discussed above, How to configure and implement this deployment scheme?

Now I'll show you how to use the remote_read Realization Prometheus Data reading process between.

Experimental environment:

*
Prometheus Edition: prometheus-1.8.2.darwin-amd64

*
NodeExporter Edition:node_exporter-0.12.0.darwin-amd64

Experimental content:

Run three locally Prometheus Server, They run in 9090, 9091, 9092 port. among 9091 and 9092 Mainly used for collection
node_exporter data,9090 For reading 9091, 9092 Data collected.

The configuration information is as follows:

*
# Data read 9090:

*
remote_read:

*
 - url: 'http://localhost:9091/api/v1/read'

*
   remote_timeout: 8s

*
 - url: 'http://localhost:9092/api/v1/read'

*
   remote_timeout: 8s

*


*
# data collection 9091

*
- job_name: 'node'

*
 static_configs:

*
   - targets: ["localhost:9100"]

*


*
# data collection 9092

*
- job_name: 'node'

*
 static_configs:

*
   - targets: ["localhost:9100"]

Complete configuration, After successfully starting these three instances, We can verify them in their own interfaces.



9091 Data collected



9092 Data collected








9090 Read data

Support aggregate operation query:



Okay, So far, we have successfully passed remote_read Configuration implementation from different Prometheus Server
Read data, It means that the deployment architecture we discussed earlier is completely feasible.

Write here, The focus of this article is basically completed, Here is to adjust your deployment architecture?

Promotion activities:

GopherChina 2018 Early bird ticket ends at1 month31 day( Last three days), click  Read the original text  Understanding ahead~