Skip to content

multi-server-exporter multi server exporter is introduced #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 5, 2018
Merged

multi-server-exporter multi server exporter is introduced #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 19 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,16 +48,19 @@ Package vendoring is handled with [`govendor`](https://github.com/kardianos/gove
Path under which to expose metrics. Default is `/metrics`.

* `disable-default-metrics`
Use only metrics supplied from `queries.yaml` via `--extend.query-path`
Use only metrics supplied from `queries.yaml` via `--extend.query-path`.

* `disable-settings-metrics`
Use the flag if you don't want to scrape `pg_settings`.

* `extend.query-path`
Path to a YAML file containing custom queries to run. Check out [`queries.yaml`](queries.yaml)
for examples of the format.

* `dumpmaps`
Do not run - print the internal representation of the metric maps. Useful when debugging a custom
queries file.

* `log.level`
Set logging level: one of `debug`, `info`, `warn`, `error`, `fatal`

Expand All @@ -74,21 +77,23 @@ The following environment variables configure the exporter:
URI may contain the username and password to connect with.

* `DATA_SOURCE_URI`
an alternative to DATA_SOURCE_NAME which exclusively accepts the raw URI
an alternative to `DATA_SOURCE_NAME` which exclusively accepts the raw URI
without a username and password component.

* `DATA_SOURCE_USER`
When using `DATA_SOURCE_URI`, this environment variable is used to specify
the username.

* `DATA_SOURCE_USER_FILE`
The same, but reads the username from a file.

* `DATA_SOURCE_PASS`
When using `DATA_SOURCE_URI`, this environment variable is used to specify
the password to connect with.

* `DATA_SOURCE_PASS_FILE`
The same as above but reads the password from a file.

* `PG_EXPORTER_WEB_LISTEN_ADDRESS`
Address to listen on for web interface and telemetry. Default is `:9187`.

Expand All @@ -98,10 +103,13 @@ The following environment variables configure the exporter:
* `PG_EXPORTER_DISABLE_DEFAULT_METRICS`
Use only metrics supplied from `queries.yaml`. Value can be `true` or `false`. Default is `false`.

* `PG_EXPORTER_DISABLE_SETTINGS_METRICS`
Use the flag if you don't want to scrape `pg_settings`. Value can be `true` or `false`. Defauls is `false`.

* `PG_EXPORTER_EXTEND_QUERY_PATH`
Path to a YAML file containing custom queries to run. Check out [`queries.yaml`](queries.yaml)
for examples of the format.

Settings set by environment variables starting with `PG_` will be overwritten by the corresponding CLI flag if given.

### Setting the Postgres server's data source name
Expand All @@ -113,6 +121,10 @@ For running it locally on a default Debian/Ubuntu install, this will work (trans

sudo -u postgres DATA_SOURCE_NAME="user=postgres host=/var/run/postgresql/ sslmode=disable" postgres_exporter

Also, you can set a list of sources to scrape different instances from the one exporter setup. Just define a string separated by comma.

sudo -u postgres DATA_SOURCE_NAME="port=5432, port=6432" postgres_exporter

See the [github.com/lib/pq](http://github.com/lib/pq) module for other ways to format the connection string.

### Adding new metrics
Expand All @@ -136,7 +148,7 @@ The -extend.query-path command-line argument specifies a YAML file containing ad
Some examples are provided in [queries.yaml](queries.yaml).

### Disabling default metrics
To work with non-officially-supported postgres versions you can try disabling (e.g. 8.2.15)
To work with non-officially-supported postgres versions you can try disabling (e.g. 8.2.15)
or a variant of postgres (e.g. Greenplum) you can disable the default metrics with the `--disable-default-metrics`
flag. This removes all built-in metrics, and uses only metrics defined by queries in the `queries.yaml` file you supply
(so you must supply one, otherwise the exporter will return nothing but internal statuses and not your database).
Expand Down
18 changes: 8 additions & 10 deletions cmd/postgres_exporter/pg_setting.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
package main

import (
"database/sql"
"errors"
"fmt"
"math"
"strconv"
Expand All @@ -13,29 +11,29 @@ import (
)

// Query the pg_settings view containing runtime variables
func querySettings(ch chan<- prometheus.Metric, db *sql.DB) error {
log.Debugln("Querying pg_setting view")
func querySettings(ch chan<- prometheus.Metric, server *Server) error {
log.Debugf("Querying pg_setting view on %q", server)

// pg_settings docs: https://www.postgresql.org/docs/current/static/view-pg-settings.html
//
// NOTE: If you add more vartypes here, you must update the supported
// types in normaliseUnit() below
query := "SELECT name, setting, COALESCE(unit, ''), short_desc, vartype FROM pg_settings WHERE vartype IN ('bool', 'integer', 'real');"

rows, err := db.Query(query)
rows, err := server.db.Query(query)
if err != nil {
return errors.New(fmt.Sprintln("Error running query on database: ", namespace, err))
return fmt.Errorf("Error running query on database %q: %s %v", server, namespace, err)
}
defer rows.Close() // nolint: errcheck

for rows.Next() {
s := &pgSetting{}
err = rows.Scan(&s.name, &s.setting, &s.unit, &s.shortDesc, &s.vartype)
if err != nil {
return errors.New(fmt.Sprintln("Error retrieving rows:", namespace, err))
return fmt.Errorf("Error retrieving rows on %q: %s %v", server, namespace, err)
}

ch <- s.metric()
ch <- s.metric(server.String())
}

return nil
Expand All @@ -47,7 +45,7 @@ type pgSetting struct {
name, setting, unit, shortDesc, vartype string
}

func (s *pgSetting) metric() prometheus.Metric {
func (s *pgSetting) metric(server string) prometheus.Metric {
var (
err error
name = strings.Replace(s.name, ".", "_", -1)
Expand Down Expand Up @@ -79,7 +77,7 @@ func (s *pgSetting) metric() prometheus.Metric {
}

desc := newDesc(subsystem, name, shortDesc)
return prometheus.MustNewConstMetric(desc, prometheus.GaugeValue, val)
return prometheus.MustNewConstMetric(desc, prometheus.GaugeValue, val, server)
}

// TODO: fix linter override
Expand Down
24 changes: 12 additions & 12 deletions cmd/postgres_exporter/pg_setting_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ var fixtures = []fixture{
unit: "seconds",
err: "",
},
d: "Desc{fqName: \"pg_settings_seconds_fixture_metric_seconds\", help: \"Foo foo foo [Units converted to seconds.]\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_seconds_fixture_metric_seconds", help: "Foo foo foo [Units converted to seconds.]", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 5,
},
{
Expand All @@ -41,7 +41,7 @@ var fixtures = []fixture{
unit: "seconds",
err: "",
},
d: "Desc{fqName: \"pg_settings_milliseconds_fixture_metric_seconds\", help: \"Foo foo foo [Units converted to seconds.]\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_milliseconds_fixture_metric_seconds", help: "Foo foo foo [Units converted to seconds.]", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 5,
},
{
Expand All @@ -57,7 +57,7 @@ var fixtures = []fixture{
unit: "bytes",
err: "",
},
d: "Desc{fqName: \"pg_settings_eight_kb_fixture_metric_bytes\", help: \"Foo foo foo [Units converted to bytes.]\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_eight_kb_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 139264,
},
{
Expand All @@ -73,7 +73,7 @@ var fixtures = []fixture{
unit: "bytes",
err: "",
},
d: "Desc{fqName: \"pg_settings_16_kb_real_fixture_metric_bytes\", help: \"Foo foo foo [Units converted to bytes.]\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_16_kb_real_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 49152,
},
{
Expand All @@ -89,7 +89,7 @@ var fixtures = []fixture{
unit: "bytes",
err: "",
},
d: "Desc{fqName: \"pg_settings_16_mb_real_fixture_metric_bytes\", help: \"Foo foo foo [Units converted to bytes.]\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_16_mb_real_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 5.0331648e+07,
},
{
Expand All @@ -105,7 +105,7 @@ var fixtures = []fixture{
unit: "bytes",
err: "",
},
d: "Desc{fqName: \"pg_settings_32_mb_real_fixture_metric_bytes\", help: \"Foo foo foo [Units converted to bytes.]\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_32_mb_real_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 1.00663296e+08,
},
{
Expand All @@ -121,7 +121,7 @@ var fixtures = []fixture{
unit: "bytes",
err: "",
},
d: "Desc{fqName: \"pg_settings_64_mb_real_fixture_metric_bytes\", help: \"Foo foo foo [Units converted to bytes.]\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_64_mb_real_fixture_metric_bytes", help: "Foo foo foo [Units converted to bytes.]", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 2.01326592e+08,
},
{
Expand All @@ -137,7 +137,7 @@ var fixtures = []fixture{
unit: "",
err: "",
},
d: "Desc{fqName: \"pg_settings_bool_on_fixture_metric\", help: \"Foo foo foo\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_bool_on_fixture_metric", help: "Foo foo foo", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 1,
},
{
Expand All @@ -153,7 +153,7 @@ var fixtures = []fixture{
unit: "",
err: "",
},
d: "Desc{fqName: \"pg_settings_bool_off_fixture_metric\", help: \"Foo foo foo\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_bool_off_fixture_metric", help: "Foo foo foo", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 0,
},
{
Expand All @@ -169,7 +169,7 @@ var fixtures = []fixture{
unit: "seconds",
err: "",
},
d: "Desc{fqName: \"pg_settings_special_minus_one_value_seconds\", help: \"foo foo foo [Units converted to seconds.]\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_special_minus_one_value_seconds", help: "foo foo foo [Units converted to seconds.]", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: -1,
},
{
Expand All @@ -185,7 +185,7 @@ var fixtures = []fixture{
unit: "",
err: "",
},
d: "Desc{fqName: \"pg_settings_rds_rds_superuser_reserved_connections\", help: \"Sets the number of connection slots reserved for rds_superusers.\", constLabels: {}, variableLabels: []}",
d: `Desc{fqName: "pg_settings_rds_rds_superuser_reserved_connections", help: "Sets the number of connection slots reserved for rds_superusers.", constLabels: {}, variableLabels: [` + serverLabelName + `]}`,
v: 2,
},
{
Expand Down Expand Up @@ -233,7 +233,7 @@ func (s *PgSettingSuite) TestMetric(c *C) {

for _, f := range fixtures {
d := &dto.Metric{}
m := f.p.metric()
m := f.p.metric("")
m.Write(d) // nolint: errcheck

c.Check(m.Desc().String(), Equals, f.d)
Expand Down
Loading