Skip to content

Commit 9174629

Browse files
authored
Merge pull request #18 from xenon8/fanny/fix-pg-elastic
Fix postgres and elastic, agent 5 cleanups
2 parents e0e7f09 + 235d894 commit 9174629

11 files changed

Lines changed: 264 additions & 124 deletions

.fixtures.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
fixtures:
22
repositories:
33
stdlib:
4-
repo: "https://github.com/puppetlabs/puppetlabs-stdlib.git" # TODO: why git: and not https:?
4+
repo: "https://github.com/puppetlabs/puppetlabs-stdlib.git"
55
ref: "v9.7.0" # Puppet >= 7.0.0 < 9.0.0
66
concat:
77
repo: "https://github.com/puppetlabs/puppetlabs-concat.git"

CHANGELOG.md

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -5,26 +5,27 @@ Changes
55
<!-- markdownlint-disable MD003 -->
66
<!-- markdownlint-disable MD001 -->
77

8-
# 4.0.0 / 2025-02-28
8+
# 4.0.0 / 2025-03-07
99

10-
This release has multiple breaking changes, you may need to update your module integration; note that module dependencies has been updated.
10+
This release has multiple breaking changes, you may need to update your module integration; note that module
11+
dependencies has been updated.
1112

12-
This release brings support for Puppet 8, which means all classes are now using defined class parameters types; this could break existing implementations.
13+
This release brings support for Puppet 8, which means all classes are now using defined class parameters types; this
14+
could break existing implementations.
1315

1416
* [FEATURE] Add Support for Puppet 8 ([#779])
1517
* [FEATURE] Update Module Dependencies including updates for StdLib, migrating to newer functions where appropriate ([#800])
16-
* [FEATURE] Class definitions updated with references to DataDog examples
18+
* [FEATURE] Class definitions updated with references to Datadog examples
1719
* [FEATURE] Update to CI Builds to work with Ruby 3
1820
* [BUGFIX] Fix issue where MSI path was not correctly parsed ([#814])
1921
* [BUGFIX] BREAKING - `datadog_agent::integrations::disk` now expects booleans for `use_mount`, `all_partitions` and `tag_by_filesystem`
20-
* [DEPRECATE] Drop support for Datadog Agent version 5, including removal of unit tests
2122
* [DEPRECATE] Drop support for Puppet 6 and below
22-
* [DEPRECATE] Remove support for supply a String to the SSL_Verify option on the elasticsearch integration. We now use tls_verify which matches core DataDog code.
23-
* [DEPRECATE] Support for Jenkins integrations is removed
24-
* [DEPRECATE] No longer possible to use the custom metrics parameter when using the PostgresSQL class
25-
* [DEPRECATE] `ganglia` configuration no longer supported as per DataDog Agent v6+
26-
* [DEPRECATE] `skip_event` setting on TCP Check class has been removed from DataDog integration
27-
* [DEPRECATE] Postgres implementation now only accepts booleans for `ssl` settings.
23+
* [DEPRECATE] Drop support for Datadog Agent version 5, including removal of unit tests
24+
* [DEPRECATE] Remove `ganglia`, `graphite`, `dogstreams`, `custom_emitters` and `use_curl_http_client` legacy configuration options which are no longer supported since Datadog Agent v6+
25+
* [DEPRECATE] Remove support for supplying a string to the `ssl_verify` option on the elasticsearch integration. We now use `tls_verify` which matches core Datadog code.
26+
* [DEPRECATE] Remove legacy Jenkins integration
27+
* [DEPRECATE] `skip_event` setting on TCP Check class has been removed from DataDog integration (removed since Datadog Agent v6.4+)
28+
* [DEPRECATE] Remove support for supplying a String to the `ssl_verify` option on the elasticsearch integration. Add updated `tls_*` options to match core Datadog code.
2829

2930
# 3.24.0 / 2025-02-25
3031

README.md

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@ This module installs the Datadog Agent and sends Puppet reports to Datadog.
44

55
### Requirements
66

7-
The Datadog Puppet module supports Linux and Windows and is compatible with Puppet >= 4.6.x or Puppet Enterprise version >= 2016.4. For detailed information on compatibility, check the [module page on Puppet Forge][1].
7+
The Datadog Puppet module supports Linux and Windows and is compatible with Puppet >= 7.34.x or Puppet Enterprise
8+
version >= 2021.7.x. For detailed information on compatibility, check the [module page on Puppet Forge][1].
89

910
### Installation
1011

@@ -16,13 +17,23 @@ puppet module install datadog-datadog_agent
1617

1718
#### Upgrading
1819

19-
- By default Datadog Agent v7.x is installed. To use an earlier Agent version, change the setting `agent_major_version`.
20-
- `agent5_enable` is no longer used, as it has been replaced by `agent_major_version`.
21-
- `agent6_extra_options` has been renamed to `agent_extra_options` since it applies to both Agent v6 and v7.
22-
- `agent6_log_file` has been renamed to `agent_log_file` since it applies to both Agent v6 and v7.
23-
- `agent5_repo_uri` and `agent6_repo_uri` become `agent_repo_uri` for all Agent versions.
24-
- `conf_dir` and `conf6_dir` become `conf_dir` for all Agent versions.
25-
- The repository file created on Linux is named `datadog` for all Agent versions instead of `datadog5`/`datadog6`.
20+
> [!IMPORTANT]
21+
> The Datadog Puppet Module v4.x drops support for Puppet <= 6 and Datadog Agent v5. To upgrade or install the Datadog
22+
> Agent v5+ on Puppet <= 6, use module v3.x.
23+
24+
- By default Datadog Agent v7.x is installed. To use Agent version 6, change the setting `agent_major_version`.
25+
- Agent v5-specific legacy options have been removed. Refer to the CHANGELOG.md for more details and the datadog_agent class comments for all available options.
26+
27+
> [!IMPORTANT]
28+
> Updates and breaking changes have been made in the below agent integrations:
29+
30+
- ElasticSearch **[BREAKING CHANGES]**
31+
- `ssl_verify` accepts only Boolean values
32+
- `tls_verify` options have been added
33+
- Disk Check **[BREAKING CHANGES]**
34+
- `use_mount`, `all_partitions`, and `tag_by_filesystem` accept only Boolean values
35+
- TCP Check
36+
- `skip_event` option has been removed sinced Datadog Agent v6.4+
2637

2738
### Configuration
2839

@@ -291,15 +302,15 @@ These variables can be set in the `datadog_agent` class to control settings in t
291302
292303
| variable name | description |
293304
|-----------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
294-
| `agent_major_version` | The version of the Agent to install: either 5, 6 or 7 (default: 7). |
305+
| `agent_major_version` | The version of the Agent to install: either 6 or 7 (default: 7). |
295306
| `agent_version` | Lets you pin a specific minor version of the Agent to install, for example: `1:7.16.0-1`. Leave empty to install the latest version. |
296307
| `collect_ec2_tags` | Collect an instance's custom EC2 tags as Agent tags by using `true`. |
297308
| `collect_instance_metadata` | Collect an instance's EC2 metadata as Agent tags by using `true`. |
298-
| `datadog_site` | The Datadog site to report to (Agent v6 and v7 only). Defaults to `datadoghq.com`, eg: `datadoghq.eu` or `us3.datadoghq.com`. |
309+
| `datadog_site` | The Datadog site to report to. Defaults to `datadoghq.com`, eg: `datadoghq.eu` or `us3.datadoghq.com`. |
299310
| `dd_url` | The Datadog intake server URL. You are unlikely to need to change this. Overrides `datadog_site` |
300311
| `host` | Overrides the node's host name. |
301312
| `local_tags` | An array of `<KEY:VALUE>` strings that are set as tags for the node. |
302-
| `non_local_traffic` | Allow other nodes to relay their traffic through this node. |
313+
| `non_local_traffic` | Allow other nodes to relay their DogstatsD traffic through this node. |
303314
| `apm_enabled` | A boolean to enable the APM Agent (defaults to false). |
304315
| `process_enabled` | A boolean to enable the process Agent (defaults to false). |
305316
| `scrub_args` | A boolean to enable the process cmdline scrubbing (defaults to true). |

manifests/init.pp

Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -49,9 +49,6 @@
4949
# $non_local_traffic
5050
# Enable you to use the agent as a proxy. Defaults to false.
5151
# See https://github.com/DataDog/dd-agent/wiki/Proxy-Configuration
52-
# $dogstreams
53-
# Optional array of logs to parse and custom parsers to use.
54-
# See https://github.com/DataDog/dd-agent/blob/ed5e698/datadog.conf.example#L149-L178
5552
# $log_level
5653
# Set value of 'log_level' variable. Default is 'info' as in dd-agent.
5754
# Valid values here are: critical, debug, error, fatal, info, warn and warning.
@@ -92,8 +89,6 @@
9289
# separate way of adding keys.
9390
# $skip_ssl_validation
9491
# Skip SSL validation.
95-
# $use_curl_http_client
96-
# Uses the curl HTTP client for the forwarder
9792
# $recent_point_threshold
9893
# Sets the threshold for accepting points.
9994
# String. Default: empty (30 second intervals)
@@ -147,12 +142,6 @@
147142
# $device_blacklist_re
148143
# Specifies pattern for device blacklisting.
149144
# String. Default: empty
150-
# $dogstreams
151-
# Specifies port for list of logstreams/modules to be used.
152-
# String. Default: empty
153-
# $custom_emitters
154-
# Specifies a comma seperated list of non standard emitters to be used
155-
# String. Default: empty
156145
# $agent_log_file
157146
# Specifies the log file location (Agent 6 and 7 only).
158147
# String. Default: empty
@@ -288,7 +277,6 @@
288277
String $puppetmaster_user = $settings::user,
289278
String $puppet_gem_provider = $datadog_agent::params::gem_provider,
290279
Boolean $non_local_traffic = false,
291-
Array $dogstreams = [],
292280
String $log_level = 'info',
293281
Boolean $log_to_syslog = true,
294282
String $service_ensure = 'running',
@@ -314,7 +302,6 @@
314302
String $extra_template = '',
315303
Boolean $skip_ssl_validation = false,
316304
Boolean $skip_apt_key_trusting = false,
317-
Boolean $use_curl_http_client = false,
318305
String $recent_point_threshold = '',
319306
Variant[Stdlib::Port, Pattern[/^\d*$/]] $listen_port = '',
320307
Optional[String] $additional_checksd = undef,
@@ -328,7 +315,6 @@
328315
String $dogstatsd_interval = '',
329316
Boolean $dogstatsd_normalize = true,
330317
String $device_blacklist_re = '',
331-
String $custom_emitters = '',
332318
String $agent_log_file = $datadog_agent::params::agent_log_file,
333319
String $collector_log_file = '',
334320
String $forwarder_log_file = '',

manifests/integrations/elasticsearch.pp

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,15 @@
3434
Boolean $pshard_stats = false,
3535
Optional[String] $ssl_cert = undef,
3636
Optional[String] $ssl_key = undef,
37-
Boolean $ssl_verify = true, #kept for backwards compatibility
37+
Boolean $ssl_verify = true, # kept for backwards compatibility
3838
Boolean $tls_verify = $ssl_verify,
39+
Boolean $tls_use_host_header = false,
40+
Boolean $tls_ignore_warning = false,
41+
Optional[String] $tls_cert = undef,
42+
Optional[String] $tls_private_key = undef,
43+
Optional[String] $tls_ca_cert = undef,
44+
Array $tls_protocols_allowed = [],
45+
Array $tls_ciphers = [],
3946
Array $tags = [],
4047
String $url = 'http://localhost:9200',
4148
Optional[String] $username = undef,
@@ -55,7 +62,14 @@
5562
'tls_verify' => $tls_verify,
5663
'tags' => $tags,
5764
'url' => $url,
58-
'username' => $username
65+
'username' => $username,
66+
'tls_use_host_header' => $tls_use_host_header,
67+
'tls_ignore_warning' => $tls_ignore_warning,
68+
'tls_cert' => $tls_cert,
69+
'tls_private_key' => $tls_private_key,
70+
'tls_ca_cert' => $tls_ca_cert,
71+
'tls_protocols_allowed' => $tls_protocols_allowed,
72+
'tls_ciphers' => $tls_ciphers,
5973
}]
6074
} elsif !$instances {
6175
$_instances = []

manifests/integrations/postgres.pp

Lines changed: 43 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,19 @@
1717
# $username
1818
# The username for the datadog user
1919
# $ssl
20-
# Boolean to enable SSL
20+
# This option determines whether or not and with what priority a secure SSL TCP/IP connection
21+
# is negotiated with the server. There are six modes:
22+
# - `disable`: Only tries a non-SSL connection.
23+
# - `allow`: First tries a non-SSL connection; if if fails, tries an SSL connection.
24+
# - `prefer`: First tries an SSL connection; if it fails, tries a non-SSL connection.
25+
# - `require`: Only tries an SSL connection. If a root CA file is present, verifies the certificate in
26+
# the same way as if verify-ca was specified.
27+
# - `verify-ca`: Only tries an SSL connection, and verifies that the server certificate is issued by a
28+
# trusted certificate authority (CA).
29+
# - `verify-full`: Only tries an SSL connection and verifies that the server certificate is issued by a
30+
# trusted CA and that the requested server host name matches the one in the certificate.
31+
#
32+
# For a detailed description of how these options work see https://www.postgresql.org/docs/current/libpq-ssl.html
2133
# $use_psycopg2
2234
# Boolean to flag connecting to postgres with psycopg2 instead of pg8000.
2335
# Warning, psycopg2 doesn't support ssl mode.
@@ -41,6 +53,10 @@
4153
# (10 + 10 per index)
4254
# $tags
4355
# Optional array of tags
56+
# $custom_metrics
57+
# A hash of custom metrics with the following keys - query, metrics,
58+
# relation, descriptors. Refer to this guide for details on those fields:
59+
# https://help.datadoghq.com/hc/en-us/articles/208385813-Postgres-custom-metric-collection-explained
4460
#
4561
# Sample Usage:
4662
#
@@ -50,6 +66,18 @@
5066
# username => 'datadog',
5167
# password => 'some_pass',
5268
# ssl => false,
69+
# custom_metrics => {
70+
# a_custom_query => {
71+
# query => "select tag_column, %s from table",
72+
# relation => false,
73+
# metrics => {
74+
# value_column => ["value_column.datadog.tag", "GAUGE"]
75+
# },
76+
# descriptors => [
77+
# ["tag_column", "tag_column.datadog.tag"]
78+
# ]
79+
# }
80+
# }
5381
# }
5482
#
5583
# Hiera Usage:
@@ -59,15 +87,23 @@
5987
# dbname: 'postgres'
6088
# username: 'datadog'
6189
# password: 'some_pass'
62-
# ssl: false
90+
# ssl: 'allow'
91+
# custom_metrics:
92+
# a_custom_query:
93+
# query: 'select tag_column, %s from table'
94+
# relation: false
95+
# metrics:
96+
# value_column: ["value_column.datadog.tag", "GAUGE"]
97+
# descriptors:
98+
# - ["tag_column", "tag_column.datadog.tag"]
6399
#
64100
class datadog_agent::integrations::postgres (
65101
Optional[String] $password = undef,
66102
String $host = 'localhost',
67103
String $dbname = 'postgres',
68104
Variant[String, Integer] $port = '5432',
69105
String $username = 'datadog',
70-
Boolean $ssl = false,
106+
String $ssl = 'allow',
71107
Boolean $use_psycopg2 = false,
72108
Boolean $collect_function_metrics = false,
73109
Boolean $collect_count_metrics = false,
@@ -76,6 +112,7 @@
76112
Boolean $collect_default_database = false,
77113
Array[String] $tags = [],
78114
Array[String] $tables = [],
115+
Hash $custom_metrics = {},
79116
Optional[Array] $instances = undef,
80117
) inherits datadog_agent::params {
81118
require datadog_agent
@@ -103,6 +140,7 @@
103140
'use_psycopg2' => $use_psycopg2,
104141
'tags' => $tags,
105142
'tables' => $tables,
143+
'custom_metrics' => $custom_metrics,
106144
'collect_function_metrics' => $collect_function_metrics,
107145
'collect_count_metrics' => $collect_count_metrics,
108146
'collect_activity_metrics' => $collect_activity_metrics,
@@ -124,4 +162,6 @@
124162
require => Package[$datadog_agent::params::package_name],
125163
notify => Service[$datadog_agent::params::service_name],
126164
}
165+
166+
create_resources('datadog_agent::integrations::postgres_custom_metric', $custom_metrics)
127167
}
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
##
2+
# The postgres_custom_metric defines a custom sql metric.
3+
# https://help.datadoghq.com/hc/en-us/articles/208385813-Postgres-custom-metric-collection-explained
4+
#
5+
# $query:
6+
# The custom metric SQL query. It must contain a '%s' for defining the metrics.
7+
#
8+
# $metrics:
9+
# a hash of column name to metric definition. a metric definition is an array
10+
# consisting of two columns -- the datadog metric name and the metric type.
11+
#
12+
# $relation:
13+
# ?
14+
#
15+
# $descriptor:
16+
# an array that maps an sql column's to a tag. Each descriptor consists of two
17+
# fields -- column name, and datadog tag.
18+
define datadog_agent::integrations::postgres_custom_metric (
19+
String $query,
20+
Hash $metrics,
21+
Boolean $relation = false,
22+
Array $descriptors = [],
23+
) {
24+
if $query !~ '^.*%s.*$' {
25+
fail('custom_metrics require %s for metric substitution')
26+
}
27+
}

0 commit comments

Comments
 (0)