Skip to content

Agent 7 support #588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
albertvaka merged 38 commits into from
Dec 18, 2019
Merged

Agent 7 support #588

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
85ff117
Updated README
albertvaka Dec 2, 2019
2ea3cc8
Remove agent5/6 from variable names
albertvaka Dec 2, 2019
2653d72
Remove uses of validate_legacy
albertvaka Dec 2, 2019
d36a56c
Merge repository classes into a single one for all agent versions
albertvaka Dec 2, 2019
774a50f
Unify conf_dir setting
albertvaka Dec 2, 2019
a05d2e9
Oops
albertvaka Dec 2, 2019
dddcdcb
Bump major version
albertvaka Dec 2, 2019
a33aa20
Update README
albertvaka Dec 2, 2019
8da3665
Fix test
albertvaka Dec 2, 2019
edd9473
Rmove service_name and package_name parameters
albertvaka Dec 2, 2019
59384e0
Add types to some parameters
albertvaka Dec 2, 2019
6b7f316
Do not change group ownership to root
albertvaka Dec 2, 2019
54ee365
Fail if agent_major_version out of range
albertvaka Dec 2, 2019
dcf1416
Re-add agent5 config dir so we can ensure => absent
albertvaka Dec 2, 2019
ddafbb4
Make linter happy
albertvaka Dec 3, 2019
da9f32d
README
albertvaka Dec 3, 2019
9db150f
Update README.md
albertvaka Dec 9, 2019
bf7a39d
More types
albertvaka Dec 9, 2019
f62859e
Fix agent6 mentions in comments
albertvaka Dec 9, 2019
01753ae
Ensure datadog-beta repo is absent
albertvaka Dec 9, 2019
261153e
Mention masterless setups in README
albertvaka Dec 9, 2019
6fef12a
Rename CONF_DIR6 to CONF_DIR
albertvaka Dec 9, 2019
92788f7
Use context blocks for Agent 5/6/7
albertvaka Dec 9, 2019
b508316
Alignment
albertvaka Dec 9, 2019
83ca7a4
Add authors
albertvaka Dec 9, 2019
03b7c19
Fix gem name
albertvaka Dec 9, 2019
153f782
Logic to auto-detect agent_major_version (#589)
albertvaka Dec 10, 2019
4aedd98
Merge remote-tracking branch 'origin/master' into albertvaka/agent-7-…
albertvaka Dec 16, 2019
4a4c7f6
Update changelog
albertvaka Dec 16, 2019
b8d485d
Point to A7 when installing latest on Windows
albertvaka Dec 17, 2019
7128420
Document agent_repo_uri and apt_release
albertvaka Dec 17, 2019
aac9500
Fix regex to include -suffix in addition to ~suffix
albertvaka Dec 17, 2019
a5a1c45
Fix ubuntu tests
albertvaka Dec 17, 2019
3c54fe9
Fix systemctl command
albertvaka Dec 17, 2019
75c1203
Make major version detection tests be Linux-only
albertvaka Dec 18, 2019
41116d8
Changes as per CR
albertvaka Dec 18, 2019
0d8bdf0
Update CHANGELOG.md
albertvaka Dec 18, 2019
b85f666
Update release date
albertvaka Dec 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 21 additions & 87 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,29 +6,18 @@ Puppet & Datadog
[![Puppet Forge Downloads](https://img.shields.io/puppetforge/dt/datadog/datadog_agent.svg)](https://forge.puppetlabs.com/datadog/datadog_agent)


Description
-----------
## Description

A module to:

1. Install the [DataDog][1] Agent.
1. Install the [Datadog][1] Agent.
2. Send Puppet run reports to [Datadog][1].

Releases
--------

The current version of this Puppet module is compatible with Puppet >= 4.6.x. For users running on older versions of Puppet the legacy module, series 1.x, should support most use-cases.

The majority of users should be able to use the current version, as the Puppet versions supported in the 1.x series of the module have been EOL'd.

Some features might be back-ported if there's enough demand, but in general only maintenance is performed on the 1.x series of the module. Future feature development is performed on the newer Puppet module.

Version 2.x
-----------

## Requirements

For detailed information on compatibility, check the [module page][2] on the Puppet forge.
The current version of this Puppet module supports both Linux and Windows and is compatible with Puppet >= 4.6.x.

For detailed information on compatibility, check the [module page on Puppet Forge][2].

## Installation

Expand All @@ -46,56 +35,24 @@ class{ 'datadog_agent':
}
```

### Upgrade from the previous module version 1.x

Most of the manifests are backward compatible. However, there are important changes to the main manifest:

- By default Datadog Agent 6.x is installed.
- To use Agent 5.x in your nodes set the `agent5_enable` boolean parameter to `True`.
- Enhanced support for our APM Tracing Agent.
- Enhanced support for our Process Agent.
- Enhanced support for additional `agent6` options.
- APM Tracing and Process Agents are now opt-in options and are disabled by default.
- The main `datadog_agent` class has parameters to enable and configure the Agents easily.
- Increased granularity for apt/yum repo configuration.
- New location for Datadog Puppet reporting configuration.


Version 1.x
-----------

## Requirements

Puppet >=2.7.x and <=4.2.x. For detailed information on compatibility, check the [module page][2] on the Puppet forge.

## Installation

Install `datadog_agent` as a module in your Puppet master's module path.

```
puppet module install datadog-datadog_agent --version 1.12.1
```

### Upgrade from previous git manual install 0.x (unreleased)
### Upgrade from the previous module version 2.x

You can keep using the `datadog` module but it becomes legacy with the release of `datadog_agent` 1.0.0. Upgrade to get new features, and use the Puppet forge system which is easier for maintenance.

* Delete the Datadog module `rm -r /etc/puppet/modules/datadog`
* Install the new module from the Puppet forge `puppet module install datadog-datadog_agent`
* Update your manifests with the new module class, basically replace `datadog` by `datadog_agent`
- By default Datadog Agent 7.x is installed. To use an earlier Agent version change the setting `agent_major_version`.
- `agent5_enable` is no longer used, as it has been replaced by `agent_major_version`.
- `agent6_extra_options` has been renamed to `agent_extra_options` since it now applies to both Agent version 6 and 7.
- `agent6_log_file` has been renamed to `agent_log_file` since it now applies to both Agent version 6 and 7.
- `agent5_repo_uri` and `agent6_repo_uri` become `agent_repo_uri` for all Agent versions.
- `conf_dir` and `conf6_dir` become `conf_dir` for all Agent versions.
- The repository file created on Linux is now be named `datadog` for all agent versions instead of `datadog5`/`datadog6`.
Comment thread
albertvaka marked this conversation as resolved.
Outdated

Usage
-----

The following instructions are for both Puppet module versions (1.x, 2.x).

Note: 2.x installs Agent v6 by default, whereas 1.x defaults to Agent v5.

Once the `datadog_agent` module is installed on your `puppetserver`/`puppetmaster`, follow these configuration steps:
Comment thread
albertvaka marked this conversation as resolved.
Outdated

1. Update the default class parameters with your [API key][3].

2. Specify the module on any nodes you wish to install the DataDog Agent.
2. Specify the module on any nodes you wish to install the Datadog Agent.

```
include datadog_agent
Expand All @@ -119,12 +76,7 @@ Once the `datadog_agent` module is installed on your `puppetserver`/`puppetmaste
```

- To support reporting, your Puppet master needs the [dogapi][4] gem installed. To install, either run the Puppet Agent on your master with this configuration or install it manually with `gem`. You might need to restart your `puppetserver` service for the freshly installed `dogapi-rb` gem to be picked up.
- On version 2.x, `puppetserver_gem` is defined as a module dependency, it is installed automatically when the module is installed.
- On version 1.x:
- If you are on Puppet Enterprise or POSS (ie. >=3.7.0), there is a soft dependency for reporting on the `puppetserver_gem` module. Install with `puppet module install puppetlabs-puppetserver_gem` - installing manually with `gem` does *not* work.
- You may install `dogapi-rb` with `gem` as the system-level Ruby is used.
- The gem provider is configurable by setting it to `puppetserver_gem` (set by default on PE/POSS (>=3.7.0)) or `gem` if on older versions of Puppet.
- For users running puppet in standalone/masterless mode you will need to set the `puppet_gem_provider` to `puppet_gem` (or `gem` depending on versions) to ensure the `dogapi-rb` is available.
- `puppetserver_gem` is defined as a module dependency, it is installed automatically when the module is installed.

3. Include any other integrations you want the agent to use, for example:

Expand Down Expand Up @@ -306,50 +258,32 @@ datadog_agent::tags:

These variables can be set in the `datadog_agent` class to control settings in the Agent:

#### 2.x

| variable name | description |
|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `agent_major_version` | The version of the Agent to install: either 5, 6 or 7 (default: 7). |
Copy link
Copy Markdown
Contributor

@KSerrania KSerrania Dec 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think agent_repo_uri should be mentioned here, as it's the only way to use a custom repo. For ubuntu / debian, agent_default_release should be documented as well.

Moreover, the behavior and expected values of agent_repo_uri should be indicated (right now, you have to dig through the source code to know what you're supposed to do):

  • for apt, the repo url (without release branch and component) (eg. https://apt.datadoghq.com/)
  • for yum, the repo url with the release branch, component and arch (eg. https://yum.datadoghq.com/stable/7/x86_64)
  • for Windows, the bucket url prefix (eg. https://https://s3.amazonaws.com/ddagent-windows-stable/)

Copy link
Copy Markdown
Contributor Author

@albertvaka albertvaka Dec 17, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was undocumented before A7 support, so I left it as it was. I documented both in init.pp (in the source code, but that's where most parameter descriptions are).

| `agent_version` | Lets you pin a specific minor version of the agent to install, eg: `1:7.16.0-1`. Leave empty to install the latest version (not recommended). |
| `collect_ec2_tags` | Set this to yes to have an instance's custom EC2 tags used as agent tags. |
| `collect_instance_metadata` | Set this to yes to have an instance's EC2 metadata used as agent tags. |
| `datadog_site` | The Datadog site to report to. Defaults to `datadoghq.com`, set to `datadoghq.eu` to report to the EU site. Supported since v2.4.0 of the module, and only with Agent v6+ |
| `dd_url` | The Datadog intake server URL. You are unlikely to need to change this. Overrides `datadog_site` |
| `datadog_site` | The Datadog site to report to. Defaults to `datadoghq.com`, set to `datadoghq.eu` to report to the EU site (Agent versions 6 and 7 only). |
| `dd_url` | The Datadog intake server URL. You are unlikely to need to change this. Overrides `datadog_site` |
| `host` | Overrides the node's host name. |
| `local_tags` | An array of <KEY:VALUE> strings that are set as tags for the node. |
| `non_local_traffic` | Set this to allow other nodes to relay their traffic through this one. |
| `agent5_enable` | A boolean to install Agent v5 and override the Agent v6 default. |
| `apm_enabled` | A boolean to enable the APM Agent (defaults to false). |
| `apm_analyzed_spans` | A hash to add APM events for the Trace Search & Analytics tool. (defaults to undef). For example: `{ 'app\|rails.request' => 1, 'service-name\|operation-name' => 0.8 }` |
| `process_enabled` | A boolean to enable the process agent (defaults to false). |
| `scrub_args` | A boolean to enable the process cmdline scrubbing (defaults to true). |
| `custom_sensitive_words` | An array to add more words beyond the default ones used by the scrubbing feature (defaults to []). |
| `logs_enabled` | A boolean to enable the logs agent (defaults to false). |
| `container_collect_all` | A boolean to enable logs collection for all containers. |
| `agent6_extra_options` | A hash to provide additional configuration options to Agent v6. |
| `agent_extra_options` | A hash to provide additional configuration options (Agent versions 6 and 7 only). |
| `hostname_extraction_regex` | A regex used to extract the hostname captured group to report the run in Datadog instead of reporting the Puppet nodename, for example:<br>`'^(?<hostname>.*\.datadoghq\.com)(\.i-\w{8}\..*)?$'` |

##### Notes:

- `agent6_extra_options` is used to provide a fine grain control of additional Agent v6 config options. A deep merge is performed that may override options provided in the `datadog_agent` class parameters.
- `agent_extra_options` is used to provide a fine grain control of additional Agent v6/v7 config options. A deep merge is performed that may override options provided in the `datadog_agent` class parameters.
- `hostname_extraction_regex` is useful when the Puppet module and the Datadog Agent are reporting different host names for the same host in the infrastructure list.

#### 1.x

| variable name | description |
|-----------------------------|---------------------------------------------------------------------------|
| `collect_ec2_tags` | Set this to yes to have an instance's custom EC2 tags used as agent tags. |
| `collect_instance_metadata` | Set this to yes to have an instance's EC2 metadata used as agent tags. |
| `dd_url` | The Datadog intake server URL. You are unlikely to need to change this. |
| `host` | Overrides the node's host name. |
| `local_tags` | An array of <KEY:VALUE> strings that are set as tags for the node. |
| `non_local_traffic` | Set this to allow other nodes to relay their traffic through this one. |
| `agent6_enable` | A boolean to install Agent v6 and override the Agent v5 default. |

### Proxy Settings

To connect to the Internet through a proxy, set `proxy_host`, `proxy_port`, `proxy_user` and `proxy_password`.


Module Development and Testing
==============================

Expand Down
31 changes: 0 additions & 31 deletions lib/facter/agent_repo.rb
View file
Open in desktop

This file was deleted.

Loading
⚡