Skip to content

Moises.botarro/adding new config fields for process agent scrubbing #426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
truthbk merged 6 commits into from
Apr 25, 2018
Merged

Moises.botarro/adding new config fields for process agent scrubbing #426

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
24c0942
Added default values for scrub_args and custom_sensitive_words; updat…
mbotarro Apr 18, 2018
03b96ef
Added test for conf and yaml files
mbotarro Apr 18, 2018
0f3736b
Format changes
mbotarro Apr 20, 2018
3f11608
Fixed regex patterns on data scrubbing tests
mbotarro Apr 23, 2018
404e646
Merge branch 'master' into moises.botarro/adding-new-config-fields-fo…
truthbk Apr 25, 2018
e0e0d3b
[process][spec] address test-case conflict - `enabled` not `process_e…
truthbk Apr 25, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,8 @@ Here are some of the other variables that be set in the datadog_agent class to c
| agent5_enable | boolean to install agent5 and override agent6 default |
| apm_enabled | boolean to enable the APM agent; defaults to true |
| process_enabled | boolean to enable the process agent; defaults to true |
| scrub_args | boolean to enable the process cmdline scrubbing; defaults to true |
| custom_sensitive_words| an array to add more words beyond the default ones used by the scrubbing feature; defaults to [] |
| agent6_extra_options | hash to provide additional configuration options to agent6. |

_NOTE: `agent6_extra_options` may be used to provide a fine grain control of additional agent6 config options. A deep merge is performed that may override options provided in the `datadog_agent` class parameters_
Expand Down
24 changes: 23 additions & 1 deletion manifests/init.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,12 @@
# $process_enabled
# String to enable the process/container agent
# Boolean. Default: false
# $scrub_args
# Boolean to enable or disable the process cmdline scrubbing by the process-agent
# Boolean. Default: true
# $custom_sensitive_words
# Array to add more words to be used on the process cdmline scrubbing by the process-agent
# Array. Default: []
#
# Actions:
#
Expand Down Expand Up @@ -262,6 +268,8 @@
$apm_enabled = $datadog_agent::params::apm_default_enabled,
$apm_env = '',
$process_enabled = $datadog_agent::params::process_default_enabled,
$scrub_args = $datadog_agent::params::process_default_scrub_args,
$custom_sensitive_words = $datadog_agent::params::process_default_custom_words,
Hash[String[1], Data] $agent6_extra_options = {},
$agent5_repo_uri = $datadog_agent::params::agent5_default_repo,
$agent6_repo_uri = $datadog_agent::params::agent6_default_repo,
Expand Down Expand Up @@ -341,6 +349,8 @@
validate_legacy(Boolean, 'validate_bool', $agent5_enable)
validate_legacy(String, 'validate_string', $apm_env)
validate_legacy(Boolean, 'validate_bool', $process_enabled)
validate_legacy(Boolean, 'validate_bool', $scrub_args)
validate_legacy(Array, 'validate_array', $custom_sensitive_words)
validate_legacy(String, 'validate_string', $agent5_repo_uri)
validate_legacy(String, 'validate_string', $agent6_repo_uri)
validate_legacy(String, 'validate_string', $apt_release)
Expand Down Expand Up @@ -490,14 +500,26 @@
order => '07',
}
}

if ($process_enabled == true) {
concat::fragment{ 'datadog process agent footer':
target => '/etc/dd-agent/datadog.conf',
content => template('datadog_agent/datadog_process_footer.conf.erb'),
order => '08',
}
}
} else {

# lint:ignore:quoted_booleans
$process_enabled_str = $process_enabled ? { true => 'true' , default => 'disabled' }
# lint:endignore
$base_extra_config = {
'apm_config' => { 'apm_enabled' => $apm_enabled },
'process_config' => { 'enabled' => $process_enabled_str },
'process_config' => {
'enabled' => $process_enabled_str,
'scrub_args' => $scrub_args,
'custom_sensitive_words' => $custom_sensitive_words,
},
}
$extra_config = deep_merge($base_extra_config, $agent6_extra_options)

Expand Down
28 changes: 15 additions & 13 deletions manifests/params.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,21 @@
# Sample Usage:
#
class datadog_agent::params {
$agent5_enable = false
$conf_dir = '/etc/dd-agent/conf.d'
$conf6_dir = '/etc/datadog-agent/conf.d'
$dd_user = 'dd-agent'
$dd_group = 'root'
$dd_groups = undef
$package_name = 'datadog-agent'
$service_name = 'datadog-agent'
$dogapi_version = 'installed'
$conf_dir_purge = false
$apt_default_release = 'stable'
$apm_default_enabled = false
$process_default_enabled = false
$agent5_enable = false
$conf_dir = '/etc/dd-agent/conf.d'
$conf6_dir = '/etc/datadog-agent/conf.d'
$dd_user = 'dd-agent'
$dd_group = 'root'
$dd_groups = undef
$package_name = 'datadog-agent'
$service_name = 'datadog-agent'
$dogapi_version = 'installed'
$conf_dir_purge = false
$apt_default_release = 'stable'
$apm_default_enabled = false
$process_default_enabled = false
$process_default_scrub_args = true
$process_default_custom_words = []

case $::operatingsystem {
'Ubuntu','Debian' : {
Expand Down
102 changes: 96 additions & 6 deletions spec/classes/datadog_agent_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,7 @@
)}
end
context 'with skip_ssl_validation set to true' do
let(:params) {{ :skip_ssl_validation => true,
let(:params) {{ :skip_ssl_validation => true,
:agent5_enable => true,
}}
it { should contain_concat__fragment('datadog header').with(
Expand Down Expand Up @@ -556,7 +556,7 @@
)}
end
context 'with ganglia_host set to localhost and ganglia_port set to 12345' do
let(:params) {{ :ganglia_host => 'testhost',
let(:params) {{ :ganglia_host => 'testhost',
:ganglia_port => '12345',
:agent5_enable => true,
}}
Expand Down Expand Up @@ -691,10 +691,10 @@
)}
end
context 'with service_discovery enabled' do
let(:params) {{ :service_discovery_backend => 'docker',
:sd_config_backend => 'etcd',
:sd_backend_host => 'localhost',
:sd_backend_port => '8080',
let(:params) {{ :service_discovery_backend => 'docker',
:sd_config_backend => 'etcd',
:sd_backend_host => 'localhost',
:sd_backend_port => '8080',
:sd_jmx_enable => true,
:agent5_enable => true,
}}
Expand Down Expand Up @@ -773,7 +773,47 @@
)}
end

context 'with data scrubbing disabled' do
let(:params) {{
:process_enabled => true,
:agent5_enable => true,
:scrub_args => false
}}
it { should contain_concat__fragment('datadog footer').with(
'content' => /^process_agent_enabled: true\n/,
)}
it { should contain_concat__fragment('datadog process agent footer').with(
'content' => /^\[process.config\]\n/,
)}
it { should contain_concat__fragment('datadog process agent footer').with(
'content' => /^scrub_args: false\n/,
)}
it { should contain_concat__fragment('datadog process agent footer').with(
'content' => /^custom_sensitive_words: \n/,
)}
end

context 'with data scrubbing enabled with custom sensitive_words' do
let(:params) {{
:process_enabled => true,
:agent5_enable => true,
:custom_sensitive_words => ['consul_token','dd_key']
}}
it { should contain_concat__fragment('datadog footer').with(
'content' => /^process_agent_enabled: true\n/,
)}
it { should contain_concat__fragment('datadog process agent footer').with(
'content' => /^\[process.config\]\n/,
)}
it { should contain_concat__fragment('datadog process agent footer').with(
'content' => /^scrub_args: true\n/,
)}
it { should contain_concat__fragment('datadog process agent footer').with(
'content' => /^custom_sensitive_words: consul_token,dd_key\n/,
)}
end

end
end

if DEBIAN_OS.include?(operatingsystem)
Expand Down Expand Up @@ -848,6 +888,12 @@
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ enabled: disabled\n/,
)}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ scrub_args: true\n/,
)}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ custom_sensitive_words: \[\]\n/,
)}
end
end

Expand Down Expand Up @@ -931,6 +977,50 @@
)}
end
end

context 'with data scrubbing custom options' do
context 'with data scrubbing disabled' do
let(:params) {{
:process_enabled => true,
:scrub_args => false
}}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^process_config:\n/,
)}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ process_enabled: 'true'\n/,
)}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ scrub_args: false\n/,
)}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ custom_sensitive_words: \[\]\n/,
)}
end

context 'with data scrubbing enabled with custom sensitive_words' do
let(:params) {{
:process_enabled => true,
:custom_sensitive_words => ['consul_token','dd_key']
}}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^process_config:\n/,
)}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ process_enabled: 'true'\n/,
)}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ scrub_args: true\n/,
)}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ -\ consul_token\n/,
)}
it { should contain_file('/etc/datadog-agent/datadog.yaml').with(
'content' => /^\ \ -\ dd_key\n/,
)}

end
end
end
end
end
Expand Down
5 changes: 5 additions & 0 deletions templates/datadog_process_footer.conf.erb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<% if @process_enabled -%>
[process.config]
scrub_args: <%= @scrub_args %>
custom_sensitive_words: <%= @custom_sensitive_words.join(',') %>
<% end -%>
⚡