contributing: doc to run a single test

[chaen]

It's my first time interacting with ruby and puppet stdlib (puppetlabs/puppetlabs-stdlib#1208), and I found this page extremely useful, but I wasted quite some time figuring how to run a specific test, so I thought I'd add it to the doc :-)

[sanfrancrisko]

Thanks for the feedback @chaen

It shouldn't be necessary to specify the SPEC= option to do this.

Also, it's possible to run a specific test in a file too, if you reference the line number:

pdk bundle exec rake spec spec/classes/my_test.rb:42

I can enhance the docs if you want, along with incorporating your suggestion.

Thanks again

[sanfrancrisko]

Suggested change

	% pdk bundle exec rake spec SPEC=<path to rb file>
	% pdk bundle exec rake spec <path to rb file>

[sanfrancrisko]

Suggested change

	To run a single test file, you can use the `SPEC=<path>` option
	To run a single test file, you can directly reference the spec file that you wish to run.

[sanfrancrisko]

Suggested change

	% pdk bundle exec rake spec SPEC=spec/functions/loadjson_spec.rb
	% pdk bundle exec rake spec spec/functions/loadjson_spec.rb

[chaen]

Hi @sanfrancrisko and sorry for the late reply !
I tried without the SPEC= but it does not work:

root@943f4a62533a:/tmp/puppetlabs-stdlib# pdk bundle exec rake spec SPEC=spec/functions/loadjson_spec.rb  
pdk (INFO): Using Ruby 2.7.3
pdk (INFO): Using Puppet 7.12.0
[...]

Coverage Report:

Total resources:   0
Touched resources: 0
Resource coverage: 100.00%

Finished in 0.48076 seconds (files took 1.45 seconds to load)
13 examples, 0 failures

root@943f4a62533a:/tmp/puppetlabs-stdlib# pdk bundle exec rake spec spec/functions/loadjson_spec.rb                                                                                                                                                           
pdk (INFO): Using Ruby 2.7.3                                                                                                                                                                                                                                  
pdk (INFO): Using Puppet 7.12.0                                                                                                                                                                                                                               
I, [2021-11-22T06:21:14.529857 #2020]  INFO -- : Creating symlink from spec/fixtures/modules/stdlib to /tmp/puppetlabs-stdlib                                                                                                                                 
I, [2021-11-22T06:21:14.529981 #2020]  INFO -- : Creating symlink from spec/fixtures/modules/test to /tmp/puppetlabs-stdlib/spec/fixtures/test                                                                                                                
Cloning into 'spec/fixtures/modules/provision'...                                                                                                                                                                                                             
Cloning into 'spec/fixtures/modules/facts'...                                                                                                                                                                                                                 
Cloning into 'spec/fixtures/modules/puppet_agent'...                                                                                                                                                                                                          
remote: Enumerating objects: 72, done.              
remote: Enumerating objects: 225, done.               
remote: Counting objects: 100% (72/72), done.
remote: Counting objects: 100% (225/225), done.     
remote: Enumerating objects: 67, done.                
remote: Counting objects: 100% (67/67), done.         
remote: Compressing objects: 100% (57/57), done.        
remote: Compressing objects: 100% (201/201), done.   
remote: Compressing objects: 100% (57/57), done.   
remote: Total 72 (delta 6), reused 46 (delta 4), pack-reused 0
Receiving objects: 100% (72/72), 33.66 KiB | 1.77 MiB/s, done.
Resolving deltas: 100% (6/6), done.                
remote: Total 67 (delta 9), reused 33 (delta 2), pack-reused 0
Receiving objects: 100% (67/67), 41.71 KiB | 2.20 MiB/s, done.
Resolving deltas: 100% (9/9), done.                       
remote: Total 225 (delta 34), reused 106 (delta 11), pack-reused 0
Receiving objects: 100% (225/225), 164.96 KiB | 2.39 MiB/s, done.
Resolving deltas: 100% (34/34), done.
/opt/puppetlabs/pdk/private/ruby/2.7.3/bin/ruby -I/opt/puppetlabs/pdk/share/cache/ruby/2.7.0/gems/rspec-core-3.10.1/lib:/root/.pdk/cache/ruby/2.7.0/gems/rspec-support-3.10.2/lib /opt/puppetlabs/pdk/share/cache/ruby/2.7.0/gems/rspec-core-3.10.1/exe/rspec 
--pattern spec/\{aliases,classes,defines,functions,hosts,integration,plans,tasks,type_aliases,types,unit\}/\*\*/\*_spec.rb
Run options: exclude {:bolt=>true}
                                                                                                                               
any2array
  is expected not to eq nil
  is expected to run any2array() and return []
  is expected to run any2array("") and return []
  is expected to run any2array(true) and return [true]                                                                         
  is expected to run any2array("one") and return ["one"]                                                                       
  is expected to run any2array("one", "two") and return ["one", "two"]                                                                                                                                                                                        
  is expected to run any2array([]) and return []                                                                               
  is expected to run any2array(["one"]) and return ["one"]                                                                     
  is expected to run any2array(["one", "two"]) and return ["one", "two"]                           
  is expected to run any2array({}) and return []                                                                               
  is expected to run any2array({"key"=>"value"}) and return ["key", "value"]                                                                                                                                                                                  
  is expected to run any2array("\u2030") and return ["\u2030"]                                                                 
  is expected to run any2array("\u7AF9") and return ["\u7AF9"]                                                                 
  is expected to run any2array("\u00DC") and return ["\u00DC"]                                                                 
  is expected to run any2array("\u2207") and return ["\u2207"]                                                                 
  is expected to run any2array("\u20AC", "\u4E07", "\u00D6", "\u2665", "\u5272") and return ["\u20AC", "\u4E07", "\u00D6", "\u2665", "\u5272"]
                                                                                                                               
any2bool                                                                                                                       
  is expected not to eq nil                                                                                                    
  is expected to run any2bool() and raise an Puppet::ParseError with the message matching /wrong number of arguments/i
  is expected to run any2bool(true) and return true                                                                                                                                                                                                           
  is expected to run any2bool(false) and return false                                                                          
  is expected to run any2bool("1.5") and return true                                                                           
  when testing stringy values that mean "true"             
  ....
  
  (and then I stopped because it goes for all the tests

[chaen]

Also, something that may be interesting in the doc for totally ignorant people (like me :-) ) is how I can run the tests for a specific version of puppet/ruby. As you can see, I have failures only for one version of that couple, but I am struggling in reproducing:
https://github.com/puppetlabs/puppetlabs-stdlib/runs/3859406073?check_suite_focus=true

I think ultimately what would be nice is a couple of commands, starting with docker run, which would put people in the correct environment they need to run pdk bundle exec rake spec.

I tried reproducing what's in the workflow file (https://github.com/puppetlabs/puppetlabs-stdlib/blob/15e13b2b63f0f4b24e68a46cd4c2703682006d8f/.github/workflows/spec.yml#L77-L130)

If I try to set the env variable PUPPET_GEM_VERSION, pdk complains

root@943f4a62533a:/tmp/puppetlabs-stdlib# PUPPET_GEM_VERSION=6.0 pdk bundle exec rake spec SPEC=spec/functions/loadjson_spec.rb 
pdk (INFO): Using Ruby 2.7.3
pdk (INFO): Using Puppet 7.12.0
pdk (WARN): PUPPET_GEM_VERSION is not supported by PDK. Use the --puppet-version option on your PDK command or set the PDK_PUPPET_VERSION environment variable instead

If I try with PDK_PUPPET_VERSION, I have another issue

root@943f4a62533a:/tmp/puppetlabs-stdlib# PDK_PUPPET_VERSION=6.0 pdk bundle exec rake spec SPEC=spec/functions/loadjson_spec.rb 
pdk (INFO): Using Ruby 2.5.9
pdk (INFO): Using Puppet 6.0.10
pdk (WARN): FACTER_GEM_VERSION is not supported by PDK.
Bundler could not find compatible versions for gem "puppet":
  In Gemfile:
    puppet (= 6.0.10)

    puppet-module-posix-system-r2.5 (~> 1.0) was resolved to 1.1.1, which depends on
      puppet_litmus (~> 0.20) was resolved to 0.30.0, which depends on
        bolt (>= 2.0.1, < 4.0.0) was resolved to 3.20.0, which depends on
          puppet (>= 6.18.0)