Support threading in joined column creation

[bkamins]

A minor PR to speed up column creation in joins using threading.

Its major benefit is for left, right and outerjoins having many extra columns (and producing over 1_000_000 rows)

[bkamins]

since getindex already uses threading this is only to ensure we are fast in case left and right table have very different processing times.

[bkamins]

I did not add threading for on-columns creation as in practice I expect that on-columns are minority of created columns and I avoid adding too much boilerplate code this way.

[bkamins]

Example run.

Setup:

julia> Random.seed!(1234);

julia> df1 = sort!(DataFrame(rand(10^7, 10), :auto));

julia> df2 = sort!(DataFrame(rand(10^7, 10), :auto));

Execution on main:

julia> @benchmark outerjoin($df1, $df2, on=:x1, renamecols="left"=>"right")
BenchmarkTools.Trial: 
  memory estimate:  3.47 GiB
  allocs estimate:  479
  --------------
  minimum time:     2.890 s (27.14% GC)
  median time:      2.938 s (28.04% GC)
  mean time:        2.938 s (28.04% GC)
  maximum time:     2.986 s (28.92% GC)
  --------------
  samples:          2
  evals/sample:     1

Execution on this PR:

julia> @benchmark outerjoin($df1, $df2, on=:x1, renamecols="left"=>"right")
BenchmarkTools.Trial: 
  memory estimate:  3.47 GiB
  allocs estimate:  636
  --------------
  minimum time:     1.380 s (7.04% GC)
  median time:      1.632 s (19.31% GC)
  mean time:        1.640 s (18.85% GC)
  maximum time:     1.915 s (23.12% GC)
  --------------
  samples:          4
  evals/sample:     1

(using 8 threads in both cases)

[nalimilan]

Cool! I just wish we could find a way to avoid putting if checks everywhere in the long term.

[nalimilan]

Why do you use let here but not above?

[bkamins]

You are right - let is not needed, as just writing cols_i = col_idx will ensure we get a new binding for cols_i. I will change this.

[nalimilan]

Can you add types to the signature? Maybe also add a ! at the end of the function name.

[bkamins]

OK - fixed

[bkamins]

find a way to avoid putting if checks everywhere in the long term.

When we stop supporting Julia 1.0 it will be possible to significantly simplify the codes.

[bkamins]

Thank you!