pendding in DistanceAnalyzer.scala at line89:closePointsInsideBoxes.union (closePointsInDifferentBoxes)

[xspring14]

my data has 1.8million samples and each sample has 64 dimension

[Lllll921]

Did you solve it? Same problem here

[longjuncai1994]

The same problem with me
I found the pendding code in DistanceAnalyzer.countClosePointsWithinPartition.
It will build a boxesTree when create PartitionInedex instance in this function.
And then find the loop code in BoxCalculator.generateCombinationOfSplit.
It's a recurse function that will create a very large collection,and then cause my Spark Executor oom.
But i dont know how to solve it