Skip to content

Commit 2a9a713

Browse files
committed
Rolling window queries
CubeStore extension to implement rolling window measures in CubeJS. Tests are in the CubeStore repository. The syntax is of the form: ``` SELECT dim, key1, key2, other, ROLLING(SUM(x) RANGE BETWEEN 7 PRECEDING AND UNBOUNDED FOLLOWING) ROLLING(AVG(x) RANGE BETWEEN 7 PRECEDING AND UNBOUNDED FOLLOWING) FROM input ROLLING_WINDOW DIMENSION dim PARTITION BY key1, key2 FROM 0 to 10 EVERY 2 ``` Semantics are roughly: - compute rolling window aggregations over input data, - window "rolls over" the `DIMENSION` column, only values defined in the range by `FROM .. TO .. EVERY ..` are reported, - each "group" defined by columns in `PARTITION BY` is handled and reported independently, Current limitations: - only integer ranges (timestamps and intervals are coming), - only ranges with up to 10M points are supported to avoid infinite loops are accidental DOS. This is still a fairly large limit and can lead to DOS given enough input data.
1 parent 7b44028 commit 2a9a713

13 files changed

Lines changed: 954 additions & 58 deletions

File tree

datafusion/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ ahash = "0.7"
4848
hashbrown = "0.11"
4949
arrow = { git = "https://github.com/cube-js/arrow-rs.git", branch = "cube", features = ["prettyprint"] }
5050
parquet = { git = "https://github.com/cube-js/arrow-rs.git", branch = "cube", features = ["arrow"] }
51-
sqlparser = "0.9.0"
51+
sqlparser = { git = "https://github.com/cube-js/sqlparser-rs.git", rev = "a04701767d9d6a9eb99cce9a64dee7e1b7c25b7c" }
5252
paste = "^1.0"
5353
num_cpus = "1.13.0"
5454
chrono = "0.4"

datafusion/src/cube_ext/mod.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
pub mod alias;
1919
pub mod join;
2020
pub mod joinagg;
21+
pub mod rolling;
2122
pub mod sequence;
2223
pub mod stream;
2324
pub mod util;

0 commit comments

Comments
 (0)