Environment
Delta-rs version: 0.19.1 (rev 81f79169f8d128435b0efa1ad90a6167a8313674)
Binding:
Environment:
- Cloud provider: On-Prem
- OS: MacOS
- Other:
Bug
What happened: The DeltaTable::peek_next_commit() function panics when it reads commit log bytes with invalid json
What you expected to happen: It should return a DeltaTableError to the caller
How to reproduce it:
Run this unit test
#[tokio::test]
async fn test_delta_commit_reader_returns_error() {
let memory_store = Arc::new(InMemory::new());
let log_path = Path::from("_delta_log/00000000000000000001.json");
let log_content = r#"{invalid_json"#;
memory_store
.put(&log_path, log_content.into())
.await
.expect("Failed to write log file");
let table_uri = "memory:///delta-table";
let table = DeltaTableBuilder::from_valid_uri(&table_uri).unwrap()
.with_storage_backend(memory_store, Url::parse(&table_uri).unwrap())
.build()
.context("failed to build Delta table").unwrap();
let result = table.peek_next_commit(0).await;
assert!(result.is_err());
}
More details:
The snippet of problematic code:
pub async fn peek_next_commit(
&self,
current_version: i64,
) -> Result<PeekCommit, DeltaTableError> {
let next_version = current_version + 1;
let commit_log_bytes = match self.log_store.read_commit_entry(next_version).await {
Ok(Some(bytes)) => Ok(bytes),
Ok(None) => return Ok(PeekCommit::UpToDate),
Err(err) => Err(err),
}?;
let actions = logstore::get_actions(next_version, commit_log_bytes).await;
Ok(PeekCommit::New(next_version, actions.unwrap()))
}
Environment
Delta-rs version: 0.19.1 (rev 81f79169f8d128435b0efa1ad90a6167a8313674)
Binding:
Environment:
Bug
What happened: The
DeltaTable::peek_next_commit()function panics when it reads commit log bytes with invalid jsonWhat you expected to happen: It should return a
DeltaTableErrorto the callerHow to reproduce it:
Run this unit test
More details:
The snippet of problematic code: