Skip to content

Commit a493e43

Browse files
andygroveandygrove
committed
regenerate docs
1 parent 241fbdf commit a493e43

2 files changed

Lines changed: 34 additions & 47 deletions

File tree

‎docs/source/user-guide/latest/compatibility.md‎

Lines changed: 33 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -182,27 +182,22 @@ suffixes (e.g. `Europe/Moscow`), and the full Spark timestamp year range
182182
- **float -> string**: There can be differences in precision. For example, the input "1.4E-45" will produce 1.0E-45 instead of 1.4E-45
183183
- **string -> date**: Only supports years between 262143 BC and 262142 AD
184184
<!-- prettier-ignore-end -->
185-
<!--END:CAST_LEGACY_TABLE-->
186-
187-
### Try Mode
188-
189-
<!--BEGIN:CAST_TRY_TABLE-->
190-
<!-- prettier-ignore-start -->
191-
192-
| | binary | boolean | byte | date | decimal | double | float | integer | long | short | string | timestamp |
193-
| --------- | ------ | ------- | ---- | ---- | ------- | ------ | ----- | ------- | ---- | ----- | ------ | --------- |
194-
| binary | - | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | C | N/A |
195-
| boolean | N/A | - | C | N/A | C | C | C | C | C | C | C | U |
196-
| byte | U | C | - | N/A | C | C | C | C | C | C | C | C |
197-
| date | N/A | U | U | - | U | U | U | U | U | U | C | C |
198-
| decimal | N/A | C | C | N/A | - | C | C | C | C | C | C | C |
199-
| double | N/A | C | C | N/A | I | - | C | C | C | C | C | C |
200-
| float | N/A | C | C | N/A | I | C | - | C | C | C | C | C |
201-
| integer | U | C | C | N/A | C | C | C | - | C | C | C | C |
202-
| long | U | C | C | N/A | C | C | C | C | - | C | C | C |
203-
| short | U | C | C | N/A | C | C | C | C | C | - | C | C |
204-
| string | C | C | C | C | C | C | C | C | C | C | - | C |
205-
| timestamp | N/A | U | U | C | U | U | U | U | C | U | C | - |
185+
<!--BEGIN:CAST_TRY_TABLE-->
186+
<!-- prettier-ignore-start -->
187+
| | binary | boolean | byte | date | decimal | double | float | integer | long | short | string | timestamp |
188+
| --------- | ------ | ------- | ---- | ---- | ------- | ------ | ----- | ------- | ---- | ----- | ------ | --------- |
189+
| binary | - | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | C | N/A |
190+
| boolean | N/A | - | C | N/A | C | C | C | C | C | C | C | U |
191+
| byte | U | C | - | N/A | C | C | C | C | C | C | C | C |
192+
| date | N/A | U | U | - | U | U | U | U | U | U | C | C |
193+
| decimal | N/A | C | C | N/A | - | C | C | C | C | C | C | C |
194+
| double | N/A | C | C | N/A | I | - | C | C | C | C | C | C |
195+
| float | N/A | C | C | N/A | I | C | - | C | C | C | C | C |
196+
| integer | U | C | C | N/A | C | C | C | - | C | C | C | C |
197+
| long | U | C | C | N/A | C | C | C | C | - | C | C | C |
198+
| short | U | C | C | N/A | C | C | C | C | C | - | C | C |
199+
| string | C | C | C | C | C | C | C | C | C | C | - | C |
200+
| timestamp | N/A | U | U | C | U | U | U | U | C | U | C | - |
206201

207202
**Notes:**
208203

@@ -212,27 +207,22 @@ suffixes (e.g. `Europe/Moscow`), and the full Spark timestamp year range
212207
- **float -> string**: There can be differences in precision. For example, the input "1.4E-45" will produce 1.0E-45 instead of 1.4E-45
213208
- **string -> date**: Only supports years between 262143 BC and 262142 AD
214209
<!-- prettier-ignore-end -->
215-
<!--END:CAST_TRY_TABLE-->
216-
217-
### ANSI Mode
218-
219-
<!--BEGIN:CAST_ANSI_TABLE-->
220-
<!-- prettier-ignore-start -->
221-
222-
| | binary | boolean | byte | date | decimal | double | float | integer | long | short | string | timestamp |
223-
| --------- | ------ | ------- | ---- | ---- | ------- | ------ | ----- | ------- | ---- | ----- | ------ | --------- |
224-
| binary | - | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | C | N/A |
225-
| boolean | N/A | - | C | N/A | C | C | C | C | C | C | C | U |
226-
| byte | U | C | - | N/A | C | C | C | C | C | C | C | C |
227-
| date | N/A | U | U | - | U | U | U | U | U | U | C | C |
228-
| decimal | N/A | C | C | N/A | - | C | C | C | C | C | C | C |
229-
| double | N/A | C | C | N/A | I | - | C | C | C | C | C | C |
230-
| float | N/A | C | C | N/A | I | C | - | C | C | C | C | C |
231-
| integer | U | C | C | N/A | C | C | C | - | C | C | C | C |
232-
| long | U | C | C | N/A | C | C | C | C | - | C | C | C |
233-
| short | U | C | C | N/A | C | C | C | C | C | - | C | C |
234-
| string | C | C | C | C | C | C | C | C | C | C | - | C |
235-
| timestamp | N/A | U | U | C | U | U | U | U | C | U | C | - |
210+
<!--BEGIN:CAST_ANSI_TABLE-->
211+
<!-- prettier-ignore-start -->
212+
| | binary | boolean | byte | date | decimal | double | float | integer | long | short | string | timestamp |
213+
| --------- | ------ | ------- | ---- | ---- | ------- | ------ | ----- | ------- | ---- | ----- | ------ | --------- |
214+
| binary | - | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | C | N/A |
215+
| boolean | N/A | - | C | N/A | C | C | C | C | C | C | C | U |
216+
| byte | U | C | - | N/A | C | C | C | C | C | C | C | C |
217+
| date | N/A | U | U | - | U | U | U | U | U | U | C | C |
218+
| decimal | N/A | C | C | N/A | - | C | C | C | C | C | C | C |
219+
| double | N/A | C | C | N/A | I | - | C | C | C | C | C | C |
220+
| float | N/A | C | C | N/A | I | C | - | C | C | C | C | C |
221+
| integer | U | C | C | N/A | C | C | C | - | C | C | C | C |
222+
| long | U | C | C | N/A | C | C | C | C | - | C | C | C |
223+
| short | U | C | C | N/A | C | C | C | C | C | - | C | C |
224+
| string | C | C | C | C | C | C | C | C | C | C | - | C |
225+
| timestamp | N/A | U | U | C | U | U | U | U | C | U | C | - |
236226

237227
**Notes:**
238228

@@ -241,7 +231,4 @@ suffixes (e.g. `Europe/Moscow`), and the full Spark timestamp year range
241231
- **float -> decimal**: There can be rounding differences
242232
- **float -> string**: There can be differences in precision. For example, the input "1.4E-45" will produce 1.0E-45 instead of 1.4E-45
243233
- **string -> date**: Only supports years between 262143 BC and 262142 AD
244-
<!-- prettier-ignore-end -->
245-
<!--END:CAST_ANSI_TABLE-->
246-
247-
See the [tracking issue](https://github.com/apache/datafusion-comet/issues/286) for more details.
234+
<!-- prettier-ignore-end -->

‎docs/source/user-guide/latest/configs.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Comet provides the following configuration settings.
2929
|--------|-------------|---------------|
3030
| `spark.comet.scan.enabled` | Whether to enable native scans. When this is turned on, Spark will use Comet to read supported data sources (currently only Parquet is supported natively). Note that to enable native vectorized execution, both this config and `spark.comet.exec.enabled` need to be enabled. | true |
3131
| `spark.comet.scan.icebergNative.dataFileConcurrencyLimit` | The number of Iceberg data files to read concurrently within a single task. Higher values improve throughput for tables with many small files by overlapping I/O latency, but increase memory usage. Values between 2 and 8 are suggested. | 1 |
32-
| `spark.comet.scan.icebergNative.enabled` | Whether to enable native Iceberg table scan using iceberg-rust. When enabled, Iceberg tables are read directly through native execution, bypassing Spark's DataSource V2 API for better performance. | false |
32+
| `spark.comet.scan.icebergNative.enabled` | Whether to enable native Iceberg table scan using iceberg-rust. When enabled, Iceberg tables are read directly through native execution, bypassing Spark's DataSource V2 API for better performance. | true |
3333
| `spark.comet.scan.unsignedSmallIntSafetyCheck` | Parquet files may contain unsigned 8-bit integers (UINT_8) which Spark maps to ShortType. When this config is true (default), Comet falls back to Spark for ShortType columns because we cannot distinguish signed INT16 (safe) from unsigned UINT_8 (may produce different results). Set to false to allow native execution of ShortType columns if you know your data does not contain unsigned UINT_8 columns from improperly encoded Parquet files. For more information, refer to the [Comet Compatibility Guide](https://datafusion.apache.org/comet/user-guide/compatibility.html). | true |
3434
| `spark.hadoop.fs.comet.libhdfs.schemes` | Defines filesystem schemes (e.g., hdfs, webhdfs) that the native side accesses via libhdfs, separated by commas. Valid only when built with hdfs feature enabled. | |
3535
<!-- prettier-ignore-end -->

0 commit comments

Comments
 (0)
⚡