TIMESTAMP inside STRUCT fails with 'failed to convert to time.Time type'

[timveil]

Bug Description

When a query returns a STRUCT containing a TIMESTAMP field, the emulator fails with:

failed to scan rows: failed to convert 1742040000.0 to time.Time type

This prevents any query that returns TIMESTAMP values nested inside STRUCT from working. The same queries work correctly against real BigQuery. TIMESTAMP works fine as a top-level column — the issue is specifically when TIMESTAMP is a field inside a STRUCT.

Steps to Reproduce

Run any of these queries against the emulator:

-- Simple STRUCT with TIMESTAMP
SELECT STRUCT(TIMESTAMP '2025-03-15 12:00:00 UTC' as mydate) as a

-- UNNEST with STRUCT containing TIMESTAMP
SELECT a
FROM UNNEST([STRUCT(CURRENT_TIMESTAMP() as mydate), STRUCT(CURRENT_TIMESTAMP() as mydate)]) as a

-- STRUCT with multiple typed fields including TIMESTAMP
SELECT STRUCT(
  42 as id,
  'Alice' as name,
  TIMESTAMP '2025-03-15 12:00:00 UTC' as created_at,
  true as active
) as person

All three fail with the same failed to convert ... to time.Time type error.

Expected Behavior

The queries should return results with the TIMESTAMP field properly converted, matching real BigQuery behavior.

Actual Behavior

failed to scan rows: failed to convert 1742040000.0 to time.Time type

Root Cause Analysis

We traced this through the codebase and believe the bug is a type mismatch between how TIMESTAMP values are handled at top-level vs inside STRUCTs:

The Flow

1. Storage (internal/contentdata/repository.go → AddTableData()): Top-level TIMESTAMP columns get properly parsed via zetasqlite.TimeFromTimestampValue(), but TIMESTAMP fields nested inside STRUCTs are stored as raw epoch floats (e.g. 1742040000.0) in SQLite — no timestamp conversion is applied to sub-fields.

2. Retrieval (internal/types/types.go → TableCell.AppendValueToARROWBuilder()): When building Arrow output, the float 1742040000.0 is converted to the string "1742040000.0" via fmt.Sprint(v).

3. Conversion (types/arrow.go → AppendValueToARROWBuilder(), TimestampBuilder case): Calls zetasqlite.TimeFromTimestampValue("1742040000.0") which expects a formatted timestamp string like "1970-01-20T11:47:20.000000", not an epoch float. This fails.

Key Files

File	Role
types/arrow.go	Where the error is thrown (TimestampBuilder case)
internal/contentdata/repository.go	Where TIMESTAMP parsing is missing for STRUCT sub-fields
internal/types/types.go	Where float→string conversion happens without type awareness

Possible Fix Directions

Either:

• Apply TimeFromTimestampValue() to TIMESTAMP fields inside STRUCTs during storage (in AddTableData)
• Or handle epoch float strings in the TimestampBuilder case in types/arrow.go (detect numeric format and convert from epoch seconds accordingly)

Environment

• Emulator image: ghcr.io/recidiviz/bigquery-emulator:latest
• Discovered while building JDBC driver integration tests: Two-Bear-Capital/tbc-bq-jdbc#39

[timveil]

@ohaibbq any thoughts?

[ohaibbq]

Thanks for the report! I've got a patch ready locally but am busy the rest of the week. I'll have it up on Monday.