Confusing API exceptions during join

[rbasralian]

Running something like this:

TableService ts = clientSession;
TableHandle root = ts.execute(TableSpec.empty(42).select("I=ii", "X=randomDouble(0, 1)", "Y=randomDouble(0, 1)"));
TableHandle h1 = ts.execute(root.table().where("I % 2 == 0").sort("X"));
TableHandle h2 = ts.execute(root.table().where("I % 2 == 0").sort("Y"));
TableHandle joined = h1.view("I", "X").naturalJoin(h2.view("I", "Y"), "I");

throws:

Exception in thread "main" java.lang.IllegalArgumentException: Can't mix multiple exportStates() with TableHandle
	at io.deephaven.client.impl.TableHandle.adapt(TableHandle.java:194)
	at io.deephaven.client.impl.TableHandle.adapt(TableHandle.java:32)
	at io.deephaven.api.TableOperationsAdapter.naturalJoin(TableOperationsAdapter.java:120)

If you use a stateful table service, then it throws a different exception. Running this:

TableService ts = clientSession.newStatefulTableService();
TableHandle root = ts.execute(TableSpec.empty(42).select("I=ii", "X=randomDouble(0, 1)", "Y=randomDouble(0, 1)"));
TableHandle h1 = ts.execute(root.table().where("I % 2 == 0").sort("X"));
TableHandle h2 = ts.execute(root.table().where("I % 2 == 0").sort("Y"));
TableHandle joined = h1.view("I", "X").naturalJoin(h2.view("I", "Y"), "I");

throws:

Exception in thread "main" java.lang.IllegalArgumentException: Unable to complete request, contains an unreferenceable table: io.deephaven.qst.table.ImmutableWhereTable@7e7bafe6. This is an indication that the query is trying to export a strict sub-DAG of the existing exports; this is problematic because there isn't (currently) a way to construct a query that guarantees the returned export would refer to the same physical table that the existing exports are based on. See https://github.com/deephaven/deephaven-core/issues/4733 for future improvements in this regard.
	at io.deephaven.client.impl.ExportStates.ensureNoUnreferenceableTables(ExportStates.java:238)
	at io.deephaven.client.impl.ExportStates.exportRequestImpl(ExportStates.java:98)
	at io.deephaven.client.impl.ExportStates.exportRequest(ExportStates.java:90)
	at io.deephaven.client.impl.TableServiceImpl.executeImpl(TableServiceImpl.java:79)
	at io.deephaven.client.impl.TableServiceImpl.execute(TableServiceImpl.java:46)
	at io.deephaven.client.impl.TableServiceImpl.execute(TableServiceImpl.java:29)
	at io.deephaven.client.impl.TableHandleManagerBatch.execute(TableHandleManagerBatch.java:47)
	at io.deephaven.client.impl.TableHandleManagerDelegate.execute(TableHandleManagerDelegate.java:32)

Using 0.33.3 for both client and server.

Note that the following example works using either TableService:

TableHandle root = ts.execute(TableSpec.empty(42).select("I=ii", "X=randomDouble(0, 1)", "Y=randomDouble(0, 1)"));
TableHandle h1 = root.where("I % 2 == 0").sort("X");
TableHandle h2 = root.where("I % 2 == 0").sort("Y");
TableHandle joined = h1.view("I", "X1=X", "Y1=Y").naturalJoin(h2.view("I", "X2=X", "Y2=Y"), "I");

https://illumon.slack.com/archives/G01K6UHJ00K/p1713888221045079?thread_ts=1713880314.724629&cid=G01K6UHJ00K

[devinrsmith]

This ticket should mainly serve as ticket for better documentation and better error messages.

As a broad API, even though the types are compatible, there are limitations on what kinds of TableHandles can be crossed. For example, in the following case where there are TableHandles from different clients, we don't expect to be able to join them:

TableHandle t1 = client1.execute(...);
TableHandle t2 = client2.execute(...);
TableHandle t3 = t1.naturalJoin(t2, ...); // this will throw error

A similar concept is true intra-client based on how the client manages the statefulness of exports:

TableHandle t1 = client.newStatefulTableService().execute(...);
TableHandle t2 = client.newStatefulTableService().execute(...);
TableHandle t3 = t1.naturalJoin(t2, ...); // this will throw error

In fact, the above example is exactly the same as

TableHandle t1 = client.execute(...);
TableHandle t2 = client.execute(...);
TableHandle t3 = t1.naturalJoin(t2, ...); // this will throw error

io.deephaven.client.impl.Session will create a newStatefulTableService for each execute implicitly. TableHandles can't cross newStatefulTableService boundaries; this is in service of trying to be maximally efficient in terms of not re-doing work within a stateful set. I do think the motivation for this deserves further discussion and documentation.