Support for reading large resultsets [DATAJDBC-356]

[spring-projects-issues]

alpet opened DATAJDBC-356 and commented

Enable support for reading large resultsets (milions of rows). Two suggestions:

1. With respect of mybatis there is already Cursor that can be wrapped in a stream
2. With respect of pure Jdbc there is one unmaintained framework implementation jdbc-stream https://github.com/APNIC-net/spring-jdbctemplate-streams that may be usefull.

Without this methods as findAll or findAllByProperty will benefit from having such functionality

---

No further details from DATAJDBC-356

[spring-projects-issues]

Marcos Vinicius da Silva commented

I stared working on a solution for this issue. I did not started mybatis support yet and will start working on tests and better documentation / examples, but would like to discuss the solution API and implementation wise.

For this case, I created some support classes to return an SqlRowSet backed by a connected ResultSet and it's DataSource. I did it this way so I can keep track of all resources and close them when appropriated.
On the API side, I created a new @QueryStream annotation, to specify the ResultSet fetchSize and to not break current uses of the API (client code that don't close the resulting Stream).

I would much appraciate some review on this work in progress, as it is my first Spring contribution.

Commit link: detinho@913e8c5

[spring-projects-issues]

Marcos Vinicius da Silva commented

I opened a pull request so it's easier to track changes and get feedback.
#176

[spring-projects-issues]

Jens Schauder commented

The problem with directly using the a Stream is that it is very likely not to get properly closed, resulting in leaking of resources.

But if you could build a solution based on ClosableIterator I'd be interested in the PR.

[denniseffing]

I opened a new pull request for this issue. See #903.