Query results
This document contains details on query result - fetching, formatting and etc.
Lens query service allows users to fetch results of query by specifying the handle-id. It provides rest api to fetch result-metadata and the results.
Here are various options provided for result formatting and fetching:
Persisting results by server
Server can persists the results in configured location, also format the results of the query, with a custom format.
User can provide a custom output formatter which implements InMemoryOutputFormatter or PersistedOutputFormatter
Default implementation of formatting offers writing it into a file, but a new implementation can be provided which persist in a db or whatever.
The file based formatting provides following options:
- Optionally write header
- Format the rows of the results like csv, or separate them with ctrl character characters, which can be described through a Hive SerDe.
- Optionally write the output with a charset encoding.
- Optionally write footer
- Provide an extension to the file name
- Optionally compress the file
- Skip formatting if result size exceeds a configured threshold
- Optionally allow result to be split into multiple files, with configured number of rows in ech file.
Persisting results by driver
The driver can persist the query output in a location. When driver persists the results, the rows of the result can be formatted by the driver.
HiveDriver allows users to specify the format of the result through a configuration. It will persist the result in queryhandle-id directory inside hdfsout directory in the parent result directory.
Since persisting the results by driver only allows formatting of the rows, if user requires further formatting, the results should be persisted by server again.
Formatting summary
Below table depicts all the formatting options provided by query service
| Formatting of the results | Not persisted by driver | Persisted by driver |
| Not persisted by server | No formatting | Row level formatting |
| Persisted by server | All query output formatting | All query output formatting on top of row level formatting done by driver |
See here for all the configuration property names
Fetching the results
If the results are persisted either by server or driver, the user would get the result set as PersistentQueryResult. If results are not persisted, then user would get the result as InMemoryQueryResult.
| ResultSet | Not persisted by driver | Persisted by driver |
| Not persisted by server | InMemoryQueryResult | PersistentQueryResult |
| Persisted by server | PersistentQueryResult | PersistentQueryResult |
If results are persisted by server, user can get the result as a http end point, which he can download or see it in the browser. The http result set can be obtained by the rest endpoint /queryapi/queries/queryhandle/httpresultresult
The results can be fetched as PersistentQueryResult or InMemoryQueryResult from the end point /queryapi/queries/queryhandle/resultresult