ChangeLog for ARQ ================= ARQ 1.5.1 --------- + Fixed the choice of basic pattern matcher to not block fastpath on databases ARQ 1.5 ------- + Property function for lists: added list:length, list:index. All property function for lists iterate over all lists if the subject is not bound. + Sort order of arbirary terms changed: value-based ORDER BY remains the same. If two terms are value-same or not comparable, ORDER BY by sorts by lexical form, then by language tag or datatype URI. + Faster in-memory query processing. Noticable for queries where a subblock is executed for many different possible solutions. + Intenrally, variables are instances of class "Var". Strings, for variable names, are not passed around internally. The external API does not change. + No more ElementTriplePattern. This was not used directly by the parser and has become redundant. Add Triples (with Var or Nodes) directly to ElementBasicGraphPattern Similarly PlanElementTriplePattern has gone (butthis wasn;t used) + Use of class Var for SPARQL variables (extend Jena's graph Node so it can go in triples). When building queries programmtically, use in preference to Node.createVariable() + Jar changes: Added lucene-core-2.0.0.jar + Added free text search via Lucene (2.0.0) See doc/lucene-arq.html + Engine2 - an implementation of the SPARQL algebra directly Functionally correct. + Basic graph patterns no longer have filter - these are part of the group. Only affects applications programmatically building queries - all existing code should work though. + Bug fix: ORDER BY lost duplicates; it shouldn't. + Minor syntax changes to align with the working group + No bnode for GRAPH variable: GRAPH [] { ... } not legal + \u escape now allowed anywehere in the query + Filters can't syntactcically have bnodes: no FILTER(_:x < 5 ) Variable values may still be blank nodes. + LIMIT/OFFSET in either order Now prints as OFFSET/LIMIT which is slightly more logical. + Support for an alternative regular expression engine (Xerces XPath implementation). The default java.util.regex supports more than the language of XPath/XQuery Functions and Operatiors. Stricter adherence to the regular expression language for SPARQL can be achieved by using the Xerces engine: gloabbly: ARQ.getContext().set(ARQ.regexImpl, ARQ.xercesRegex) ; (or set in a query execution context on a per query basis) The Xerces engine may become the default later. Neither engine supports the "x" flag (removal of whitespace in regexs) ARQ 1.4 ------- + Split jar file into arq.jar and arq-extra.jar arq.jar is the com.hp.hpl.jena.query package without the tests arq-extra.jar includes commands, tests, test support code and examples + Upgraded jars: wstx-asl-3.0.0 , commons-logging-1.1 + Property functions No longer a way to call Extensions but now a first-class concept. Automatic adapter for old code (writes log messages) New package specifically for property functions with interface and helper classes com.hp.hpl.jena.query.pfunction Library com.hp.hpl.jena.query.pfunction.library list@memeber, rdfs:member and examples rewritten to new interfaces + Examples in src-examples/ directory + Added support for xsd:date in FILTERS (<, <=, >=, >, =, !=, ORDER BY) + API: deprectate Query.getQueryElement in favour of getQueryPattern() ditto setter + Removed use of ElementBlock (and PlanBlock) as root of the query pattern + Internal: reorganisation of expr package (should be transparent except for users of the ExprVisitor) + Internal: removed "Constraint" and use Expr directly (which is the parser generated type) + Internal: reorganisation parsing to make the internal abstract parse tree as close to SPARQL, moving some of the transformations out of the parse tree into the query engine ElementBasicGraphPattern added to directly reflect the SPARQL query structure PlanBasicGraphPattern to Renamed PlanBasicPattern as PlanBlockTriples + Internal: Transformation framework to make adding new query matchers (e.g. wrappers for non-RDF data). See src-examples/arq/examples/extMyQueryEngine.java + CONSTRUCT patterns now see any initial bindings. + Added execConstruct(Model) and execDescribe(Model) which places the RDF statements into the model supplied. + ARQ.enableBlankNodeResultLabels() to switch on/off constant labels in XML and JSON result formats. The default is off (the default for reading <_:aaa> in queries is on though) + Add standard library bindings for some F&O functions: now has: fn:boolean, fn:not, fn:not, fn:abs, fn:celing, fn:floor, fn:contains, fn:startsWith, fn:endsWith (2 argument forms only) fn:substring, fn:string-length, fn:string-join fn:matches ARQ 1.3 ------- + Added JSON input and output for result sets. http://www.w3.org/TR/sparql-results-json/ (draft: http://www.w3.org/2001/sw/DataAccess/json-sparql/) + BNodes are scoped to the basic graph pattern, not the whole query, in line with the SPARQL spec. Will only matter for bNodes in the named form (e.g. _:a0) used in two basic graph patterns in query. + Properties can't be bNodes (as per SPARQL spec). + Internal BasePattern => BasicPattern to align with language in SPARQL spec + Added argument testing to QueryExecutionFactory. + Changed SPARQL/ARQ testing to work on plain graphs (no sameValueAs) Does not affect normal execution which uses Jena's d-entailment (sameValueAs) + The order of triples generated when the triple object itself generated triples was such that the outer triple came last. That is: { :x :p [ :q :v ] } generated { _:a :q :v . :x :p _:a . } This gives all the right answers for graph matching but with property functions, it meant that :x :p [ list:member ?member ] invoked the list code with the list unbound. Now { :x :p [ :q :v ] } generates { :x :p _:a . _:a :q :v . } which works with property functions. + Printed queries can now print doubles as 123e0, not in long, typed form. + Added/completed formatting support for ASK results (XML, text, N3) ARQ 1.2 ------- + Added options inputGraphBNodeLabels outputGraphBNodeLabels to force told bNode handling in XML result sets across the net. + Conflicts between Java 1.4 and 1.5 (BigDecimals, regex flags) should be resolved. + QueryExecutionFactory operations for remote execution rename from 'create' to 'sparqlService' + Add QueryExecutionFactory that take a query string inplace of a Query object. + Switched to using Java regular expression code (from ORO) Can switch back to ORO - see Javadoc for ARQ class + Jar change: now uses Woodstox StAX parser wstx-asl-2.8.jar, not stax-1.1-dev.jar (clearer statement of license) + Function properties enabled Supported list:member and rdfs:member Disable with: "ARQ.setFalse(ARQ.enablePropertyFunctions);" or "qExec.getContext().setFalse(ARQ.enablePropertyFunctions);" + Switch between reading XML result sets with StAX (full streamed end-to-end) and SAX via "ARQ.getContext().setTrue(ARQ.useSAX) ;" Default is StAX. ARQ 1.1 ------- SPARQL change: + xsd:decimal used for a query term like 1.3 (no quotes), rather than xsd:double. + EffectiveBooleanValue rules changed to be consistent with XPath/XQuery EBV only applies to the value of the FILTER and the operators && || and ! + Added QuerySolutionMap, an implementation of QuerySolution that can be used to pass in initial values when creating a query execution (as can a solution from a previous query) + Internally, in filters, integers are BigIntegers to cope with arbitrary precision. + Add command "arq.qexpr" to evaluate expressions + Change to Function.exec (but this is masked by the common implementation classes) + Add support for XSD decimal. + Restructured/rewrote basic value handling Corner cases for value testing corrected. ARQ 1.0 ------- (Released as part of the Jena 2.3 download) + Add sha1sum(?x) - a function to calculate the SHA1 checksum of a URI or lexical form + Tuned pretty printing of triples + Describe handlers now have a start/describe/finish cycle Internal changes + Interface changes for custom functions and extensions Makes the current dataset and active graph available if needed. Full access to the execution context + ExecutioContext now caries the active graph and Dataset[Graph] is a static structure DatsetGraph removed from QueryIter constructors + Removed "Mark" - the QueryIterator logging helper Can use EXT cache() and EXT trace() to debug iterators. + Parameters per query engine implementation, including a global set of defaults. Functions and extensions have access to the execution context (rarely needed). + Registries (Describe handlers, functions, extensions) placed into the global context. + Queries now fixup the result variables (e.g. SELECT *) as part of QueryExecution. + Package for "describe" to move it out of main API. + Lock classes moved to com.hp.hpl.jena.shared ARQ 0.9.7 --------- SPARQL related changes: + Query Results format namespace change to http://www.w3.org/2005/sparql-results# + Variables can start with a "_" now. API changes: + Reorganised ResultSet output Consistent naming (including ResultSetFormatter) Orthogonal design Other: + Default for command arq.rset changed to output text verison, not XML. use --out=xml for XNL output. + build() methods for functions and extensions base implementation classes provide an implementation + Pretty printing triples in queries - now uses ";" + Query.equals is a structural equality. Query.hashCode is structiural. + added to function library min(expr1, expr2), max(expr1, expr2) round(expr), abs(expr), sqrt(expr), floor(expr), ceiling(expr) e(), pi() + Functions (and extensions) loaded via factories Usual implementations have convenience "auto-factory" including when using "java:" Internal: + The package "...query.parser" is now "...query.lang" ARQ 0.9.6 --------- SPARQL related changes: + QNames can no longer end in "." (DOT) + Syntax change: FILTER expressions need () unless a function call or builtin (e.g. regex) + Syntax change: ORDER BY ASC[?x] becomes ORDER BY ASC(?x) Added implementation of ORDER BY expression + BNodes as predicates added. + Multiple FROM clauses cause RDF merge of graphs for default graph + Added "--service" for remote query execution + Added remote query engine + Function library: :now() function + Added "java:" URI scheme for function extensions (SPARQL) + Changed from using string to constant objects for syntax names and data formats Constants changed - Syntax.* constants made compatible. + Add extensible graph matcher loading via java: (ARQ only) + StAX parsing of XML results - end-to-end-streaming of XML results now done. + "DataSet" renamed "Dataset" + Added the graph matching extensions (ARQ only) via "EXT" (experimental) + Changed to newer query results format for XML output + Dataset.getGraph() and .getModel() changed to .getDefaultGraph() and getDefaultModel() Deprecated methods left for compatibilitty for one release. + New query engine architecture - add your own query engine! + Removed QueryExecution.init() ARQ 0.9.5 --------- + Added ORDER BY, and OFFSET (LIMIT was already there) Sorting is in-memory, limiting the size of results that can be handled. + Methods deprecated in 0.9.94 have been removed. + GRAPH * removed - use GRAPH [] instead. + Expression visitors as part of generalized serialization process + Syntax for ORDER BY now uses ASC[] and DESC[] for direction + Added back FROM and FROM NAMED as per SPARQL working draft + Wrote some .bat scripts + Closed some iterators that might not close ARQ 0.9.4 --------- QL SYNTAX CHANGES 1/ DAWG has resolved to adopt a Turtle-like syntax. To convert queries, use "qparse --in SPARQL-alt ..." (does not get regexps and some other constraints right) 2/ regular expressions are regexp(?x, "foo.*bar") 3/ Expressions use "=", "!=" "<", ">" on anything 4/ AND => FILTER 5/ Reification sugar using << ... >> 6/ No WITH or FROM ARQ has additional features, principally: A1/ FROM and FROM NAMED (was WITH and FROM) A2/ UNSAID + API change: QueryFactory to make queries (from string or URL) separated out from Query class. Old Query operations are deprecated + API change: Put syntax URI constants in own class. + API change: Query.unparse => Query.serialize + SPARQL change: removed [] for optionals in sparql.jj and arq.jj + Grammar/syntax changes to expressions "==", "eq", "ne" are no more. Use "=", "!=" on anything ; use "<", ">" etc etc. + =~ becomes regex(,,) + Printing: output query now UTF-8 safe + Printing: retain and output "*" for SELECT, CONSTRUCT, DESCRIBE + Preservation of lexical forms of integers and doubles + Converted RDQL test suite to SPARQL, fixing those tests with different semantics in value testing + Removed "BRQL" the hybrid RDQL-SPARQL language, reverting to plain RDQL. It could not express many useful queries so it was neither one thing nor the other. + Implemented XSD DateTime (via java.util.GregorianCalendar) + Implemented casting + Refactoring ARQ 0.9.3 --------- Fully implements SPARQL except for some cases of expression evaluation + In line with editor's draft of the DAWG spec after Finland face-to-face meeting including syntax changes (NB Change in meaning to FROM) (Exact details of evaluation of constrainat awaiting editor's decisions) + Dataset implemented + Extension functions implemented + Upgrade XML result format to working draft + Refining application API + Test suite growing + Start examples + Move attaching DataSets to QueryFactory + Rename QueryFactory as QueryExecutionFactory rename createQueryExecution(0 static methods as create() + Add Query.read operations to read a query from a file. + QueryExecution objects are the unit to close, not ResultSets or formatters for result sets + Recoded expressions in a simpler fashion. ARQ 0.9.2 --------- + Added variables node to XML result set format. + Prefix mapping applied to text-formatted results Currently, its the prefix mappings of the query only. For CONSTRUCT/DESCRIBE it includes the prefix mappings of the target + Rewrite of command "query" - now has consistent syntax + Numerous tweaks. ARQ 0.9.1 --------- + Lots of internal refactoring + Bug fixes + Command line utility tidy up. arq, sparql, query, qparse, qtest. (not finsihed) ARQ 0.9.0 --------- + Name change! + Command line app package now called "arq." + DISTINCT + LIMIT + Use visitor pattern to do unparsing - allows more output forms BRQL 0.9 -------- + Split syntaxes supported: ARQ (native to the query engine) DAWG (the working group syntax) RDQL (the BRQL-RDQL syntax) + ASK done (grammar and command line apop finished) + Preliminary implementation of UNSAID - semantics not clear, awaiting WG Currently implmented (the direct as-written way) query order changes the outcome. + SOURCE implemented + Added DataSource to abstract away from what the query is over. BRQL 0.8 -------- + Disjunction implemented + CONSTRUCT implemented + FROM implemented (list of URIs) + Two parsers : RDQL-like syntax and BRQL-{} syntax Command line app takes --rdql and --brql flags. Default is BRQL. + Tests have a "syntax" property to name the syntax used. + Printing: only prints in BRQL-{} syntax. ~ FileManager moved to Jena + Separate constraints package for programmatic creation of expressions + General evaluation based on Bindings. + Evaluate expressions from bindings. (Need to reintegrate back into Jena graph query sometime this code may be the best way to do it). + BindingIndex: Convert bewteen Bindings and IndexValues/VariableIndexes retain all the old eval(IndexValues) in parser nodes for now. + Do own evaluation of constriants - ConstriantFilter. Query engine does not push constriants down the jena.graph. The new generalisations in BRQL make this the wrong structure. BRQL 0.7 -------- + Added alternative parser (only - not connected to query engine) for syntax experiments. Command line wrapper: brql.Parse + Refactoring to create a public API and implemntation packages (Not complete in this release) + Added FileManager for access to system resources in the filing system or classpath Allows test data files to be located any where and indirect URLs to local copies This will migrate into the Jena codebase + Added model cache to cahe read models BRQL 0.6 -------- + Added getValue back (deprecated) for backwards compatibility + XML examples include use of XQuery. + XML result format only includes the selected variables. + Can be run on .Net using IKVM. + Added result set walker. BRQL 0.5 -------- + Changed DESCRIBE to return a single (merged) model + Fixed parser to accept all DESCRIBE queries, inc. no WHERE clause. + Fixed query printer for the above change. + Started DAWG test cases: added to TestSuite. BRQL 0.4 -------- + Better formatting of raw result sets in text form. Instead of just .toString(), it now prints as teh conventional formats for qnames, bNodes, URIs or literals. BRQL 0.3 -------- + Added XML output in a simple XML format for further transformation. See also xml/ for a stylesheet to turn results into an HTML table. Removed hardcoded HTML generation from Java code. + Added output of results in a nested structure based on the internal binding datastructure. This is like nested subqueries: it is sensitive to the precise multiple optionals interpretation which is currently result set widening. Output option rs-raw