TinkerPop 4.x Design Ideas

TinkerPop 4.x should focus on the most successful aspects of TinkerPop 3.x and it should avoid the traps realized in TinkerPop 3.x. These items include:

Originally from the mailing list:

Throughout our documentation we show uses of the “Blueprints API” (i.e. Graph/Vertex/Edge/etc. classes & methods) as well as the use of the Traversal API (i.e. Gremlin).

Enabling users to have two ways of interacting with the graph system has its problems:

In TinkerPop4 the solution might be as follows:

There should be two “Graph APIs.”

The first API is well known, but the second bears further discussion. ReferenceGraph is simply a reference/dummy/proxy to the provider Graph API. ReferenceGraph has the following API:

That is it. What does this entail? Assume the following traversal:

ReferenceGraph is almost like a RemoteGraph (RemoteStrategy) in that it makes a connection (remote or inter-JVM) to the provider Graph API. When g.V(1).out(‘knows’) executes, it is really sending the bytecode to the provider Graph for execution (as specified by the config of ReferenceGraph.open()). Thus, once it hits the provider’s graph, ProviderVertex, ProviderEdge, etc. are the objects being processed. However, what the traversal’s Iterator<Vertex> returns is ReferenceVertex! That is, it never returns ProviderVertex. In this way, regardless if the user is going “over the wire” or within the same JVM or against a different provider’s graph database or from Gremlin-Python/C#/etc., all the vertices are simply ‘reference vertices’ (id + label). This makes it so that users never interact with the graph element objects themselves directly. They can ONLY interact with the graph via traversals! At most they can ReferenceVertex.id() and ReferenceVertex.label(). Thats it, — no mutations, not walking edges, nada! And moreover, since ReferenceXXX has enough information to re-attach to the source graph, they can always do the following to get more information:

This split into two Graph APIs will enables us to make a hard boundary between what the provider (vendor) needs to implement and what the user (developer) gets to access.

On TINKERPOP-1417, it was suggested that we "Create a Gremlin language subset that is easy to implement on any VM". Implementing the Gremlin VM in another language is pretty straightforward. However, its a lot of code.. all these steps implementations. One thing we could do to make it easy for database providers not on the JVM (e.g. ArangoDB and C) is to create "Gremlito" (Gremlin--). This language subset wouldn’t support side-effects, sacks, match, etc. Basically, just simple traversal steps and reducing barrier terminals.

Thus:

Have we yet found the appropriate serialization model? We didn’t have it in 2.x at all. In 3.x we went with a use case based approach that made a lot of sense in the first few releases of 3.x, but the use cases couldn’t have conceived of what was to come with the development of GLVs. GLVs rendered Gryo, the decided "network option" from the use cases, to be pretty useless given that it is of the JVM only and GraphSON has gone through three versions now trying to find the appropriate format to cover the various features we’ve attempted to support. While GraphSON 3.0 seems to have met the mark for supporting our needs, it seems bloated with Java types and doesn’t perform terribly well in some cases.

An ideal serialization format would be:

On TINKERPOP-1909, it was suggested that we are going to use reference (id/label) based object model. And, the direction is move towards more tidy object model contracts going forward. Reference model definitely provides big performance improvements especially with multi-property vertices/edges. One thing that we can consider is to provide a configurable object model. Enabling users to configure the object model (OutputFormat) as server settings (Exposing server setting is being discussed here TINKERPOP-1636). There will three types of output format.

Consider a testing framework based on the Gherkin tests from 3.x and a gremlin-test parent module for all the test framework modules that will potentially be needed. In 3.x, we found situation where having test modules beyond gremlin-test would have been helpful, so a parent module that held all those would probably be smart.

In TINKERPOP-2051 we tried to make vertex property ids local to their vertex. Several approaches were leading nowhere, in most cases we just broke the Gryo compatibility test suite. The basic idea was to remember the parent element (at least its id) and use it in property equality comparisons. This way, the property [name→marko] would only match another [name→marko] property if the parent elements were the same. In the end it was questioned if that’s even the right approach / what users expect. From a user perspective it would probably make more sense if two properties would match if the keys and values match. That said, we should question whether we actually need ids on properties and thus whether properties are actually Elements.