Title: Stanbol Enhancer RESTful Services The RESTful service endpoint provided by the Stanbol Enhancer is a stateless interface that allows the caller to submit content and get the resulting enhancements formatted as RDF at once without storing anything on the server-side. More advanced options also allow to parse pre-existing metadata, parse and request alternate content versions and additional metadata created by the Enhancer or specific Enhancement Engines. The RESTful interface described below is provided on several endpoints * __'/enhancer':__ The main endpoint of the Stanbol Enhancer. Parsed content will get enhanced by using the default enhancement chain. * __'/enhancer/chain/{chain-name}'__: The Stanbol Enhancer supports the configuration of multiple [Enhancement Chains](chains). Users can lookup active chains by requests to the 'enhancer/chain' endpoint. * __'/enhancer/engine/{engine-name}'__: This can be used to enhance parsed Content with a single [Enhancement Engine](engines). Note that the parsed Content MUST be processable by the referenced engine. So if the engine is not able to directly process the parsed content you might need to send existing metadata such as explained in the section [Parsing multiple ContentParts](#parsing_multiple_contentparts). This feature is e.g. useful to directly send a MP3 file to the [TikaEnigne](engines/tikaengine.html) to extract the metadata. * __'/engines':__ Same as '/enhancer' this ensures backward compatibility to older Stanbol versions. ## Basic Enhancement Service This sections describes how to parse content to the Stanbol Enhancer which then gets analyzed. Results are sent back in the form of a serialized RDF graph. The content to analyze should be sent in a POST request with the mime-type specified in the Content-type header. The response will hold the RDF enhancement serialized in the format specified in the Accept header: :::bash curl -X POST -H "Accept: text/turtle" -H "Content-type: text/plain" \ --data "The Stanbol enhancer can detect famous cities such as Paris \ and people such as Bob Marley." \ http://localhost:8080/enhancer The list of mime-types accepted as inputs depends on the deployed engines. By default most Enhancement Engines can only process plain text content. However EnhancementEngines like [Metaxa](engines/metaxaengine.html) can be used to create 'text/plain' versions of parsed content. This allows also to enhance contents with mime-types such as html, pdf and MS office documents (see the Metaxa documentation for details) Stanbol Enhancer is able to serialize the response in the following RDF formats: :::text application/json (JSON-LD) application/rdf+xml (RDF/XML) application/rdf+json (RDF/JSON) text/turtle (Turtle) text/rdf+nt (N-TRIPLES) ### Additional Parameters * __uri={content-item-uri}:__ By default the URI of the content item being enhanced is a local, non de-referencable URI automatically built out of a hash digest of the binary content. Sometimes it might be helpful to provide the URI of the [ContentItem](contentitem.html) to be used in the enhancements RDF graph. * __executionmetadata=true/false:__ Allows the include of [execution metadata](executionmetadata.html) in the enhancement metadata of the response. Such data include also the [execution plan](chains/executionplan.html) used to enhance the parsed content. This information is typically only useful to clients that want to know how the parsed content was processed by the enhancer. NOTE that the execution metadata can also be requested by using the multi-part content item API described below. The following example shows how to send an enhancement request with a custom content item URI that will include the execution metadata in the response. In addition this request is directed to a [Enhancement Chain](chains) with the name "dbpedia-keyword" :::bash curl -X POST -H "Accept: text/turtle" -H "Content-type: text/plain" \ --data "The Stanbol enhancer can detect famous cities such as Paris \ and people such as Bob Marley." \ "http://localhost:8080/enhancer/chain/dbpedia-keyword?uri=urn:fise-example-content-item&executionmetadata=true" ## Enhancer Configuration The Stanbol Enhancer supports several RESTful services to inspect the configuration. This services allow to retrieve currently active [Enhancement Engines](engines) and [Enhancement Chains](chains). * __'/enhancer':__ GET requests to the main Stanbol Enhancer endpoint the do used an 'Accept' header compatible to one of the supported RDF serializations will return the current configuration as RDF. * __'/enhancer/engine':__ Same as above however the response will only include active enhancement engines * __'/enhancer/chain':__ Returns the currently active enhancement chains. * __'/enhancer/sparql':__ SPARQL endpoint that allows to query the configuration. Example Response as 'application/rdf' serialization of the default configuration of the Stanbol Enhancer. The request :::bash curl -v -X GET -H "Accept: application/rdf+xml" "http://localhost:8080/enhancer/ep" returns the following results :::xml langid language ner [...] ### Executionplan of Enhancement Chains The [ExecutionPlan](chains/executionplan.html) can be also requested by sending a GET request with an supported RDF serialization as 'Accept' header to * __'/enhancer/ep'__ * __'/enhancer/chain/{chain-name}/ep'__ * __'/engines/ep'__ ## Multi-part ContentItem support The multipart ContentItem extensions to the basic RESTful services are provided by the Stanbol Enhancer. It was introduced (by [STANBOL-481](https://issues.apache.org/jira/browse/STANBOL-481)) to allow advanced usage scenarios. Users will want to use this extensions if they need to: * parse multiple versions of the content: Most CMS already do have support for converting content to plain text. This API allows to parse both the original AND multiple transcoded versions of the content to the Enhancer. * parse pre-existing metadata: Typically CMS do have already some metadata about content parsed to the Stanbol Enhancer (e.g. User provided Tags, Categories …). The multi-part extensions do allow to parse such data in addition to the content. * request transcoded versions of the parsed content: This API extensions allows to include transcoded (e.g. the 'plain/text') version of parsed content in the response. It also allows requests that directly returns transcoded content by omitting extracted metadata. * request additional metadata that are normally not included within the metadata of the Enhancement response: This can to request the [execution metadata](executionmetadata.html) in an own RDF graph, but it can also be used to request metadata of specific enhancement engines (TODO: add example) ### QueryParameters The following QueryParameters are defined by the multi-part content item extension: * __outputContent=[mediaType]:__ Allows to specify the Mime-types of content included within the Response of the Stanbol Enhancer. This parameter supports wild cards (e.g. '*' ... all, 'text/*'' ... all text versions, 'text/plain' ... only the plain text version). This parameter can be used multiple times. Responses to requests with this parameter will be encoded as multipart/form-data. If the "Accept" header of the request is not compatible to multipart/form-data it is assumed as a 400 BAD_REQUEST. For details see the documentation of the [Multipart MIME format for ContentItems](contentitem.html#multipart_mime_serialization). * __omitParsed=[true/false]:__ Makes only sense in combination with the outputContent parameter. This allows to exclude all content included in the request from the response. A typical combination is outputContentPart=*/*&omitParsed=true. The default value of this parameter is false. * __outputContentPart=[uri/'*']:__ This parameter allows to explicitly include content parts with a specific URI in the response. Currently this only supports [ContentParts](contentitem.html#content_parts) that are stored as RDF graphs. Responses to requests with this parameter will be encoded as multipart/form-data. If the "Accept" header of the request is not compatible to multipart/form-data it is assumed as a 400 BAD_REQUEST. The selected content parts will be included as MIME parts in the returned [Multipart MIME formated ContentItems](contentitem.html#multipart_mime_serialization). The URI of the part will be used as name. Such parts will be added after the "metadata" and the "content" (if present). * __omitMetadata=[true/false]:__ This allows to enable/disable the inclusion of the metadata in the response. The default is false. Typically omitMetadata=true is used when users want to use the Stanbol Enhancer just to get one or more ContentParts as an response. Note that Requests that use an Accept: {mimeType} header AND omitMetadata=true will directly return the content version of {mimeType} and NOT wrap the result as multipart/form-data. See also the example further down this documentation. * __rdfFormat=[rdfMimeType]:__ This allows for requests that result in multipart/form-data encoded responses to specify the used RDF serialization format. Supported formats and defaults are the same as for normal Enhancer Requests. ### Parsing multiple ContentParts Requests to the Stanbol Enhancer with the Content-Type: multipart/form-data are considered to contain a ContentItem serialized as MultiPart MIME. The exact specification of the [MultiPart MIME format for ContentItems](contentitem.html#multipart_mime_serialization) is provided by the documentation of the ContentItem. The combination of multipart/form-data encoded requests with QueryParameters as described above allow for the usage of [MultiPart MIME format for ContentItems](contentitem.html#multipart_mime_serialization) for both request and response. ## Using the multi-part content item RESTful API extensions The following examples show typical usage scenarios of the multi-part content item RESTful API. Note that for better readability the values of the query parameters are not URL-encoded. ### Example 1: Return metadata and content The first example shows how users can request both the metadata and transcoded versions of the parsed content. This can be achieved relatively easy by using the "outputContent=*/*" in combination with "omitParsed=true". :::bash curl -v -X POST -H "Accept: multipart/form-data" \ -H "Content-type: text/html; charset=UTF-8" \ --data "

The Stanbol enhancer can detect famous cities \ such as Paris and people such as Bob Marley.

" \ "${it.serviceUrl}?outputContent=*/*&omitParsed=true&rdfFormat=application/rdf+xml" This will result in a response with the mime-type "Content-Type: multipart/form-data; charset=UTF-8; boundary=contentItem" and the metadata as well as the plain text version of the parsed HTML document as content. :::text --contentItem Content-Disposition: form-data; name="metadata"; filename="urn:content-item-sha1-76e44d4b51c626bbed38ce88370be88702de9341" Content-Type: application/rdf+xml; charset=UTF-8; Content-Transfer-Encoding: 8bit --contentItem Content-Disposition: form-data; name="content" Content-Type: multipart/alternate; boundary=contentParts; charset=UTF-8 Content-Transfer-Encoding: 8bit --contentParts Content-Disposition: form-data; name="urn:metaxa:plain-text:2daba9dc-21f6-7ea1-70dd-a2b0d5c6cd08" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Stanbol enhancer can detect famous cities such as Paris and people such as Bob Marley. --contentParts-- --contentItem-- Se also the formal specification of the [MultiPart MIME format for ContentItems](contentitem.html#multipart_mime_serialization) for ContentItems. ### Example 2: Directly return the plain text version of parsed content The using the 'omitMetadata=true' together with the "Accept: {requested-content-type}" the multi-part content API allows to directly request the transcoded version of the content with the format {requested-content-type}. curl -v -X POST -H "Accept: text/plain" \ -H "Content-type: text/html; charset=UTF-8" \ --data "

The Stanbol enhancer can detect famous cities \ such as Paris and people such as Bob Marley.

" \ "${it.serviceUrl}?omitMetadata=true" The response will use Content-Type: text/plain and contain the string :::text The Stanbol enhancer can detect famous cities such as Paris and people such as Bob Marley. To make this work the requested [Enhancement Chain](chains) will need to include an engine (e.g. [Metaxa](engines/metaxaengine.html)) that supports transcoding the parsed content. If no content with the request type is available the request will answer with a "404 NOT FOUND". Note also that because the metadata are omitted by responses to such requests it is also recommended to configure/use a chain that does no further processing on the transcoded content. ### Example 3: Parse multiple content versions This example will use the "httpmime" part of the Apache commons httpcomponents to create the Multipart MIME sent to the Stanbol enhancer. :::xml org.apache.httpcomponents httpmime 4.1.2 The created Multipart MIME content MUST follow the specifications as defined by the [MultiPart MIME format for ContentItems](contentitem.html#multipart_mime_serialization). :::java InputStream wordIn; //The MS Word version of the Content InputStream plainIn; //The plain text version of the Content HttpClient httpClient; //The client used to execute the request //create the multipart/form-data container for the ContentItem //MultipartEntity also implements HttpEntity MultipartEntity contentItem = new MultipartEntity(null, null ,UTF8); //The multipart/alternate container for the contents HttpMultipart content = new HttpMultipart("alternate", UTF8 ,"contentParts"); //now add the container for the content to the content item container contentItem.addPart( "content", //the name MUST BE "content"! new MultipartContentBody(content)); //now add the MS word content at the first location //this will make it the "original" content content.addBodyPart(new FormBodyPart( "http://www.example.com/example.docx", //the id of the content part new InputStreamBody( wordIn, "application/vnd.openxmlformats-officedocument.wordprocessingml.document", "example.docx"))); //now add the alternate plain text version content.addBodyPart(new FormBodyPart( "http://www.example.com/example.docx", //the id of the content part new StringBody( //use a StringBody to avoid binary encoding for text IOUtils.toString(plainIn), //apache commons IO utility "text/plain", Charset.forName("UTF-8")))); //now we are ready to create and execute the POST request to the //Stanbol Enhancer HttpPost request = new HttpPost("http://localhost:8080/enhancer"); request.setEntity(contentItem); request.setHeader("Accept","application/rdf+xml"); Response response = httpClient.execute(request); Note that for such requests [Metaxa](engines/metaxaengine.html) will still try to extract metadata of the parsed MS Word document, but all other engines will use the plain text version as parsed by the request for processing. ### Example 4: Parse existing free text annotations This example shows how the multi-part content item API can be used to parse already existing tags for an parsed content to the Stanbol Enhancer. For this example it is important to understand that parsed metadata need to confirm to the Stanbol Enhancement Structure. Because of that this example consist of two main steps: 1. Convert user tags to TextAnnotations 2. Send existing Metadata along with the Content to the Stanbol Enhancer Also note that the code snippets will use utilities provided by the "org.apache.stannbol.enhancer.servicesapi" module. As RDF framework Apache Clerezza is used. Both dependencies are easily replaceable. First lets have a look at the required information :::java MGraph graph; //the RDF graph to store the metadata UriRef ciUri; //the URI for the contentItem String tag; // user provided tag UriRef tagType; //the type of the Tag Regarding the tag type: Stanbol natively supports the following types * __Person__ (http://dbpedia.org/ontology/Person) * __Organization__ (http://dbpedia.org/ontology/Organisation): NOTE the British spelling * __Place__ (http://dbpedia.org/ontology/Place) The processing of parsed tags that use other or no type depends on the used [enhancement engines](engines) and their configurations. Especially the configuration of the [Named Entity Tagging Engine](engines/namedentitytaggingengine.html) is important in that respect. :::java Resource user; //the user that has created the tag (optional) //in case of an name just use a literal user = new PlainLiteral("Rudolf Huber"); //in case users have assigned URIs user = new UriRef("http://my.cms.org/users/rudof.huber"); Now we can convert the User Tags to TextAnnotations :::java //first create a URI for the text annotation. Here we use a random URN //If you can create a meaningful URI this would be better! UriRef ta = new UriRef("urn:user-annotation:"+EnhancementEngineHelper.randomUUID()); //The the 'rdf:type's graph.add(new TripleImpl(ta, RDF.type, TechnicalClasses.ENHANCER_TEXTANNOTATION)); graph.add(new TripleImpl(ta, RDF.type, TechnicalClasses.ENHANCER_ENHANCEMENT)); //this TextAnnotation is about the ContentItem graph.add(new TripleImpl(ta, Properties.ENHANCER_EXTRACTED_FROM, ciUri)); //if the Tag uses a type add it if(tagType != null){ graph.add(new TripleImpl(ta, Properties.DC_TYPE, tagType)); } //add the value of the tag graph.add(new TripleImpl(ta, Properties.ENHANCER_SELECTED_TEXT, new PlainLiteralImpl(tag))); //add the user if(user != null){ graph.add(new TripleImpl(ta, Properties.DC_CREATOR,user)); } Now the 'graph' contains a valid TextAnnotation for the given user tag. This should be done for all tags of the current content. In the next step we need to serialize the RDF data. Again we will use here Clerezza as API, but any RDF framework will provide similar functionality :::java ByteArrayOutputStream out = new ByteArrayOutputStream(); //this tells the Serializer to create "application/rdf+xml" serializer.serialize(out, metadata, SupportedFormat.RDF_XML); String rdfContent = new String(out.toByteArray(),UTF8); Now we need to create the MultiPart MIME content item containing the metadata and the content :::java String content; //the content we want to send to the Stanbol Enhancer //the container for the ContentITem MultipartEntity contentItem = new MultipartEntity(null, null ,UTF8); //The Metadata MUST BE the first element contentItem.addPart( "metadata", //the name MUST BE "metadata" new StringBody(rdfContent,SupportedFormat.RDF_XML,UTF8){ @Override public String getFilename() { //The filename MUST BE the return ciUri.getUnicodeString(); //uri of the ContentItem } }); Note that because the StringBody class provided my the "httpmime" framework does not set a filename we need to override this method and return the URI of the content item. This is essential, because we need ensure that the URI of the ContentItem is the same as the URI (variable 'ciUri') as used when creating the TextAnnotations for the user tags. For the following code snippet note that we can directly add the content to the content item container. Only if we would need to sent multiple alternate content versions (as shown in 'Example 3') the usage of an 'multipart/alternate' container is required. :::java //Add the content as second mime part contentItem.addPart( "content", //the name MUST BE "content" new StringBody(content,"text/plain",UTF8)); //now we are ready to create and execute the POST request to the //Stanbol Enhancer HttpPost request = new HttpPost("http://localhost:8080/enhancer"); request.setEntity(contentItem); request.setHeader("Accept", SupportedFormat.RDF_XML); Response response = httpClient.execute(request); The response of the Enhancer will now contain entity suggestions for the free text user tags.