Zebra Reference Guide

Zebra supports simple types (int, long, float, double, string, bytes), complex types (record, collection, map) and Booleans.

Zebra type names are chosen to be as “technology neutral” as possible and are in resemblance to native types in modern programming languages.

Use the Zebra store schema to write or store Zebra columns and to specify column types. The schema supports data type compatibility and conversion between Zebra/Pig, Zebra/MapReduce, and Zebra/Streaming. (In a future release, the schema will also support type compatibility between Zebra/Pig-SQL and will guide the underlying serialization formats provided by Avro for projection, filtering, and so on. )

The basic format for the store schema is shown here. The type name is optional; if not specified, the column defaults to type bytes.

column_name[:type_name] [, column_name[:type_name] ... ]

Schemas for Simple Data Types

Simple data types include int, long, float, double, string, and bytes. The following syntax also applies to Booleans.

Syntax

field_alias[:type] [, field_alias[:type] …]

Terms

Examples

In this example the schema specifies names and types for 3 columns.

ZebraSchema.createZebraSchema(JobContext, “s1:string, f1:float, i1:int”);

In this example the schema specifies names for 3 columns; all 3 columns default to type bytes.

ZebraSchema.createZebraSchema(JobContext, “f1, f2, f3”);

Schemas for Records

A record is an ordered set of fields. A field can be any Zebra type.

Syntax

record_alias:record (field_alias[:type]) [,(field_alias[:type]) …] )

Terms

Examples

In this example the schema specifies a record with two fields.

ZebraSchema.createZebraSchema(JobContext, “r1:record(f1:int,f2:long)”);

In this example the schema specifies a record with two fields. Note that f2 will default to type bytes.

ZebraSchema.createZebraSchema(JobContext, “r1:record(r2:record(f1:int,f2)”);

Schemas for Collections

A collection is a set of records.

Syntax

collection_alias:collection ([record_alias:]record(...))

Terms

Examples

In this example the schema specifies a collection of records, each consisting of two fields.

ZebraSchema.createZebraSchema(jobContext, “c1:collection(r1:record(f1:int,f2:long))”);

Schemas for Maps

A map is a set of key/value pairs.

Syntax

map_alias:map (type)

Terms

Examples

In this example the schema specifies a map with value of type string.

ZebraSchema.createZebraSchema(jJobContext, “m1:map(string)”);

In this example the schema specifies a map with value of type map (with a value of type int).

ZebraSchema.createZebraSchema(JobContext, “m2:map(map(int))”);

Use the Zebra storage specification to define Zebra column groups. The storage specification, when combined with a STORE statement, describes the physical structure of a Zebra table. Suppose we have the following statement:

STORE A INTO '$PATH/mytable' USING org.apache.hadoop.zebra.pig.TableStorer('[a1, a2] AS cg1; [a3, a4, a5] AS cg2');

The statement describes a table that has two column groups; the first column group has two columns, the second column group has three columns. The statement can be interpreted as follows:

• $PATH/mytable - the table, a file path to a directory named mytable
• $PATH/mytable/cg1 - the first column group, a subdirectory named cg1 under directory mytable
• $PATH/mytable/cg1/part00001 - a file consisting, conceptually, of columns a1 and a2
• $PATH/mytable/cg2 - the second column group, a subdirectory named cg2 under directory mytable
• $PATH/mytable/cg2/part00001 - a file consisting, conceptually, of columns a3, a4, and a5

Specification

The basic format for the Zebra storage specification is shown here. For this specification, note that the straight brackets [ ] designate a column group and the curly brackets { } designate an optional syntax component.

Syntax

[column_name {, column_name ...} ] {AS column_group_name} {COMPRESS BY compressor_name} {SERIALIZE BY serializer_name}
{; [column_name {, column_name ...} ] {AS column_group_name} {COMPRESS BY compressor_name} {SERIALIZE BY serializer_name} ... }

Terms

Examples

In this example, one column group is specified; the two statements are equivalent.

STORE A INTO '$PATH' USING org.apache.hadoop.zebra.pig.TableStorer('[c1]');

STORE A INTO '$PATH' USING org.apache.hadoop.zebra.pig.TableStorer('[c1] AS CG0 COMPRESS BY gz SERIALIZE BY pig;');

In this example, two column groups are specified. The first column group, C1, has two columns. The second column group, C2, has three columns.

STORE A INTO '$PATH' USING org.apache.hadoop.zebra.pig.TableStorer('[a1, a2] AS C1; [a3, a4, a5] AS C2');

Use the Zebra load schema to load or read table columns.

Schema

The basic format for the Zebra load (read) schema is shown here. The column name can be any valid Zebra type. If no columns are specified, the entire Zebra table is loaded.

column_name [, column_name ... ]

Terms

Example

Three Pig examples are shown here.

-- All columns are loaded
A = LOAD '$PATH/tbl1' USING org.apache.hadoop.pig.zebra.pig.TableLoader();
   
-- Two columns are projected
B = LOAD '$PATH/tbl2' USING org.apache.hadoop.zebra.pig.TableLoader('c1,c2');
   
-- Three columns are projected: a simple field, a map, and a record
C = LOAD '$PATH/tbl3' USING org.apache.hadoop.zebra.pig.TableLoader('c1,c2#{key1},col4.{f1}')