public class StructType extends DataType implements scala.collection.Seq<StructField>, scala.Product, scala.Serializable
StructType
object can be constructed by
StructType(fields: Seq[StructField])
For a StructType
object, one or multiple StructField
s can be extracted by names.
If multiple StructField
s are extracted, a StructType
object will be returned.
If a provided name does not have a matching field, it will be ignored. For the case
of extracting a single StructField, a null
will be returned.
Example:
import org.apache.spark.sql._
val struct =
StructType(
StructField("a", IntegerType, true) ::
StructField("b", LongType, false) ::
StructField("c", BooleanType, false) :: Nil)
// Extract a single StructField.
val singleField = struct("b")
// singleField: StructField = StructField(b,LongType,false)
// This struct does not have a field called "d". null will be returned.
val nonExisting = struct("d")
// nonExisting: StructField = null
// Extract multiple StructFields. Field names are provided in a set.
// A StructType object will be returned.
val twoFields = struct(Set("b", "c"))
// twoFields: StructType =
// StructType(List(StructField(b,LongType,false), StructField(c,BooleanType,false)))
// Any names without matching fields will be ignored.
// For the case shown below, "d" will be ignored and
// it is treated as struct(Set("b", "c")).
val ignoreNonExisting = struct(Set("b", "c", "d"))
// ignoreNonExisting: StructType =
// StructType(List(StructField(b,LongType,false), StructField(c,BooleanType,false)))
A Row
object is used as a value of the StructType.
Example:
import org.apache.spark.sql._
val innerStruct =
StructType(
StructField("f1", IntegerType, true) ::
StructField("f2", LongType, false) ::
StructField("f3", BooleanType, false) :: Nil)
val struct = StructType(
StructField("a", innerStruct, true) :: Nil)
// Create a Row with the schema defined by struct
val row = Row(Row(1, 2, true))
// row: Row = {@link 1,2,true}
scala.PartialFunction.AndThen<A,B,C>, scala.PartialFunction.Lifted<A,B>, scala.PartialFunction.OrElse<A,B>, scala.PartialFunction.Unlifted<A,B>
scala.collection.SeqLike.CombinationsItr, scala.collection.SeqLike.PermutationsItr
scala.collection.TraversableLike.WithFilter
scala.collection.TraversableOnce.BufferedCanBuildFrom<A,Coll extends scala.collection.TraversableOnce<java.lang.Object>>, scala.collection.TraversableOnce.FlattenOps<A>, scala.collection.TraversableOnce.ForceImplicitAmbiguity, scala.collection.TraversableOnce.MonadOps<A>, scala.collection.TraversableOnce.OnceCanBuildFrom<A>
Constructor and Description |
---|
StructType()
No-arg constructor for kryo.
|
StructType(StructField[] fields) |
Modifier and Type | Method and Description |
---|---|
StructType |
add(java.lang.String name,
DataType dataType)
Creates a new
StructType by adding a new nullable field with no metadata. |
StructType |
add(java.lang.String name,
DataType dataType,
boolean nullable)
Creates a new
StructType by adding a new field with no metadata. |
StructType |
add(java.lang.String name,
DataType dataType,
boolean nullable,
Metadata metadata)
Creates a new
StructType by adding a new field and specifying metadata. |
StructType |
add(java.lang.String name,
java.lang.String dataType)
Creates a new
StructType by adding a new nullable field with no metadata where the
dataType is specified as a String. |
StructType |
add(java.lang.String name,
java.lang.String dataType,
boolean nullable)
Creates a new
StructType by adding a new field with no metadata where the
dataType is specified as a String. |
StructType |
add(java.lang.String name,
java.lang.String dataType,
boolean nullable,
Metadata metadata)
Creates a new
StructType by adding a new field and specifying metadata where the
dataType is specified as a String. |
StructType |
add(StructField field)
Creates a new
StructType by adding a new field. |
StructField |
apply(int fieldIndex) |
StructType |
apply(scala.collection.immutable.Set<java.lang.String> names)
Returns a
StructType containing StructField s of the given names, preserving the
original order of fields. |
StructField |
apply(java.lang.String name)
Extracts a
StructField of the given name. |
int |
defaultSize()
The default size of a value of the StructType is the total default sizes of all field types.
|
int |
fieldIndex(java.lang.String name)
Returns index of a given field
|
java.lang.String[] |
fieldNames()
Returns all field names in an array.
|
StructField[] |
fields() |
protected static StructType |
fromAttributes(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> attributes) |
org.apache.spark.sql.catalyst.expressions.InterpretedOrdering |
interpretedOrdering() |
scala.collection.Iterator<StructField> |
iterator() |
int |
length() |
void |
printTreeString() |
java.lang.String |
simpleString()
Readable string representation for the type.
|
protected scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.AttributeReference> |
toAttributes() |
java.lang.String |
treeString() |
buildFormattedString, fromCaseClassString, fromJson, json, prettyJson, typeName
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
andThen, applyOrElse, isDefinedAt, lift, orElse, runWith
andThen$mcDD$sp, andThen$mcDF$sp, andThen$mcDI$sp, andThen$mcDJ$sp, andThen$mcFD$sp, andThen$mcFF$sp, andThen$mcFI$sp, andThen$mcFJ$sp, andThen$mcID$sp, andThen$mcIF$sp, andThen$mcII$sp, andThen$mcIJ$sp, andThen$mcJD$sp, andThen$mcJF$sp, andThen$mcJI$sp, andThen$mcJJ$sp, andThen$mcVD$sp, andThen$mcVF$sp, andThen$mcVI$sp, andThen$mcVJ$sp, andThen$mcZD$sp, andThen$mcZF$sp, andThen$mcZI$sp, andThen$mcZJ$sp, apply, apply$mcDD$sp, apply$mcDF$sp, apply$mcDI$sp, apply$mcDJ$sp, apply$mcFD$sp, apply$mcFF$sp, apply$mcFI$sp, apply$mcFJ$sp, apply$mcID$sp, apply$mcIF$sp, apply$mcII$sp, apply$mcIJ$sp, apply$mcJD$sp, apply$mcJF$sp, apply$mcJI$sp, apply$mcJJ$sp, apply$mcVD$sp, apply$mcVF$sp, apply$mcVI$sp, apply$mcVJ$sp, apply$mcZD$sp, apply$mcZF$sp, apply$mcZI$sp, apply$mcZJ$sp, compose, compose$mcDD$sp, compose$mcDF$sp, compose$mcDI$sp, compose$mcDJ$sp, compose$mcFD$sp, compose$mcFF$sp, compose$mcFI$sp, compose$mcFJ$sp, compose$mcID$sp, compose$mcIF$sp, compose$mcII$sp, compose$mcIJ$sp, compose$mcJD$sp, compose$mcJF$sp, compose$mcJI$sp, compose$mcJJ$sp, compose$mcVD$sp, compose$mcVF$sp, compose$mcVI$sp, compose$mcVJ$sp, compose$mcZD$sp, compose$mcZF$sp, compose$mcZI$sp, compose$mcZJ$sp, toString
$colon$plus, $plus$colon, combinations, contains, containsSlice, corresponds, diff, distinct, endsWith, indexOfSlice, indexOfSlice, indexWhere, indices, intersect, isEmpty, lastIndexOfSlice, lastIndexOfSlice, lastIndexWhere, lengthCompare, padTo, parCombiner, patch, permutations, reverse, reverseIterator, reverseMap, segmentLength, size, sortBy, sorted, sortWith, startsWith, thisCollection, toCollection, toSeq, toString, union, updated, view, view
canEqual, copyToArray, drop, dropRight, exists, find, foldRight, forall, foreach, grouped, head, reduceRight, sameElements, slice, sliding, sliding, take, takeRight, takeWhile, toIterable, toIterator, toStream, zip, zipAll, zipWithIndex
$plus$plus, $plus$plus$colon, $plus$plus$colon, collect, dropWhile, filter, filterNot, flatMap, groupBy, hasDefiniteSize, headOption, init, inits, isTraversableAgain, last, lastOption, map, partition, repr, scan, scanLeft, scanRight, sliceWithKnownBound, sliceWithKnownDelta, span, splitAt, stringPrefix, tail, tails, to, toTraversable, withFilter
$colon$bslash, $div$colon, addString, addString, addString, aggregate, collectFirst, copyToArray, copyToArray, copyToBuffer, count, fold, foldLeft, max, maxBy, min, minBy, mkString, mkString, mkString, nonEmpty, product, reduce, reduceLeft, reduceLeftOption, reduceOption, reduceRightOption, reversed, sum, toArray, toBuffer, toIndexedSeq, toList, toMap, toSet, toVector
public StructType(StructField[] fields)
public StructType()
protected static StructType fromAttributes(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> attributes)
public StructField[] fields()
public java.lang.String[] fieldNames()
public StructType add(StructField field)
StructType
by adding a new field.
val struct = (new StructType)
.add(StructField("a", IntegerType, true))
.add(StructField("b", LongType, false))
.add(StructField("c", StringType, true))
field
- (undocumented)public StructType add(java.lang.String name, DataType dataType)
StructType
by adding a new nullable field with no metadata.
val struct = (new StructType) .add("a", IntegerType) .add("b", LongType) .add("c", StringType)
name
- (undocumented)dataType
- (undocumented)public StructType add(java.lang.String name, DataType dataType, boolean nullable)
StructType
by adding a new field with no metadata.
val struct = (new StructType) .add("a", IntegerType, true) .add("b", LongType, false) .add("c", StringType, true)
name
- (undocumented)dataType
- (undocumented)nullable
- (undocumented)public StructType add(java.lang.String name, DataType dataType, boolean nullable, Metadata metadata)
StructType
by adding a new field and specifying metadata.
val struct = (new StructType)
.add("a", IntegerType, true, Metadata.empty)
.add("b", LongType, false, Metadata.empty)
.add("c", StringType, true, Metadata.empty)
name
- (undocumented)dataType
- (undocumented)nullable
- (undocumented)metadata
- (undocumented)public StructType add(java.lang.String name, java.lang.String dataType)
StructType
by adding a new nullable field with no metadata where the
dataType is specified as a String.
val struct = (new StructType)
.add("a", "int")
.add("b", "long")
.add("c", "string")
name
- (undocumented)dataType
- (undocumented)public StructType add(java.lang.String name, java.lang.String dataType, boolean nullable)
StructType
by adding a new field with no metadata where the
dataType is specified as a String.
val struct = (new StructType)
.add("a", "int", true)
.add("b", "long", false)
.add("c", "string", true)
name
- (undocumented)dataType
- (undocumented)nullable
- (undocumented)public StructType add(java.lang.String name, java.lang.String dataType, boolean nullable, Metadata metadata)
StructType
by adding a new field and specifying metadata where the
dataType is specified as a String.
val struct = (new StructType)
.add("a", "int", true, Metadata.empty)
.add("b", "long", false, Metadata.empty)
.add("c", "string", true, Metadata.empty)
name
- (undocumented)dataType
- (undocumented)nullable
- (undocumented)metadata
- (undocumented)public StructField apply(java.lang.String name)
StructField
of the given name. If the StructType
object does not
have a name matching the given name, null
will be returned.name
- (undocumented)public StructType apply(scala.collection.immutable.Set<java.lang.String> names)
StructType
containing StructField
s of the given names, preserving the
original order of fields. Those names which do not have matching fields will be ignored.names
- (undocumented)public int fieldIndex(java.lang.String name)
name
- (undocumented)protected scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.AttributeReference> toAttributes()
public java.lang.String treeString()
public void printTreeString()
public StructField apply(int fieldIndex)
apply
in interface scala.collection.GenSeqLike<StructField,scala.collection.Seq<StructField>>
apply
in interface scala.collection.SeqLike<StructField,scala.collection.Seq<StructField>>
public int length()
length
in interface scala.collection.GenSeqLike<StructField,scala.collection.Seq<StructField>>
length
in interface scala.collection.SeqLike<StructField,scala.collection.Seq<StructField>>
public scala.collection.Iterator<StructField> iterator()
iterator
in interface scala.collection.GenIterableLike<StructField,scala.collection.Seq<StructField>>
iterator
in interface scala.collection.IterableLike<StructField,scala.collection.Seq<StructField>>
public int defaultSize()
defaultSize
in class DataType
public java.lang.String simpleString()
DataType
simpleString
in class DataType
public org.apache.spark.sql.catalyst.expressions.InterpretedOrdering interpretedOrdering()