public static final class AcidUtils.OrcAcidVersion extends Object
ACID_FORMAT
file is written to each
base/delta/delete_delta dir written by a full acid write or compaction. This is the primary
mechanism for versioning acid data.
Each individual ORC file written stores the current version as a user property in ORC footer.
All data files produced by Acid write should have this (starting with Hive 3.0), including
those written by compactor. This is more for sanity checking in case someone moved the files
around or something like that.Modifier and Type | Field and Description |
---|---|
static int |
ORC_ACID_VERSION
2 is the version of Acid released in Hive 3.0.
|
static int |
ORC_ACID_VERSION_DEFAULT |
Constructor and Description |
---|
OrcAcidVersion() |
Modifier and Type | Method and Description |
---|---|
static int |
getAcidVersionFromDataFile(org.apache.hadoop.fs.Path dataFile,
org.apache.hadoop.fs.FileSystem fs)
This is smart enough to handle streaming ingest where there could be a
OrcAcidUtils.DELTA_SIDE_FILE_SUFFIX side file. |
static int |
getAcidVersionFromMetaFile(org.apache.hadoop.fs.Path deltaOrBaseDir,
org.apache.hadoop.fs.FileSystem fs) |
static org.apache.hadoop.fs.Path |
getVersionFilePath(org.apache.hadoop.fs.Path deltaOrBase) |
static void |
setAcidVersionInDataFile(Writer writer)
Inlucde current acid version in file footer.
|
static void |
writeVersionFile(org.apache.hadoop.fs.Path deltaOrBaseDir,
org.apache.hadoop.fs.FileSystem fs)
This creates a version file in
deltaOrBaseDir |
public static final int ORC_ACID_VERSION_DEFAULT
public static final int ORC_ACID_VERSION
public static void setAcidVersionInDataFile(Writer writer)
writer
- - file writtenpublic static int getAcidVersionFromDataFile(org.apache.hadoop.fs.Path dataFile, org.apache.hadoop.fs.FileSystem fs) throws IOException
OrcAcidUtils.DELTA_SIDE_FILE_SUFFIX
side file.dataFile
- - ORC acid data fileORC_ACID_VERSION_DEFAULT
otherwiseIOException
public static void writeVersionFile(org.apache.hadoop.fs.Path deltaOrBaseDir, org.apache.hadoop.fs.FileSystem fs) throws IOException
deltaOrBaseDir
deltaOrBaseDir
- - where to create the version fileIOException
public static org.apache.hadoop.fs.Path getVersionFilePath(org.apache.hadoop.fs.Path deltaOrBase)
public static int getAcidVersionFromMetaFile(org.apache.hadoop.fs.Path deltaOrBaseDir, org.apache.hadoop.fs.FileSystem fs) throws IOException
IOException
Copyright © 2022 The Apache Software Foundation. All rights reserved.