public class MemoryManager extends Object
Modifier and Type | Class and Description |
---|---|
static interface |
MemoryManager.Callback |
Constructor and Description |
---|
MemoryManager(org.apache.hadoop.conf.Configuration conf)
Create the memory manager.
|
Modifier and Type | Method and Description |
---|---|
void |
addedRow(int rows)
Give the memory manager an opportunity for doing a memory check.
|
void |
addWriter(org.apache.hadoop.fs.Path path,
long requestedAllocation,
MemoryManager.Callback callback)
Add a new writer's memory allocation to the pool.
|
double |
getAllocationScale()
The scaling factor for each allocation to ensure that the pool isn't
oversubscribed.
|
long |
getTotalMemoryPool()
Get the total pool size that is available for ORC writers.
|
void |
notifyWriters()
Notify all of the writers that they should check their memory usage.
|
void |
removeWriter(org.apache.hadoop.fs.Path path)
Remove the given writer from the pool.
|
public MemoryManager(org.apache.hadoop.conf.Configuration conf)
conf
- use the configuration to find the maximum size of the memory
pool.public void addWriter(org.apache.hadoop.fs.Path path, long requestedAllocation, MemoryManager.Callback callback) throws IOException
path
- the file that is being writtenrequestedAllocation
- the requested buffer sizeIOException
public void removeWriter(org.apache.hadoop.fs.Path path) throws IOException
path
- the file that has been closedIOException
public long getTotalMemoryPool()
public double getAllocationScale()
public void addedRow(int rows) throws IOException
rows
- number of rows addedIOException
public void notifyWriters() throws IOException
IOException
Copyright © 2016 The Apache Software Foundation. All rights reserved.