org.apache.cassandra.hadoop.pig

Class CqlStorage

java.lang.Object

org.apache.pig.LoadFunc

org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

org.apache.cassandra.hadoop.pig.CqlStorage

All Implemented Interfaces:

org.apache.pig.LoadMetadata, org.apache.pig.StoreFuncInterface

public class CqlStorage
extends AbstractCassandraStorage

A LoadStoreFunc for retrieving data from and storing data to Cassandra A row from a standard CF will be returned as nested tuples: (((key1, value1), (key2, value2)), ((name1, val1), (name2, val2))).

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

AbstractCassandraStorage.CfInfo, AbstractCassandraStorage.MarshallerType

Field Summary

Fields inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

column_family, conf, DEFAULT_INPUT_FORMAT, DEFAULT_OUTPUT_FORMAT, initHostAddress, inputFormatClass, keyspace, loadSignature, outputFormatClass, PARTITION_FILTER_SIGNATURE, partitionerClass, password, PIG_INITIAL_ADDRESS, PIG_INPUT_FORMAT, PIG_INPUT_INITIAL_ADDRESS, PIG_INPUT_PARTITIONER, PIG_INPUT_RPC_PORT, PIG_INPUT_SPLIT_SIZE, PIG_OUTPUT_FORMAT, PIG_OUTPUT_INITIAL_ADDRESS, PIG_OUTPUT_PARTITIONER, PIG_OUTPUT_RPC_PORT, PIG_PARTITIONER, PIG_RPC_PORT, rpcPort, splitSize, storeSignature, usePartitionFilter, username

Constructor Summary

Constructors

Constructor and Description
CqlStorage()
CqlStorage(int pageSize)

Method Summary

Methods

Modifier and Type	Method and Description
protected java.util.List<ColumnDef>	getColumnMetadata(Cassandra.Client client) include key columns
protected java.util.List<ColumnDef>	getKeysMeta(Cassandra.Client client) get keys meta data
org.apache.pig.data.Tuple	getNext() get next row
org.apache.pig.ResourceSchema	getSchema(java.lang.String location, org.apache.hadoop.mapreduce.Job job) schema: (value, value, value) where keys are in the front.
java.nio.ByteBuffer	nullToBB() Thrift API can't handle null, so use empty byte array
void	prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader, org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit split)
void	prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)
void	putNext(org.apache.pig.data.Tuple t) output: (((name, value), (name, value)), (value ...
void	setLocation(java.lang.String location, org.apache.hadoop.mapreduce.Job job) set read configuration settings
void	setPartitionFilter(org.apache.pig.Expression partitionFilter)
void	setStoreLocation(java.lang.String location, org.apache.hadoop.mapreduce.Job job) set store configuration settings

Methods inherited from class org.apache.cassandra.hadoop.pig.AbstractCassandraStorage

cassandraToObj, cfdefFromString, cfdefToString, checkSchema, cleanupOnFailure, columnToTuple, composeComposite, getCfDefinition, getCfInfo, getCfInfo, getColumnMeta, getDefaultMarshallers, getFullyQualifiedClassName, getIndexes, getIndexType, getInputFormat, getOutputFormat, getPartitionKeys, getPigType, getQueryMap, getStatistics, getValidatorMap, initSchema, objToBB, parseType, relativeToAbsolutePath, relToAbsPathForStoreLocation, setConnectionInformation, setStoreFuncUDFContextSignature, setTupleValue, setUDFContextSignature

Methods inherited from class org.apache.pig.LoadFunc

getAbsolutePath, getLoadCaster, getPathStrings, join, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CqlStorage

public CqlStorage()

CqlStorage

public CqlStorage(int pageSize)

Parameters:

limit - number of CQL rows to fetch in a thrift request

Method Detail

prepareToRead

public void prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader,
                 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit split)

Specified by:

prepareToRead in class org.apache.pig.LoadFunc

getNext

public org.apache.pig.data.Tuple getNext()
                                  throws java.io.IOException

get next row

Specified by:

getNext in class org.apache.pig.LoadFunc

Throws:

java.io.IOException

setLocation

public void setLocation(java.lang.String location,
               org.apache.hadoop.mapreduce.Job job)
                 throws java.io.IOException

set read configuration settings

Specified by:

setLocation in class org.apache.pig.LoadFunc

Throws:

java.io.IOException

setStoreLocation

public void setStoreLocation(java.lang.String location,
                    org.apache.hadoop.mapreduce.Job job)
                      throws java.io.IOException

set store configuration settings

Throws:

java.io.IOException

getSchema

public org.apache.pig.ResourceSchema getSchema(java.lang.String location,
                                      org.apache.hadoop.mapreduce.Job job)
                                        throws java.io.IOException

schema: (value, value, value) where keys are in the front.

Throws:

java.io.IOException

setPartitionFilter

public void setPartitionFilter(org.apache.pig.Expression partitionFilter)
                        throws java.io.IOException

Throws:

java.io.IOException

prepareToWrite

public void prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)

putNext

public void putNext(org.apache.pig.data.Tuple t)
             throws java.io.IOException

output: (((name, value), (name, value)), (value ... value), (value...value))

Throws:

java.io.IOException

getColumnMetadata

protected java.util.List<ColumnDef> getColumnMetadata(Cassandra.Client client)
                                               throws InvalidRequestException,
                                                      UnavailableException,
                                                      TimedOutException,
                                                      SchemaDisagreementException,
                                                      org.apache.thrift.TException,
                                                      java.nio.charset.CharacterCodingException,
                                                      InvalidRequestException,
                                                      ConfigurationException,
                                                      NotFoundException

include key columns

Specified by:

getColumnMetadata in class AbstractCassandraStorage

Throws:

InvalidRequestException

UnavailableException

TimedOutException

SchemaDisagreementException

org.apache.thrift.TException

java.nio.charset.CharacterCodingException

ConfigurationException

NotFoundException

getKeysMeta

protected java.util.List<ColumnDef> getKeysMeta(Cassandra.Client client)
                                         throws java.lang.Exception

get keys meta data

Throws:

java.lang.Exception

nullToBB

public java.nio.ByteBuffer nullToBB()

Thrift API can't handle null, so use empty byte array

Specified by:

nullToBB in class AbstractCassandraStorage