FixedLengthArrayNoneEncoder Class Reference

#include <FixedLengthArrayNoneEncoder.h>

Inheritance diagram for FixedLengthArrayNoneEncoder:

Collaboration diagram for FixedLengthArrayNoneEncoder:

Public Member Functions
	FixedLengthArrayNoneEncoder (AbstractBuffer *buffer, size_t as)
size_t	getNumElemsForBytesEncodedDataAtIndices (const int8_t *index_data, const std::vector< size_t > &selected_idx, const size_t byte_limit) override
size_t	getNumElemsForBytesInsertData (const std::vector< ArrayDatum > *srcData, const int start_idx, const size_t numAppendElems, const size_t byteLimit, const bool replicating=false)
std::shared_ptr< ChunkMetadata >	appendEncodedDataAtIndices (const int8_t *index_data, int8_t *data, const std::vector< size_t > &selected_idx) override
std::shared_ptr< ChunkMetadata >	appendEncodedData (const int8_t *index_data, int8_t *data, const size_t start_idx, const size_t num_elements) override
std::shared_ptr< ChunkMetadata >	appendData (int8_t *&src_data, const size_t num_elems_to_append, const SQLTypeInfo &ti, const bool replicating=false, const int64_t offset=-1) override
std::shared_ptr< ChunkMetadata >	appendData (const std::vector< ArrayDatum > *srcData, const int start_idx, const size_t numAppendElems, const bool replicating=false)
void	getMetadata (const std::shared_ptr< ChunkMetadata > &chunkMetadata) override
std::shared_ptr< ChunkMetadata >	getMetadata (const SQLTypeInfo &ti) override
void	updateStats (const int64_t, const bool) override
void	updateStats (const double, const bool) override
void	reduceStats (const Encoder &) override
void	updateStats (const int8_t *const src_data, const size_t num_elements) override
void	updateStats (const std::vector< std::string > *const src_data, const size_t start_idx, const size_t num_elements) override
void	updateStats (const std::vector< ArrayDatum > *const src_data, const size_t start_idx, const size_t num_elements) override
void	writeMetadata (FILE *f) override
void	readMetadata (FILE *f) override
void	copyMetadata (const Encoder *copyFromEncoder) override
void	updateMetadata (int8_t *array)
bool	resetChunkStats (const ChunkStats &stats) override
	: Reset chunk level stats (min, max, nulls) using new values from the argument. More...
void	resetChunkStats () override
Public Member Functions inherited from Encoder
	Encoder (Data_Namespace::AbstractBuffer *buffer)
virtual	~Encoder ()
virtual void	updateStatsEncoded (const int8_t *const dst_data, const size_t num_elements)
size_t	getNumElems () const
void	setNumElems (const size_t num_elems)

Static Public Member Functions
static bool	is_null_ignore_not_null (const SQLTypeInfo &type, int8_t *array)
static bool	is_null (const SQLTypeInfo &type, int8_t *array)
Static Public Member Functions inherited from Encoder
static Encoder *	Create (Data_Namespace::AbstractBuffer *buffer, const SQLTypeInfo sqlType)

Public Attributes
Datum	elem_min
Datum	elem_max
bool	has_nulls
bool	initialized

Private Member Functions
bool	is_null (int8_t *array)
bool	is_null_ignore_not_null (int8_t *array)
void	update_elem_stats (const ArrayDatum &array)

Private Attributes
std::mutex	EncoderMutex_
std::mutex	print_mutex_
size_t	array_size

Additional Inherited Members
Protected Attributes inherited from Encoder
size_t	num_elems_
Data_Namespace::AbstractBuffer *	buffer_
DecimalOverflowValidator	decimal_overflow_validator_
DateDaysOverflowValidator	date_days_overflow_validator_

Detailed Description

Definition at line 40 of file FixedLengthArrayNoneEncoder.h.

Constructor & Destructor Documentation

FixedLengthArrayNoneEncoder::FixedLengthArrayNoneEncoder ( AbstractBuffer *  buffer, size_t  as  )

inline

Definition at line 42 of file FixedLengthArrayNoneEncoder.h.

      : Encoder(buffer), has_nulls(false), initialized(false), array_size(as) {}

Member Function Documentation

std::shared_ptr<ChunkMetadata> FixedLengthArrayNoneEncoder::appendData ( int8_t *&  src_data, const size_t  num_elems_to_append, const SQLTypeInfo &  ti, const bool  replicating = false, const int64_t  offset = -1  )

inlineoverridevirtual

Append data to the chunk buffer backing this encoder.

Parameters

src_data	Source data for the append
num_elems_to_append	Number of elements to append
ti	SQL Type Info for the column TODO(adb): used?
replicating	Pass one value and fill the chunk with it
offset	Write data starting at a given offset. Default is -1 which indicates an append, an offset of 0 rewrites the chunk up to num_elems_to_append.

Implements Encoder.

Definition at line 97 of file FixedLengthArrayNoneEncoder.h.

References UNREACHABLE.

Referenced by Chunk_NS::Chunk::appendData(), appendEncodedData(), and appendEncodedDataAtIndices().

                                                                                {
    UNREACHABLE();  // should never be called for arrays
    return nullptr;
  }

Here is the caller graph for this function:

std::shared_ptr<ChunkMetadata> FixedLengthArrayNoneEncoder::appendData ( const std::vector< ArrayDatum > *  srcData, const int  start_idx, const size_t  numAppendElems, const bool  replicating = false  )

inline

Definition at line 106 of file FixedLengthArrayNoneEncoder.h.

References Data_Namespace::AbstractBuffer::append(), array_size, Encoder::buffer_, CHECK_EQ, getMetadata(), Data_Namespace::AbstractBuffer::isDirty(), Encoder::num_elems_, Data_Namespace::AbstractBuffer::reserve(), Data_Namespace::AbstractBuffer::setDirty(), and updateStats().

                                                                            {
    const size_t existing_data_size = num_elems_ * array_size;
    const size_t append_data_size = array_size * numAppendElems;
    buffer_->reserve(existing_data_size + append_data_size);
    std::vector<int8_t> append_buffer(append_data_size);
    int8_t* append_ptr = append_buffer.data();

    // There was some worry about the change implemented to write the append data to an
    // intermediate buffer, but testing on import and ctas of 20M points, we never append
    // more than 1.6MB and 1MB of data at a time, respectively, so at least for fixed
    // length types this should not be an issue (varlen types, which can be massive even
    // for a single field/row, are a different story however)

    if (replicating) {
      const size_t len = (*srcData)[0].length;
      CHECK_EQ(len, array_size);
      const int8_t* replicated_ptr = (*srcData)[0].pointer;
      for (size_t i = 0; i < numAppendElems; ++i) {
        std::memcpy(append_ptr + i * array_size, replicated_ptr, array_size);
      }
    } else {
      for (size_t i = 0; i < numAppendElems; ++i) {
        // Length of the appended array should be equal to the fixed length,
        // all others should have been discarded, assert if something slips through
        const size_t source_idx = start_idx + i;
        const size_t len = (*srcData)[source_idx].length;
        CHECK_EQ(len, array_size);
        // NULL arrays have been filled with subtype's NULL sentinels,
        // should be appended as regular data, same size
        std::memcpy(
            append_ptr + i * array_size, (*srcData)[source_idx].pointer, array_size);
      }
    }

    buffer_->append(append_ptr, append_data_size);

    if (replicating) {
      updateStats(srcData, 0, 1);
    } else {
      updateStats(srcData, start_idx, numAppendElems);
    }

    // make sure buffer_ is flushed even if no new data is appended to it
    // (e.g. empty strings) because the metadata needs to be flushed.
    if (!buffer_->isDirty()) {
      buffer_->setDirty();
    }

    num_elems_ += numAppendElems;
    auto chunk_metadata = std::make_shared<ChunkMetadata>();
    getMetadata(chunk_metadata);
    return chunk_metadata;
  }

Here is the call graph for this function:

std::shared_ptr<ChunkMetadata> FixedLengthArrayNoneEncoder::appendEncodedData ( const int8_t *  index_data, int8_t *  data, const size_t  start_idx, const size_t  num_elements  )

inlineoverridevirtual

Append encoded data to the chunk buffer backing this encoder.

Parameters

index_data	- (optional) the index data of data to append
data	- the data to append
start_idx	- the position to start encoding from in the data array
num_elements	- the number of elements to encode from the data array

Returns

updated chunk metadata for the chunk buffer backing this encoder

NOTE: index_data must be non-null for varlen encoder types.

Implements Encoder.

Definition at line 83 of file FixedLengthArrayNoneEncoder.h.

References appendData(), and array_size.

                                                                                       {
    std::vector<ArrayDatum> data_subset;
    data_subset.reserve(num_elements);
    for (size_t count = 0; count < num_elements; ++count) {
      auto current_data = data + array_size * (start_idx + count);
      data_subset.emplace_back(
          ArrayDatum(array_size, current_data, false, DoNothingDeleter{}));
    }
    return appendData(&data_subset, 0, num_elements, false);
  }

Here is the call graph for this function:

std::shared_ptr<ChunkMetadata> FixedLengthArrayNoneEncoder::appendEncodedDataAtIndices ( const int8_t *  index_data, int8_t *  data, const std::vector< size_t > &  selected_idx  )

inlineoverridevirtual

Append selected encoded data to the chunk buffer backing this encoder.

Parameters

index_data	- (optional) the index data of data to append
data	- the data to append
selected_idx	- which indices in the encoded data to append

Returns

updated chunk metadata for the chunk buffer backing this encoder

NOTE: index_data must be non-null for varlen encoder types.

Implements Encoder.

Definition at line 67 of file FixedLengthArrayNoneEncoder.h.

References appendData(), array_size, and is_null_ignore_not_null().

                                                      {
    std::vector<ArrayDatum> data_subset;
    data_subset.reserve(selected_idx.size());
    for (const auto& index : selected_idx) {
      auto current_data = data + array_size * (index);
      data_subset.emplace_back(ArrayDatum(array_size,
                                          current_data,
                                          is_null_ignore_not_null(current_data),
                                          DoNothingDeleter{}));
    }
    return appendData(&data_subset, 0, selected_idx.size(), false);
  }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::copyMetadata ( const Encoder *  copyFromEncoder )

inlineoverridevirtual

Implements Encoder.

Definition at line 217 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, Encoder::getNumElems(), has_nulls, initialized, and Encoder::num_elems_.

                                                             {
    num_elems_ = copyFromEncoder->getNumElems();
    auto array_encoder =
        dynamic_cast<const FixedLengthArrayNoneEncoder*>(copyFromEncoder);
    elem_min = array_encoder->elem_min;
    elem_max = array_encoder->elem_max;
    has_nulls = array_encoder->has_nulls;
    initialized = array_encoder->initialized;
  }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::getMetadata ( const std::shared_ptr< ChunkMetadata > &  chunkMetadata )

inlineoverridevirtual

Reimplemented from Encoder.

Definition at line 163 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, Encoder::getMetadata(), and has_nulls.

Referenced by appendData().

                                                                               {
    Encoder::getMetadata(chunkMetadata);  // call on parent class
    chunkMetadata->fillChunkStats(elem_min, elem_max, has_nulls);
  }

Here is the call graph for this function:

Here is the caller graph for this function:

std::shared_ptr<ChunkMetadata> FixedLengthArrayNoneEncoder::getMetadata ( const SQLTypeInfo &  ti )

inlineoverridevirtual

Implements Encoder.

Definition at line 169 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, and has_nulls.

                                                                           {
    auto chunk_metadata = std::make_shared<ChunkMetadata>(
        ti, 0, 0, ChunkStats{elem_min, elem_max, has_nulls});
    return chunk_metadata;
  }

size_t FixedLengthArrayNoneEncoder::getNumElemsForBytesEncodedDataAtIndices ( const int8_t *  index_data, const std::vector< size_t > &  selected_idx, const size_t  byte_limit  )

inlineoverridevirtual

Compute the maximum number of variable length encoded elements given a byte limit

Parameters

index_data	- (optional) index data for the encoded type
selected_idx	- which indices in the encoded data to consider
byte_limit	- byte limit that must be respected

Returns

the number of elements

NOTE: optional parameters above may be ignored by the implementation, but may or may not be required depending on the encoder type backing the implementation.

Implements Encoder.

Definition at line 45 of file FixedLengthArrayNoneEncoder.h.

References array_size.

                                                                                   {
    size_t data_size = selected_idx.size() * array_size;
    if (data_size > byte_limit) {
      data_size = byte_limit;
    }
    return data_size / array_size;
  }

size_t FixedLengthArrayNoneEncoder::getNumElemsForBytesInsertData ( const std::vector< ArrayDatum > *  srcData, const int  start_idx, const size_t  numAppendElems, const size_t  byteLimit, const bool  replicating = false  )

inline

Definition at line 55 of file FixedLengthArrayNoneEncoder.h.

References array_size.

Referenced by Chunk_NS::Chunk::getNumElemsForBytesInsertData().

                                                                       {
    size_t dataSize = numAppendElems * array_size;
    if (dataSize > byteLimit) {
      dataSize = byteLimit;
    }
    return dataSize / array_size;
  }

Here is the caller graph for this function:

static bool FixedLengthArrayNoneEncoder::is_null ( const SQLTypeInfo &  type, int8_t *  array  )

inlinestatic

Definition at line 281 of file FixedLengthArrayNoneEncoder.h.

References SQLTypeInfo::get_notnull(), and is_null_ignore_not_null().

Referenced by Fragmenter_Namespace::FixedLenArrayChunkConverter::convertToColumnarFormat(), and updateMetadata().

                                                              {
    if (type.get_notnull()) {
      return false;
    }
    return is_null_ignore_not_null(type, array);
  }

Here is the call graph for this function:

Here is the caller graph for this function:

bool FixedLengthArrayNoneEncoder::is_null ( int8_t *  array )

inlineprivate

Definition at line 315 of file FixedLengthArrayNoneEncoder.h.

References Encoder::buffer_, Data_Namespace::AbstractBuffer::getSqlType(), and is_null().

Referenced by is_null().

{ return is_null(buffer_->getSqlType(), array); }

Here is the call graph for this function:

Here is the caller graph for this function:

static bool FixedLengthArrayNoneEncoder::is_null_ignore_not_null ( const SQLTypeInfo &  type, int8_t *  array  )

inlinestatic

Definition at line 231 of file FixedLengthArrayNoneEncoder.h.

References CHECK_EQ, SQLTypeInfo::get_compression(), SQLTypeInfo::get_subtype(), kBIGINT, kBOOLEAN, kCHAR, kDATE, kDECIMAL, kDOUBLE, kENCODING_DICT, kFLOAT, kINT, kNUMERIC, kSMALLINT, kTEXT, kTIME, kTIMESTAMP, kTINYINT, kVARCHAR, NULL_ARRAY_BIGINT, NULL_ARRAY_BOOLEAN, NULL_ARRAY_DOUBLE, NULL_ARRAY_FLOAT, NULL_ARRAY_INT, NULL_ARRAY_SMALLINT, NULL_ARRAY_TINYINT, and UNREACHABLE.

Referenced by appendEncodedDataAtIndices(), is_null(), and is_null_ignore_not_null().

                                                                              {
    switch (type.get_subtype()) {
      case kBOOLEAN: {
        return (array[0] == NULL_ARRAY_BOOLEAN);
      }
      case kINT: {
        const int32_t* int_array = (int32_t*)array;
        return (int_array[0] == NULL_ARRAY_INT);
      }
      case kSMALLINT: {
        const int16_t* smallint_array = (int16_t*)array;
        return (smallint_array[0] == NULL_ARRAY_SMALLINT);
      }
      case kTINYINT: {
        const int8_t* tinyint_array = (int8_t*)array;
        return (tinyint_array[0] == NULL_ARRAY_TINYINT);
      }
      case kBIGINT:
      case kNUMERIC:
      case kDECIMAL: {
        const int64_t* bigint_array = (int64_t*)array;
        return (bigint_array[0] == NULL_ARRAY_BIGINT);
      }
      case kFLOAT: {
        const float* flt_array = (float*)array;
        return (flt_array[0] == NULL_ARRAY_FLOAT);
      }
      case kDOUBLE: {
        const double* dbl_array = (double*)array;
        return (dbl_array[0] == NULL_ARRAY_DOUBLE);
      }
      case kTIME:
      case kTIMESTAMP:
      case kDATE: {
        const int64_t* tm_array = reinterpret_cast<int64_t*>(array);
        return (tm_array[0] == NULL_ARRAY_BIGINT);
      }
      case kCHAR:
      case kVARCHAR:
      case kTEXT: {
        CHECK_EQ(type.get_compression(), kENCODING_DICT);
        const int32_t* int_array = (int32_t*)array;
        return (int_array[0] == NULL_ARRAY_INT);
      }
      default:
        UNREACHABLE();
    }
    return false;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

bool FixedLengthArrayNoneEncoder::is_null_ignore_not_null ( int8_t *  array )

inlineprivate

Definition at line 317 of file FixedLengthArrayNoneEncoder.h.

References Encoder::buffer_, Data_Namespace::AbstractBuffer::getSqlType(), and is_null_ignore_not_null().

                                              {
    return is_null_ignore_not_null(buffer_->getSqlType(), array);
  }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::readMetadata ( FILE *  f )

inlineoverridevirtual

Implements Encoder.

Definition at line 208 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, has_nulls, initialized, and Encoder::num_elems_.

                                      {
    // assumes pointer is already in right place
    fread((int8_t*)&num_elems_, sizeof(size_t), 1, f);
    fread((int8_t*)&elem_min, sizeof(Datum), 1, f);
    fread((int8_t*)&elem_max, sizeof(Datum), 1, f);
    fread((int8_t*)&has_nulls, sizeof(bool), 1, f);
    fread((int8_t*)&initialized, sizeof(bool), 1, f);
  }

void FixedLengthArrayNoneEncoder::reduceStats ( const Encoder &  )

inlineoverridevirtual

Implements Encoder.

Definition at line 179 of file FixedLengthArrayNoneEncoder.h.

References CHECK.

{ CHECK(false); }

bool FixedLengthArrayNoneEncoder::resetChunkStats ( const ChunkStats &  )

inlineoverridevirtual

: Reset chunk level stats (min, max, nulls) using new values from the argument.

Returns

: True if an update occurred and the chunk needs to be flushed. False otherwise. Default false if metadata update is unsupported. Only reset chunk stats if the incoming stats differ from the current stats.

Reimplemented from Encoder.

Definition at line 288 of file FixedLengthArrayNoneEncoder.h.

References Encoder::buffer_, DatumEqual(), elem_max, elem_min, SQLTypeInfo::get_elem_type(), Data_Namespace::AbstractBuffer::getSqlType(), ChunkStats::has_nulls, has_nulls, initialized, ChunkStats::max, and ChunkStats::min.

                                                         {
    auto elem_type = buffer_->getSqlType().get_elem_type();
    if (initialized && DatumEqual(elem_min, stats.min, elem_type) &&
        DatumEqual(elem_max, stats.max, elem_type) && has_nulls == stats.has_nulls) {
      return false;
    }
    elem_min = stats.min;
    elem_max = stats.max;
    has_nulls = stats.has_nulls;
    return true;
  }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::resetChunkStats ( )

inlineoverridevirtual

Resets chunk metadata stats to their default values.

Implements Encoder.

Definition at line 300 of file FixedLengthArrayNoneEncoder.h.

References has_nulls, and initialized.

                                  {
    has_nulls = false;
    initialized = false;
  }

void FixedLengthArrayNoneEncoder::update_elem_stats ( const ArrayDatum &  array )

inlineprivate

Definition at line 321 of file FixedLengthArrayNoneEncoder.h.

References Datum::bigintval, Datum::boolval, Encoder::buffer_, CHECK_EQ, Encoder::decimal_overflow_validator_, Datum::doubleval, elem_max, elem_min, Datum::floatval, SQLTypeInfo::get_compression(), SQLTypeInfo::get_subtype(), Data_Namespace::AbstractBuffer::getSqlType(), has_nulls, initialized, Datum::intval, kBIGINT, kBOOLEAN, kCHAR, kDATE, kDECIMAL, kDOUBLE, kENCODING_DICT, kFLOAT, kINT, kNUMERIC, kSMALLINT, kTEXT, kTIME, kTIMESTAMP, kTINYINT, kVARCHAR, NULL_BIGINT, NULL_BOOLEAN, NULL_DOUBLE, NULL_FLOAT, NULL_INT, NULL_SMALLINT, NULL_TINYINT, Datum::smallintval, Datum::tinyintval, UNREACHABLE, and DecimalOverflowValidator::validate().

Referenced by updateMetadata(), and updateStats().

                                                  {
    if (array.is_null) {
      has_nulls = true;
    }
    switch (buffer_->getSqlType().get_subtype()) {
      case kBOOLEAN: {
        if (!initialized) {
          elem_min.boolval = true;
          elem_max.boolval = false;
        }
        if (array.is_null) {
          break;
        }
        const int8_t* bool_array = array.pointer;
        for (size_t i = 0; i < array.length / sizeof(bool); i++) {
          if (bool_array[i] == NULL_BOOLEAN) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.boolval = std::min(elem_min.boolval, bool_array[i]);
            elem_max.boolval = std::max(elem_max.boolval, bool_array[i]);
          } else {
            elem_min.boolval = bool_array[i];
            elem_max.boolval = bool_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kINT: {
        if (!initialized) {
          elem_min.intval = 1;
          elem_max.intval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int32_t* int_array = (int32_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int32_t); i++) {
          if (int_array[i] == NULL_INT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.intval = std::min(elem_min.intval, int_array[i]);
            elem_max.intval = std::max(elem_max.intval, int_array[i]);
          } else {
            elem_min.intval = int_array[i];
            elem_max.intval = int_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kSMALLINT: {
        if (!initialized) {
          elem_min.smallintval = 1;
          elem_max.smallintval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int16_t* smallint_array = (int16_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int16_t); i++) {
          if (smallint_array[i] == NULL_SMALLINT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.smallintval = std::min(elem_min.smallintval, smallint_array[i]);
            elem_max.smallintval = std::max(elem_max.smallintval, smallint_array[i]);
          } else {
            elem_min.smallintval = smallint_array[i];
            elem_max.smallintval = smallint_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kTINYINT: {
        if (!initialized) {
          elem_min.tinyintval = 1;
          elem_max.tinyintval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int8_t* tinyint_array = (int8_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int8_t); i++) {
          if (tinyint_array[i] == NULL_TINYINT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.tinyintval = std::min(elem_min.tinyintval, tinyint_array[i]);
            elem_max.tinyintval = std::max(elem_max.tinyintval, tinyint_array[i]);
          } else {
            elem_min.tinyintval = tinyint_array[i];
            elem_max.tinyintval = tinyint_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kBIGINT:
      case kNUMERIC:
      case kDECIMAL: {
        if (!initialized) {
          elem_min.bigintval = 1;
          elem_max.bigintval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int64_t* bigint_array = (int64_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int64_t); i++) {
          if (bigint_array[i] == NULL_BIGINT) {
            has_nulls = true;
          } else if (initialized) {
            decimal_overflow_validator_.validate(bigint_array[i]);
            elem_min.bigintval = std::min(elem_min.bigintval, bigint_array[i]);
            elem_max.bigintval = std::max(elem_max.bigintval, bigint_array[i]);
          } else {
            decimal_overflow_validator_.validate(bigint_array[i]);
            elem_min.bigintval = bigint_array[i];
            elem_max.bigintval = bigint_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kFLOAT: {
        if (!initialized) {
          elem_min.floatval = 1.0;
          elem_max.floatval = 0.0;
        }
        if (array.is_null) {
          break;
        }
        const float* flt_array = (float*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(float); i++) {
          if (flt_array[i] == NULL_FLOAT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.floatval = std::min(elem_min.floatval, flt_array[i]);
            elem_max.floatval = std::max(elem_max.floatval, flt_array[i]);
          } else {
            elem_min.floatval = flt_array[i];
            elem_max.floatval = flt_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kDOUBLE: {
        if (!initialized) {
          elem_min.doubleval = 1.0;
          elem_max.doubleval = 0.0;
        }
        if (array.is_null) {
          break;
        }
        const double* dbl_array = (double*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(double); i++) {
          if (dbl_array[i] == NULL_DOUBLE) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.doubleval = std::min(elem_min.doubleval, dbl_array[i]);
            elem_max.doubleval = std::max(elem_max.doubleval, dbl_array[i]);
          } else {
            elem_min.doubleval = dbl_array[i];
            elem_max.doubleval = dbl_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kTIME:
      case kTIMESTAMP:
      case kDATE: {
        if (!initialized) {
          elem_min.bigintval = 1;
          elem_max.bigintval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int64_t* tm_array = reinterpret_cast<int64_t*>(array.pointer);
        for (size_t i = 0; i < array.length / sizeof(int64_t); i++) {
          if (tm_array[i] == NULL_BIGINT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.bigintval = std::min(elem_min.bigintval, tm_array[i]);
            elem_max.bigintval = std::max(elem_max.bigintval, tm_array[i]);
          } else {
            elem_min.bigintval = tm_array[i];
            elem_max.bigintval = tm_array[i];
            initialized = true;
          }
        }
        break;
      }
      case kCHAR:
      case kVARCHAR:
      case kTEXT: {
        CHECK_EQ(buffer_->getSqlType().get_compression(), kENCODING_DICT);
        if (!initialized) {
          elem_min.intval = 1;
          elem_max.intval = 0;
        }
        if (array.is_null) {
          break;
        }
        const int32_t* int_array = (int32_t*)array.pointer;
        for (size_t i = 0; i < array.length / sizeof(int32_t); i++) {
          if (int_array[i] == NULL_INT) {
            has_nulls = true;
          } else if (initialized) {
            elem_min.intval = std::min(elem_min.intval, int_array[i]);
            elem_max.intval = std::max(elem_max.intval, int_array[i]);
          } else {
            elem_min.intval = int_array[i];
            elem_max.intval = int_array[i];
            initialized = true;
          }
        }
        break;
      }
      default:
        UNREACHABLE();
    }
  };

Here is the call graph for this function:

Here is the caller graph for this function:

void FixedLengthArrayNoneEncoder::updateMetadata ( int8_t *  array )

inline

Definition at line 227 of file FixedLengthArrayNoneEncoder.h.

References array_size, is_null(), and update_elem_stats().

                                     {
    update_elem_stats(ArrayDatum(array_size, array, is_null(array), DoNothingDeleter()));
  }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::updateStats ( const int64_t  , const bool    )

inlineoverridevirtual

Implements Encoder.

Definition at line 175 of file FixedLengthArrayNoneEncoder.h.

References CHECK.

Referenced by appendData().

{ CHECK(false); }

Here is the caller graph for this function:

void FixedLengthArrayNoneEncoder::updateStats ( const double  , const bool    )

inlineoverridevirtual

Implements Encoder.

Definition at line 177 of file FixedLengthArrayNoneEncoder.h.

References CHECK.

{ CHECK(false); }

void FixedLengthArrayNoneEncoder::updateStats ( const int8_t *const  src_data, const size_t  num_elements  )

inlineoverridevirtual

Update statistics for data without appending.

Parameters

src_data	- the data with which to update statistics
num_elements	- the number of elements to scan in the data

Implements Encoder.

Definition at line 181 of file FixedLengthArrayNoneEncoder.h.

References UNREACHABLE.

                                                                                     {
    UNREACHABLE();
  }

void FixedLengthArrayNoneEncoder::updateStats ( const std::vector< std::string > *const  src_data, const size_t  start_idx, const size_t  num_elements  )

inlineoverridevirtual

Update statistics for string data without appending.

Parameters

src_data	- the string data with which to update statistics
start_idx	- the offset into src_data to start the update
num_elements	- the number of elements to scan in the string data

Implements Encoder.

Definition at line 185 of file FixedLengthArrayNoneEncoder.h.

References UNREACHABLE.

                                                       {
    UNREACHABLE();
  }

void FixedLengthArrayNoneEncoder::updateStats ( const std::vector< ArrayDatum > *const  src_data, const size_t  start_idx, const size_t  num_elements  )

inlineoverridevirtual

Update statistics for array data without appending.

Parameters

src_data	- the array data with which to update statistics
start_idx	- the offset into src_data to start the update
num_elements	- the number of elements to scan in the array data

Implements Encoder.

Definition at line 191 of file FixedLengthArrayNoneEncoder.h.

References anonymous_namespace{Utm.h}::n, and update_elem_stats().

                                                       {
    for (size_t n = start_idx; n < start_idx + num_elements; n++) {
      update_elem_stats((*src_data)[n]);
    }
  }

Here is the call graph for this function:

void FixedLengthArrayNoneEncoder::writeMetadata ( FILE *  f )

inlineoverridevirtual

Implements Encoder.

Definition at line 199 of file FixedLengthArrayNoneEncoder.h.

References elem_max, elem_min, has_nulls, initialized, and Encoder::num_elems_.

                                       {
    // assumes pointer is already in right place
    fwrite((int8_t*)&num_elems_, sizeof(size_t), 1, f);
    fwrite((int8_t*)&elem_min, sizeof(Datum), 1, f);
    fwrite((int8_t*)&elem_max, sizeof(Datum), 1, f);
    fwrite((int8_t*)&has_nulls, sizeof(bool), 1, f);
    fwrite((int8_t*)&initialized, sizeof(bool), 1, f);
  }

Member Data Documentation

size_t FixedLengthArrayNoneEncoder::array_size

private

Definition at line 313 of file FixedLengthArrayNoneEncoder.h.

Referenced by appendData(), appendEncodedData(), appendEncodedDataAtIndices(), getNumElemsForBytesEncodedDataAtIndices(), getNumElemsForBytesInsertData(), and updateMetadata().

Datum FixedLengthArrayNoneEncoder::elem_max

Definition at line 306 of file FixedLengthArrayNoneEncoder.h.

Referenced by copyMetadata(), getMetadata(), readMetadata(), resetChunkStats(), update_elem_stats(), and writeMetadata().

Datum FixedLengthArrayNoneEncoder::elem_min

Definition at line 305 of file FixedLengthArrayNoneEncoder.h.

Referenced by copyMetadata(), getMetadata(), readMetadata(), resetChunkStats(), update_elem_stats(), and writeMetadata().

std::mutex FixedLengthArrayNoneEncoder::EncoderMutex_

private

Definition at line 311 of file FixedLengthArrayNoneEncoder.h.

bool FixedLengthArrayNoneEncoder::has_nulls

Definition at line 307 of file FixedLengthArrayNoneEncoder.h.

Referenced by copyMetadata(), getMetadata(), heavydb.thrift.ttypes.TColumnRange::read(), readMetadata(), resetChunkStats(), update_elem_stats(), heavydb.thrift.ttypes.TColumnRange::write(), and writeMetadata().

bool FixedLengthArrayNoneEncoder::initialized

Definition at line 308 of file FixedLengthArrayNoneEncoder.h.

Referenced by copyMetadata(), readMetadata(), resetChunkStats(), update_elem_stats(), and writeMetadata().

std::mutex FixedLengthArrayNoneEncoder::print_mutex_

private

Definition at line 312 of file FixedLengthArrayNoneEncoder.h.

---

The documentation for this class was generated from the following file:

• /home/jenkins-slave/workspace/core-os-doxygen/DataMgr/FixedLengthArrayNoneEncoder.h