foreign_storage::FileReaderMap Class Reference

#include <ParquetShared.h>

Public Member Functions
const ReaderPtr	getOrInsert (const std::string &path, std::shared_ptr< arrow::fs::FileSystem > &file_system)
const ReaderPtr	insert (const std::string &path, std::shared_ptr< arrow::fs::FileSystem > &file_system)
void	initializeIfEmpty (const std::string &path)
void	clear ()

Private Attributes
std::mutex	mutex_
std::map< const std::string, UniqueReaderPtr >	map_

Detailed Description

Definition at line 68 of file ParquetShared.h.

Member Function Documentation

void foreign_storage::FileReaderMap::clear ( )

inline

Definition at line 93 of file ParquetShared.h.

References map_, and mutex_.

               {
    heavyai::unique_lock<std::mutex> cache_lock(mutex_);
    map_.clear();
  }

const ReaderPtr foreign_storage::FileReaderMap::getOrInsert ( const std::string &  path, std::shared_ptr< arrow::fs::FileSystem > &  file_system  )

inline

Definition at line 70 of file ParquetShared.h.

References map_, mutex_, and foreign_storage::open_parquet_table().

Referenced by foreign_storage::RowGroupIntervalTracker::advanceToNextRowGroup(), foreign_storage::LazyParquetChunkLoader::appendRowGroups(), and foreign_storage::LazyParquetChunkLoader::previewFiles().

                                                                             {
    heavyai::unique_lock<std::mutex> cache_lock(mutex_);
    if (map_.count(path) < 1 || !(map_.at(path))) {
      map_[path] = open_parquet_table(path, file_system);
    }
    return map_.at(path).get();
  }

Here is the call graph for this function:

Here is the caller graph for this function:

void foreign_storage::FileReaderMap::initializeIfEmpty ( const std::string &  path )

inline

Definition at line 86 of file ParquetShared.h.

References map_, and mutex_.

Referenced by foreign_storage::LazyParquetChunkLoader::metadataScan().

                                                {
    heavyai::unique_lock<std::mutex> cache_lock(mutex_);
    if (map_.count(path) < 1) {
      map_.emplace(path, UniqueReaderPtr());
    }
  }

Here is the caller graph for this function:

const ReaderPtr foreign_storage::FileReaderMap::insert ( const std::string &  path, std::shared_ptr< arrow::fs::FileSystem > &  file_system  )

inline

Definition at line 79 of file ParquetShared.h.

References map_, mutex_, and foreign_storage::open_parquet_table().

Referenced by foreign_storage::LazyParquetChunkLoader::metadataScan().

                                                                        {
    heavyai::unique_lock<std::mutex> cache_lock(mutex_);
    map_[path] = open_parquet_table(path, file_system);
    return map_.at(path).get();
  }

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

std::map<const std::string, UniqueReaderPtr> foreign_storage::FileReaderMap::map_

private

Definition at line 100 of file ParquetShared.h.

Referenced by clear(), getOrInsert(), initializeIfEmpty(), and insert().

std::mutex foreign_storage::FileReaderMap::mutex_

mutableprivate

Definition at line 99 of file ParquetShared.h.

Referenced by clear(), getOrInsert(), initializeIfEmpty(), and insert().

---

The documentation for this class was generated from the following file:

• /home/jenkins-slave/workspace/core-os-doxygen/DataMgr/ForeignStorage/ParquetShared.h