CSVLoader Class Reference

Utility class for loading market quotes and fixings from a file. More...

#include <ored/marketdata/csvloader.hpp>

Inheritance diagram for CSVLoader:

Collaboration diagram for CSVLoader:

Public Member Functions
	CSVLoader ()
	Constructor. More...
	CSVLoader (const string &marketFilename, const string &fixingFilename, bool implyTodaysFixings=false, Date fixingCutOffDate=Date())
	CSVLoader (const vector< string > &marketFiles, const vector< string > &fixingFiles, bool implyTodaysFixings=false, Date fixingCutOffDate=Date())
	CSVLoader (const string &marketFilename, const string &fixingFilename, const string &dividendFilename, bool implyTodaysFixings=false, Date fixingCutOffDate=Date())
	CSVLoader (const vector< string > &marketFiles, const vector< string > &fixingFiles, const vector< string > &dividendFiles, bool implyTodaysFixings=false, Date fixingCutOffDate=Date())
std::vector< QuantLib::ext::shared_ptr< MarketDatum > >	loadQuotes (const QuantLib::Date &) const override
	get all quotes, TODO change the return value to std::set More...
QuantLib::ext::shared_ptr< MarketDatum >	get (const string &name, const QuantLib::Date &d) const override
	get quote by its unique name, throws if not existent, override in derived classes for performance More...
std::set< QuantLib::ext::shared_ptr< MarketDatum > >	get (const std::set< std::string > &names, const QuantLib::Date &asof) const override
	get quotes matching a set of names, this should be overridden in derived classes for performance More...
std::set< QuantLib::ext::shared_ptr< MarketDatum > >	get (const Wildcard &wildcard, const QuantLib::Date &asof) const override
	get quotes matching a wildcard More...
std::set< Fixing >	loadFixings () const override
	Load fixings. More...
std::set< QuantExt::Dividend >	loadDividends () const override
	Load dividends. More...
Public Member Functions inherited from Loader
virtual	~Loader ()
virtual bool	has (const std::string &name, const QuantLib::Date &d) const
	Default implementation, returns false if get throws or returns a null pointer. More...
virtual bool	hasQuotes (const QuantLib::Date &d) const
	check if there are quotes for a date More...
virtual QuantLib::ext::shared_ptr< MarketDatum >	get (const std::pair< std::string, bool > &name, const QuantLib::Date &d) const
virtual bool	hasFixing (const string &name, const QuantLib::Date &d) const
virtual Fixing	getFixing (const string &name, const QuantLib::Date &d) const
	Default implementation for getFixing. More...
void	setActualDate (const QuantLib::Date &d)
const Date &	actualDate () const
std::pair< bool, string >	checkFxDuplicate (const ext::shared_ptr< MarketDatum >, const QuantLib::Date &)

Private Types
enum class	DataType { Market , Fixing , Dividend }

Private Member Functions
void	loadFile (const string &, DataType)

Private Attributes
bool	implyTodaysFixings_
std::map< QuantLib::Date, std::set< QuantLib::ext::shared_ptr< MarketDatum >, SharedPtrMarketDatumComparator > >	data_
std::set< Fixing >	fixings_
std::set< QuantExt::Dividend >	dividends_
Date	fixingCutOffDate_

Additional Inherited Members
Protected Attributes inherited from Loader
Date	actualDate_ = Date()

Detailed Description

Utility class for loading market quotes and fixings from a file.

Data is loaded with the call to the constructor. Inspectors can be called to then retrieve quotes and fixings.

TODO implementation has large overlap with inmemoryloader.?pp, factor this out

Definition at line 41 of file csvloader.hpp.

Member Enumeration Documentation

DataType

enum class DataType

strongprivate

Enumerator
Market
Fixing
Dividend

Definition at line 101 of file csvloader.hpp.

{ Market, Fixing, Dividend };

Constructor & Destructor Documentation

CSVLoader() [1/5]

CSVLoader

(

)

Constructor.

Definition at line 44 of file csvloader.hpp.

{}

CSVLoader() [2/5]

CSVLoader	(	const string &	marketFilename,
		const string &	fixingFilename,
		bool	implyTodaysFixings = false,
		Date	fixingCutOffDate = Date()
	)

Parameters

marketFilename	Quote file name
fixingFilename	Fixing file name
implyTodaysFixings	Enable/disable implying today's fixings
fixingCutOffDate	Load fixings up to this date

Definition at line 38 of file csvloader.cpp.

    : CSVLoader(marketFilename, fixingFilename, "", implyTodaysFixings, fixingCutOffDate) {}

CSVLoader() [3/5]

CSVLoader	(	const vector< string > &	marketFiles,
		const vector< string > &	fixingFiles,
		bool	implyTodaysFixings = false,
		Date	fixingCutOffDate = Date()
	)

Parameters

marketFiles	Quote file name
fixingFiles	Fixing file name
implyTodaysFixings	Enable/disable implying today's fixings
fixingCutOffDate	Load fixings up to this date

Definition at line 42 of file csvloader.cpp.

    : CSVLoader(marketFiles, fixingFiles, {}, implyTodaysFixings, fixingCutOffDate) {}

CSVLoader() [4/5]

CSVLoader	(	const string &	marketFilename,
		const string &	fixingFilename,
		const string &	dividendFilename,
		bool	implyTodaysFixings = false,
		Date	fixingCutOffDate = Date()
	)

Parameters

marketFilename	Quote file name
fixingFilename	Fixing file name
dividendFilename	Dividend file name
implyTodaysFixings	Enable/disable implying today's fixings
fixingCutOffDate	Load fixings up to this date

Definition at line 45 of file csvloader.cpp.

    : implyTodaysFixings_(implyTodaysFixings), fixingCutOffDate_(fixingCutOffDate) {
 
    // load market data
    loadFile(marketFilename, DataType::Market);
    // log
    for (auto it : data_) {
        LOG("CSVLoader loaded " << it.second.size() << " market data points for " << it.first);
    }
 
    // load fixings
    loadFile(fixingFilename, DataType::Fixing);
    LOG("CSVLoader loaded " << fixings_.size() << " fixings");
 
    // load dividends
    if (dividendFilename != "") {
        loadFile(dividendFilename, DataType::Dividend);
        LOG("CSVLoader loaded " << dividends_.size() << " dividends");
    }
 
    LOG("CSVLoader complete.");
}

Here is the call graph for this function:

CSVLoader() [5/5]

CSVLoader	(	const vector< string > &	marketFiles,
		const vector< string > &	fixingFiles,
		const vector< string > &	dividendFiles,
		bool	implyTodaysFixings = false,
		Date	fixingCutOffDate = Date()
	)

Parameters

marketFiles	Quote file name
fixingFiles	Fixing file name
dividendFiles	Dividend file name
implyTodaysFixings	Enable/disable implying today's fixings
fixingCutOffDate	Load fixings up to this date

Definition at line 69 of file csvloader.cpp.

    : implyTodaysFixings_(implyTodaysFixings), fixingCutOffDate_(fixingCutOffDate) {
 
    for (auto marketFile : marketFiles)
        // load market data
        loadFile(marketFile, DataType::Market);
 
    // log
    for (auto it : data_)
        LOG("CSVLoader loaded " << it.second.size() << " market data points for " << it.first);
 
    for (auto fixingFile : fixingFiles)
        // load fixings
        loadFile(fixingFile, DataType::Fixing);
    LOG("CSVLoader loaded " << fixings_.size() << " fixings");
 
    for (auto dividendFile : dividendFiles)
        // load dividends
        loadFile(dividendFile, DataType::Dividend);
    LOG("CSVLoader loaded " << dividends_.size() << " dividends");
 
    LOG("CSVLoader complete.");
}

Here is the call graph for this function:

Member Function Documentation

loadQuotes()

vector< QuantLib::ext::shared_ptr< MarketDatum > > loadQuotes ( const QuantLib::Date &  ) const

overridevirtual

get all quotes, TODO change the return value to std::set

Implements Loader.

Definition at line 192 of file csvloader.cpp.

                                                                                            {
    auto it = data_.find(d);
    if (it == data_.end())
        return {};
    return std::vector<QuantLib::ext::shared_ptr<MarketDatum>>(it->second.begin(), it->second.end());
}

get() [1/3]

QuantLib::ext::shared_ptr< MarketDatum > get ( const string &  name, const QuantLib::Date &  d  ) const

overridevirtual

get quote by its unique name, throws if not existent, override in derived classes for performance

Reimplemented from Loader.

Definition at line 199 of file csvloader.cpp.

                                                                                                 {
    auto it = data_.find(d);
    QL_REQUIRE(it != data_.end(), "No datum for " << name << " on date " << d);
    auto it2 = it->second.find(makeDummyMarketDatum(d, name));
    QL_REQUIRE(it2 != it->second.end(), "No datum for " << name << " on date " << d);
    return *it2;
}

Here is the call graph for this function:

Here is the caller graph for this function:

get() [2/3]

std::set< QuantLib::ext::shared_ptr< MarketDatum > > get ( const std::set< std::string > &  names, const QuantLib::Date &  asof  ) const

overridevirtual

get quotes matching a set of names, this should be overridden in derived classes for performance

Reimplemented from Loader.

Definition at line 207 of file csvloader.cpp.

                                                                                             {
    auto it = data_.find(asof);
    if(it == data_.end())
        return {};
    std::set<QuantLib::ext::shared_ptr<MarketDatum>> result;
    for (auto const& n : names) {
        auto it2 = it->second.find(makeDummyMarketDatum(asof, n));
        if (it2 != it->second.end())
            result.insert(*it2);
    }
    return result;
}

Here is the call graph for this function:

get() [3/3]

std::set< QuantLib::ext::shared_ptr< MarketDatum > > get ( const Wildcard &  wildcard, const QuantLib::Date &  asof  ) const

overridevirtual

get quotes matching a wildcard

Reimplemented from Loader.

Definition at line 221 of file csvloader.cpp.

                                                                                             {
    if (!wildcard.hasWildcard()) {
        // no wildcard => use get by name function
        try {
            return {get(wildcard.pattern(), asof)};
        } catch (...) {
        }
        return {};
    }
    auto it = data_.find(asof);
    if (it == data_.end())
        return {};
    std::set<QuantLib::ext::shared_ptr<MarketDatum>> result;
    std::set<QuantLib::ext::shared_ptr<MarketDatum>>::iterator it1, it2;
    if (wildcard.wildcardPos() == 0) {
        // wildcard at first position => we have to search all of the data
        it1 = it->second.begin();
        it2 = it->second.end();
    } else {
        // search the range matching the substring of the pattern until the wildcard
        std::string prefix = wildcard.pattern().substr(0, wildcard.wildcardPos());
        it1 = it->second.lower_bound(makeDummyMarketDatum(asof, prefix));
        it2 = it->second.upper_bound(makeDummyMarketDatum(asof, prefix + "\xFF"));
    }
    for (auto it = it1; it != it2; ++it) {
        if (wildcard.isPrefix() || wildcard.matches((*it)->name()))
            result.insert(*it);
    }
    return result;
}

Here is the call graph for this function:

loadFixings()

std::set< Fixing > loadFixings ( ) const

overridevirtual

Load fixings.

Implements Loader.

Definition at line 95 of file csvloader.hpp.

{ return fixings_; }

loadDividends()

std::set< QuantExt::Dividend > loadDividends ( ) const

overridevirtual

Load dividends.

Reimplemented from Loader.

Definition at line 97 of file csvloader.hpp.

{ return dividends_; }

loadFile()

void loadFile ( const string &  filename, DataType  dataType  )

private

Definition at line 100 of file csvloader.cpp.

                                                                  {
    LOG("CSVLoader loading from " << filename);
 
    Date today = QuantLib::Settings::instance().evaluationDate();
 
    ifstream file;
    file.open(filename.c_str());
    QL_REQUIRE(file.is_open(), "error opening file " << filename);
 
    while (!file.eof()) {
        string line;
        getline(file, line);
        boost::trim(line);
        // skip blank and comment lines
        if (line.size() > 0 && line[0] != '#') {
 
            vector<string> tokens;
            boost::trim(line);
            boost::split(tokens, line, boost::is_any_of(",;\t "), boost::token_compress_on);
 
            // TODO: should we try, catch and log any invalid lines?
            QL_REQUIRE(tokens.size() == 3 || tokens.size() == 4, "Invalid CSVLoader line, 3 tokens expected " << line);
            if (tokens.size() == 4)
                QL_REQUIRE(dataType == DataType::Dividend, "CSVLoader, dataType must be of type Dividend");
            Date date = parseDate(tokens[0]);
            const string& key = tokens[1];
            Real value = parseReal(tokens[2]);
 
            if (dataType == DataType::Market) {
                // process market
                // build market datum and add to map
                try {
                    QuantLib::ext::shared_ptr<MarketDatum> md;
                    try {
                        md = parseMarketDatum(date, key, value);
                    } catch (std::exception& e) {
                        WLOG("Failed to parse MarketDatum " << key << ": " << e.what());
                    }
                    if (md != nullptr) {
                        std::pair<bool, string> addFX = {true, ""};
                        if (md->instrumentType() == MarketDatum::InstrumentType::FX_SPOT &&
                            md->quoteType() == MarketDatum::QuoteType::RATE) {
                            addFX = checkFxDuplicate(md, date);
                            if (!addFX.second.empty()) {
                                auto it2 = data_[date].find(makeDummyMarketDatum(date, addFX.second));
                                TLOG("Replacing MarketDatum " << addFX.second << " with " << key
                                                                  << " due to FX Dominance.");
                                if (it2 != data_[date].end())
                                    data_[date].erase(it2);
                            }
                        }
                        if (addFX.first && data_[date].insert(md).second) {
                            LOG("Added MarketDatum " << key);
                        } else if (!addFX.first) {
                            LOG("Skipped MarketDatum " << key << " - dominant FX already present.")
                        }
                        else {
                            LOG("Skipped MarketDatum " << key << " - this is already present.");
                        }
                    }
                } catch (std::exception& e) {
                    WLOG("Failed to parse MarketDatum " << key << ": " << e.what());
                }
            } else if (dataType == DataType::Fixing) {
                // process fixings
                if (date < today || (date == today && !implyTodaysFixings_)
            || (fixingCutOffDate_ != Date() && date <= fixingCutOffDate_)) {
                    if(!fixings_.insert(Fixing(date, key, value)).second) {
                        WLOG("Skipped Fixing " << key << "@" << QuantLib::io::iso_date(date)
                                               << " - this is already present.");
                    }
                }
            } else if (dataType == DataType::Dividend) {
                Date payDate = date;
                if (tokens.size() == 4)
                    payDate = parseDate(tokens[3]);
                // process dividends
                if (date <= today) {
                    if (!dividends_.insert(QuantExt::Dividend(date, key, value, payDate)).second) {
                        WLOG("Skipped Dividend " << key << "@" << QuantLib::io::iso_date(date)
                                                 << " - this is already present.");
                    }
                }
            } else {
                QL_FAIL("unknown data type");
            }
        }
    }
    file.close();
    LOG("CSVLoader completed processing " << filename);
}

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

implyTodaysFixings_

bool implyTodaysFixings_

private

Definition at line 104 of file csvloader.hpp.

data_

std::map<QuantLib::Date, std::set<QuantLib::ext::shared_ptr<MarketDatum>, SharedPtrMarketDatumComparator> > data_

private

Definition at line 105 of file csvloader.hpp.

fixings_

std::set<Fixing> fixings_

private

Definition at line 106 of file csvloader.hpp.

dividends_

std::set<QuantExt::Dividend> dividends_

private

Definition at line 107 of file csvloader.hpp.

fixingCutOffDate_

Date fixingCutOffDate_

private

Definition at line 108 of file csvloader.hpp.