2

I want to be able to handle something like a table in C++. I mean table as in "excel spreadsheet" or "R dataFrame". My solution need not be that powerful, however. I do not need to add columns at runtime, but I will be adding rows to create averages. I am building this table from individual data points, and this table will be read elsewhere in the program for analysis of data. I feel like solutions such as SQLite are overkill. How can I simply represent this kind of data?

To more easily discuss options, consider the following table of ocean temperatures, which we will call data:

| DataType    | DateTime         | Location  | Temperature |
| ----------- | ---------------- | --------- | ----------- |
| Observation | 2020-07-03_1325  | buoy 3882 | 18.1        |
| Observation | 2020-07-03_1512  | buoy 3882 | 16.6        |
| Observation | 2020-07-03_1701  | buoy 3882 | 15.8        |
| DailyAvg    | 2020-07-03_0000  | buoy 3882 | 16.8        |

It is important that I be able to access the data based on any property so that I can (somewhat) quickly gather all points from a specific location, all points with the same date, etc.

I've considered making a 2d array (something like std::vector< std::vector<boost::any> >) but that requires the user to remember the position of the columns that hold the data they want. (e.g. for Temperature of the second data point, the user would have to use data[1][3]). I also considered making a struct that looks something like this:

struct DataPoint {
    ObservationType observationType;
    DateTime dateTime;
    std::string location;
    double temperature;
}

And then simply using std::vector<DataPoint> to keep track of an entire table.

Thus we come to my questions: do either of these approaches make sense? Is there another approach that makes more sense?

Secondly, in my mind a data point is immutable. Does it make sense to prevent the user from modifying the data after creation? If so, what is the best way to do this?

6
  • 9
    I think the struct DataPoint idea is right on the money. Jul 23, 2020 at 19:57
  • Thank you for the feedback. In a lot of ways I feel like I have no idea what I'm doing, so it's nice to know I'm not totally off :)
    – bweber13
    Jul 23, 2020 at 20:00
  • To prevent modification, use const std::vector<DataPoint>&. Jul 23, 2020 at 20:02
  • There are times when vector-of-vector is an appropriate data representation, but almost never, and usually not for the type of things beginners think they need it for. In this case it is highly inappropriate... more so because of the variant type boost::any. Besides a struct, another potential representation is to use one vector for each column. This is for more special situations where it's more important to have column data organized sequentially in memory. Beyond these, you would be looking to much more complex data representations to meet specific needs.
    – paddy
    Jul 23, 2020 at 20:48
  • 1
    No, it's not really appropriate for pixels in an image, unless perhaps your image is enormous and you need to break it up into blocks. It's most useful when you want a bunch of vectors whose lengths are independent of each other, or at least can be different. If you were using it for images, perhaps it would be suited to mipmap pyramids, where each vector is an entire image buffer. Or you might use it for collecting node adjacency information in a graph, at least as a temporary structure. What you don't do is use it as a convenient substitute for a 2D array.
    – paddy
    Jul 26, 2020 at 22:57

3 Answers 3

5

The option of std::vector<DataPoint> looks pretty good. You don't need to prevent data from modifying, just use const after creation.

If you need fast access to rows by location, temperature, etc. you may create specific indices based on map or more complex data structures.

For example, you may create std::multimap<double, size_t> where the key is a temperature and the value is a number of row in the table (index in vector). Then, you can use std::lower_bound and std::upper_bound to retrieve all indices of rows with a temperature in a given range.

4
  • When you say "use const after creation", does that mean supply the user with a function like const std::vector<DataPoint> buildTable()? This way inside the function we can edit the vector, but the user is just provided with read-only access?
    – bweber13
    Jul 23, 2020 at 20:03
  • Additionally, what does it mean to "create specific indices"?
    – bweber13
    Jul 23, 2020 at 20:05
  • @bweber13 const std::vector<DataPoint> buildTable() actually doesn't make sense. Use const for variable that stores the table: const auto table = MakeTable(); Jul 23, 2020 at 20:10
  • 2
    @bweber13 see Mooing Duck's comment under the question. But be careful. Returning a reference to a variable is a fatal mistake if the returned variable does not outlive the reference (a local variable, for example). Jul 23, 2020 at 20:11
1

My solution need not be that powerful

You can save the same data in multiple data structures, it's surely not the most efficient approach but if you have enough space and accessing the data is more critical - it's worth considering.

It is important that I be able to access the data based on any property so that I can (somewhat) quickly gather all points from a specific location, all points with the same date, etc.

Based on the struct DataPoint you can use std::multimap, which will give you the ability to access, for instance - all data points from a specific location, if location is the key of the multimap.

#include <iostream>
#include <vector>
#include <map> 

enum ObservationType{
    eReport,
    eHumanObservation,
    eCamera
};

typedef struct DataPoint {
    ObservationType observationType;
    std::string location;
    double temperature;
} DataPoint;


class Data {
    public:
        Data()
        {
        
            DataPoint d1 = {
                .observationType = ObservationType::eReport,
                .location = "buoy 3882",
                .temperature = 25.3
            };

            DataPoint d2 = {
                .observationType = ObservationType::eReport,
                .location = "buoy 3882",
                .temperature = 32.5
            };
    
            DataPoint d3 = {
                .observationType = ObservationType::eReport,
                .location = "buoy 1659",
                .temperature = 25.3
            };
        
            dataPoints.push_back(d1);
            dataPoints.push_back(d2);
            dataPoints.push_back(d3);

            //populating all data points in location multimap
            locMap.insert(std::pair<std::string,const DataPoint>(d1.location, d1));
            locMap.insert(std::pair<std::string,const DataPoint>(d2.location, d2));    
            locMap.insert(std::pair<std::string,const DataPoint>(d3.location, d3));    
    }
    
        const std::multimap<std::string, const DataPoint>& GetLocationMuMap() const {
            return locMap;
        }
    
    private:
        std::vector<DataPoint> dataPoints;
        std::multimap<std::string, const DataPoint> locMap;
};

int main()
{
    Data data = Data();

    const DataPoint d4 = {
        .observationType = ObservationType::eCamera,
        .location = "buoy 1659",
        .temperature = 28.34
    };
    //Error : next line will fail compilation because the multimap instance is const.
    //data.GetLocationMuMap().insert(std::pair<std::string,const DataPoint>(d4.location, d4));
    
    auto range = data.GetLocationMuMap().equal_range("buoy 3882");
    std::cout << "Temperatures measured at buoy 3882" << std::endl;
    for (auto it = range.first; it != range.second; ++it)
    {
        //Error : next line will fail - The value stored in the map is of type const DataPoint
        //(*it).second.observationType = eCamera;
        std::cout << "  [" << (*it).first << ", " << (*it).second.temperature << "]" << std::endl;
    }
    
    return 0;
}

Does it make sense to prevent the user from modifying the data after creation

Yes it does. The integrity of data is critical to any program, using const & will protect from modifications of DataPoint instances, whilst using the same memory for each data point.

Note that keys should be comparable, and if your'e using reference - & the data must still exist in some context (in this example- the vector).

You can save other parameters (such as temperature) as keys in other multimaps and achieve easy access.

4
  • Your answer and especially the example code are very helpful. Could you please tell me a bit more about how you would go about using const & to prevent modification? Would you do something like const & vector<DataPoint> data = buildTable() as discussed in another answer?
    – bweber13
    Jul 24, 2020 at 19:22
  • @bweber13 Personally I wouldn't want to count on managing indices since it's not intuitive - using multiple data structures helps you organize your data in an intuitive way - by specific keys, Edited my answer to give a full example.
    – joepol
    Jul 26, 2020 at 19:44
  • Thank you for the clarification. While it was not part of my original requirements, I am curious: would it be possible to use more than one multimap together to filter data? For instance, could I use this structure to find all points from "buoy 3882" with temperature above 30 degrees?
    – bweber13
    Jul 27, 2020 at 16:25
  • @bweber13 - I added another answer (using lambdas) which is easier for filtering data , but less memory oriented.
    – joepol
    Jul 30, 2020 at 5:50
0

This is a different approach - using the <algorithm> library and lambdas.

Note that is not a memory efficient solution, but it is pretty straightforward and easy to create multiple query function (lambdas), for simple queries/filtering you can also use a conditional for loop instead.

#include <iostream>
#include <vector>
#include <algorithm>

enum ObservationType{
    eReport,
    eHumanObservation,
    eCamera
};

typedef struct DataPoint {
    ObservationType observationType;
    std::string location;
    double temperature;
} DataPoint;


class Data {
    public:
        Data()
        {
        
            DataPoint d1 = {
                .observationType = ObservationType::eReport,
                .location = "buoy 3882",
                .temperature = 25.3
            };

            DataPoint d2 = {
                .observationType = ObservationType::eReport,
                .location = "buoy 3882",
                .temperature = 32.5
            };
    
            DataPoint d3 = {
                .observationType = ObservationType::eReport,
                .location = "buoy 1659",
                .temperature = 25.3
            };
        
            dataPoints.push_back(d1);
            dataPoints.push_back(d2);
            dataPoints.push_back(d3);

    }
    
    std::vector<DataPoint> GetDataPoints(const std::string& location, double lowBoundTemp)
    {
        std::vector<DataPoint> filteredDp;
        auto predicate = [&](const DataPoint& dp)
                            {if(dp.temperature >= lowBoundTemp && dp.location.compare(location))
                                return true;};
        std::copy_if (dataPoints.begin(), dataPoints.end(), std::back_inserter(filteredDp), predicate);
        return filteredDp;
    }
    
    private:
        std::vector<DataPoint> dataPoints;
};

int main()
{
    Data data = Data();
    std::string searchedLocation = "buoy 1659";
    double searchedLowBoundTemp = 25.4;
    auto vec = data.GetDataPoints(searchedLocation, searchedLowBoundTemp);
    std::cout << "Data Points from Location : " << searchedLocation << " above temperature : " << searchedLowBoundTemp << std::endl; 
    for (const auto& item : vec){
        std::cout << "location " << item.location << ", temperature " << item.temperature << std::endl;
    }
    
    return 0;
}

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.