8

I need to wrap a dynamically allocated array(from a = new double[100] for example) into std::vector(preferably) without copying the array. This restriction is imposed by that the array I want to wrap is mmaped from a file, so just doing vector(a, a+size) will double the memory usage.

Is any tricks to do that?

1
  • 1
    Anyway you will not get entire set of capabilities of std::vector (e.g. resize, likely will not work), so may be it would be good to list what exactly you need from wrapper?
    – maxim1000
    Nov 11, 2009 at 9:40

8 Answers 8

11

One of the best solutions for this is something like STLSoft's array_proxy<> template. Unfortunately, the doc page generated from the source code by doxygen isn't a whole lot of help understanding the template. The source code might actually be a bit better:

The array_proxy<> template is described nicely in Matthew Wilson's book, Imperfect C++. The version I've used is a cut-down version of what's on the STLSoft site so I didn't have to pull in the whole library. My version's not as portable, but that makes it much simpler than what's on STLSoft (which jumps through a whole lot of portability hoops).

If you set up a variable like so:

int myArray[100];

array_proxy<int> myArrayProx( myArray);

The variable myArrayProx has many of the STL interfaces - begin(), end(), size(), iterators, etc.

So in many ways, the array_proxy<> object behaves just like a vector (though push_back() isn't there since the array_proxy<> can't grow - it doesn't manage the array's memory, it just wraps it in something a little closer to a vector).

One really nice thing with array_proxy<> is that if you use them as function parameter types, the function can determine the size of the array passed in, which isn't true of native arrays. And the size of the wrapped array isn't part of the template's type, so it's quite flexible to use.

1
  • 1
    +1, seems to be best solution, especially because proxy should not reallocate memory so it should be free from many problems. Nov 11, 2009 at 8:47
9

A boost::iterator_range provides a container-like interface:

// Memory map an array of doubles:
size_t number_of_doubles_to_map = 100;
double* from_mmap = mmap_n_doubles(number_of_doubles_to_map);

// Wrap that in an iterator_range
typedef boost::iterator_range<double*> MappedDoubles;
MappedDoubles mapped(from_mmap, from_mmap + number_of_doubles_to_map);

// Use the range
MappedDoubles::iterator b = mapped.begin();
MappedDoubles::iterator e = mapped.end();
mapped[0] = 1.1;
double first = mapped(0);

if (mapped.empty()){
    std::cout << "empty";
}
else{
    std::cout << "We have " << mapped.size() << "elements. Here they are:\n"
       << mapped;
}
1
  • 1
    Best answer for me, as it uses boost!
    – CharlesB
    Jun 29, 2011 at 13:06
2

I was once determined to accomplish the exact same thing. After a few days of thinking and trying I decided it wasn't worth it. I ended up creating my own custom vector that behaved like std::vector's but only had the functionality I actually needed like bound checking, iterators etc.

If you still desire to use std::vector, the only way I could think of back then was to create a custom allocator. I've never written one but seeing as this is the only way to control STL's memory management maybe there is something that can be done there.

2
  • 2
    I suggest not writing an own container. It's most of the time simply not worth it and leads to quite a few interoperatibility issues. Additionally, time spent on debugging reinvented stuff is wasted time. The custom allocator however is probably quite easy to implement and a better approach.
    – MP24
    Nov 11, 2009 at 9:23
  • Normally I would agree, but I really think it depends what you're going to do with it. Most of the "complex" operations in std::vector probably have something to do with memory management and it happens that in this case none of them are required. So writing a small class with bounds checking and a pointer as an iterator shouldn't take too long. I can't comment on interoperatbility since I have no knowledge of where/how the author intends to use it. But iterator support should make it work with STL's algorithms.
    – Idan K
    Nov 11, 2009 at 9:39
1

No, that is not possible using a std::vector.

But if possible you can create the vector with this size, and possible map the file to that instead.

std::vector<double> v(100);
mmapfile_double(&v[0], 100);
1

What about vector of pointers that point to your mapped area elements (reduced memory consumption as sizeof(double*) < sizeof(double))? Is this OK for you?

There is some drawbacks (primary is you need special predicates for sort) but some benefits too as you can, for example, delete elements without changing actual mapped content (or have even number of such arrays with different order of elements without any change to actual values).

There is common problem of all the solutions with std::vector on mapped file: to 'nail' vector content to mapped area. This can't be tracked, you can only watch after yourself to not use something which could lead to vector content re-allocation. So be careful in any case.

1

You could go with array_proxy<>, or take a look at Boost.Array . It gives you size(), front(), back(), at(), operator[], etc. Personally, I'd prefer Boost.Array since Boost is more prevalent anyway.

0
1

well, the vector template allows to provide your own memory allocator. I never did it myself but I guess it is not that difficult to get it to point to your array, maybe with placement new operator... just a guess, I write more if I try and succeed.

0

Here's the solution to your question. I had been attempting this off and on for quite some time before I came up with a workable solution. The caveat is that you have got to zero out the pointers after use in order to avoid double-freeing the memory.

#include <vector>
#include <iostream>

template <class T>
void wrapArrayInVector( T *sourceArray, size_t arraySize, std::vector<T, std::allocator<T> > &targetVector ) {
  typename std::_Vector_base<T, std::allocator<T> >::_Vector_impl *vectorPtr =
    (typename std::_Vector_base<T, std::allocator<T> >::_Vector_impl *)((void *) &targetVector);
  vectorPtr->_M_start = sourceArray;
  vectorPtr->_M_finish = vectorPtr->_M_end_of_storage = vectorPtr->_M_start + arraySize;
}

template <class T>
void releaseVectorWrapper( std::vector<T, std::allocator<T> > &targetVector ) {
  typename std::_Vector_base<T, std::allocator<T> >::_Vector_impl *vectorPtr =
        (typename std::_Vector_base<T, std::allocator<T> >::_Vector_impl *)((void *) &targetVector);
  vectorPtr->_M_start = vectorPtr->_M_finish = vectorPtr->_M_end_of_storage = NULL;
}

int main() {

  int tests[6] = { 1, 2, 3, 6, 5, 4 };
  std::vector<int> targetVector;
  wrapArrayInVector( tests, 6, targetVector);

  std::cout << std::hex << &tests[0] << ": " << std::dec
            << tests[1] << " " << tests[3] << " " << tests[5] << std::endl;

  std::cout << std::hex << &targetVector[0] << ": " << std::dec
            << targetVector[1] << " " << targetVector[3] << " " << targetVector[5] << std::endl;

  releaseVectorWrapper( targetVector );
}

Alternatively you could just make a class that inherits from vector and nulls out the pointers upon destruction:

template <class T>
class vectorWrapper : public std::vector<T>
{   
public:
  vectorWrapper() {
    this->_M_impl _M_start = this->_M_impl _M_finish = this->_M_impl _M_end_of_storage = NULL;
  }   

  vectorWrapper(T* sourceArray, int arraySize)
  {   
    this->_M_impl _M_start = sourceArray;
    this->_M_impl _M_finish = this->_M_impl _M_end_of_storage = sourceArray + arraySize;
  }   

  ~vectorWrapper() {
    this->_M_impl _M_start = this->_M_impl _M_finish = this->_M_impl _M_end_of_storage = NULL;
  }   

  void wrapArray(T* sourceArray, int arraySize)
  {   
    this->_M_impl _M_start = sourceArray;
    this->_M_impl _M_finish = this->_M_impl _M_end_of_storage = sourceArray + arraySize;
  }   
};  

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.