11

My goal:

Create 3 numpy arrays in python (2 of them will be initialized with specific values), then send all three of them through swig into a c++ function as vector references (this is in order to avoid copying over the data and loosing efficiency). Once in the c++ function, add 2 of the arrays and put their sum in the 3rd array.

vec_ref.h

#include <vector>
#include <iostream>

void add_vec_ref(std::vector<int>& dst, std::vector<int>& src1, std::vector<int>& src2);

vec_ref.cpp

#include "vec_ref.h"
#include <cstring> // need for size_t
#include <cassert>

void add_vec_ref(std::vector<int>& dst, std::vector<int>& src1, std::vector<int>& src2) {
    std::cout << "inside add_vec_ref" << std::endl;
    assert(src1.size() == src2.size());
    dst.resize(src1.size());

    for (size_t i = 0; i < src1.size(); i++) {
        dst[i] = src1[i] + src2[i];
    }
}

vec_ref.i

%module vec_ref
%{
    #define SWIG_FILE_WITH_INIT
    #include "vec_ref.h"
%}

%include "numpy.i"
%init %{
import_array();
%}

%include "std_vector.i"
%template(vecInt) std::vector<int>;
// %template(vecIntRef) std::vector<int> &; 

// %apply (std::vector<int> * INPLACE_ARRAY1, int DIM1) {(std::vector<int> * dst, int a),(std::vector<int> * src1, int b),(std::vector<int> * src2, int c)};
// %apply (std::vector<int> * INPLACE_ARRAY1) {(std::vector<int> * dst),(std::vector<int> * src1),(std::vector<int> * src2)};
// %apply (std::vector<int> & INPLACE_ARRAY1) {(std::vector<int> & dst),(std::vector<int> & src1),(std::vector<int> & src2)};
// %apply (std::vector<int> & INPLACE_ARRAY1, int DIM1) {(std::vector<int> & dst, int a),(std::vector<int> & src1, int b),(std::vector<int> & src2, int c)};

%include "vec_ref.h"

Makefile

all:
    rm -f *.so *.o *_wrap.* *.pyc *.gch vec_ref.py
    swig -c++ -python vec_ref.i
    g++ -O0 -g3 -fpic -c vec_ref_wrap.cxx vec_ref.h vec_ref.cpp -I/home/lmckeereid/tools/anaconda3/pkgs/python-3.7.3-h0371630_0/include/python3.7m/
    g++ -O0 -g3 -shared vec_ref_wrap.o vec_ref.o -o _vec_ref.so

tester.py

import vec_ref as vec
import numpy as np

a = np.array([1,2,3], dtype=np.intc)
b = np.array([4,5,6], dtype=np.intc)
c = np.zeros(len(a), dtype=np.intc)

print('---Before---\na:', a)
print('b:', b)
print('c:', c)

vec.add_vec_ref(c,a,b)

print('---After---\na:', a)
print('b:', b)
print('c:', c)

Output:

---Before---
a: [1 2 3]
b: [4 5 6]
c: [0 0 0]
Traceback (most recent call last):
  File "tester.py", line 12, in <module>
    vec.add_vec_ref(c,a,b)
TypeError: in method 'add_vec_ref', argument 1 of type 'std::vector< int,std::allocator< int > > &'

I've tried all the commented out %apply and %template directives found in vec_ref.i, but they didn't work.

Are there some typemaps that I should be including that I'm not?

2

2 Answers 2

4

I agree with @pschill: it’s not possible to get an std::vector without copying data.

One alternative is to use the std::span class template (introduced in C++20), or a similar span class template defined in a library.

Creating a std::span<int> would provide a view of existing data in a numpy array, and provide many convenient member functions (such as operator[], iterators, front(), back(), etc.) in C++.

Creating a span would never copy data from the numpy array.

5
  • Thanks for providing what I see to be the best alternative (aside from building my own class).
    – Otherness
    Nov 18, 2019 at 19:41
  • If I really wanted to use (and modify) a std::vector in my C++ function without copying, what alternatives would I have ? Raw pointer to std::vector ? shared_ptr to std::vector ? Apr 21, 2020 at 16:27
  • @GabrielDevillers, if I understand your question, if a vector exists and you want to modify it in your function, I would recommend using a reference to the vector: std::vector<T>& v
    – NicholasM
    Apr 21, 2020 at 17:40
  • @NicholasM I meant in an API that I want to wrap using SWIG. I'm asking because I understand that SWIG cannot wrap non-const reference to vectors. Apr 22, 2020 at 7:17
  • Oh, sorry. I’d suggest you create a new question that focuses on your specific case.
    – NicholasM
    Apr 22, 2020 at 14:04
-1

You can refer to the Facebook faiss library, which achieves what you want to achieve, in a more elegant way By:

Python-specific: numpy array <-> C++ pointer interface (vector)

You can see the code on its Github page.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.