3

I try to pass vector/array by reference from python through pybind11 to a C++ library. The C++ library may fill in data. After the call to C++, I hope the python side will get the data.

Here is the simplified C++ code:

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include <pybind11/stl.h>

class Setup
{
public:
    Setup(int version) : _version(version) {}
    int _version;
};

class Calculator
{
public:
    Calculator() {}
    static void calc(const Setup& setup, std::vector<double>& results) { ... }
}

namespace py = pybind11;

PYBIND11_MODULE(one_calculator, m) {
    // optional module docstring
    m.doc() = "pybind11 one_calculator plugin";

    py::class_<Setup>(m, "Setup")
        .def(py::init<int>());

    py::class_<Calculator>(m, "Calculator")
        .def(py::init<>())
        .def("calc", &Calculator::calc);
}

On the python side, I intend to:

import os
import sys
import numpy as np
import pandas as pd
sys.path.append(os.path.realpath('...'))
from one_calculator import Setup, Calculator

a_setup = Setup(1)
a_calculator = Calculator()

results = []
a_calculator.calc(a_setup, results)

results

Apparently the results are not passed back. Is there a neat way to do it?

5
  • Did you mean a_calculator.calc(a_setup, results)? Commented Nov 17, 2021 at 22:25
  • Yes, you are right. I have corrected it. Thank you!
    – xycs
    Commented Nov 17, 2021 at 22:31
  • 1
    I don't know much about pybind, but I have experience with both C++ and Python. You realise that std::vector is not python's [], right? While pybind perhaps can convert one to another, they are completely different objects so I don't see how pass-by-reference is even remotely possible.
    – ALX23z
    Commented Nov 17, 2021 at 22:57
  • Thank you ALX23z. I am flexible with the data type python to pass to the C++ side.
    – xycs
    Commented Nov 18, 2021 at 14:06
  • You can use numpy arrays and pass them by reference.
    – unddoch
    Commented Nov 21, 2021 at 9:41

1 Answer 1

3

Figured out a way:

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include <pybind11/stl.h>

#include "Calculator.h" // where run_calculator is

namespace py = pybind11;

// wrap c++ function with Numpy array IO
int wrapper(const std::string& input_file, py::array_t<double>& in_results) {
    if (in_results.ndim() != 2)
        throw std::runtime_error("Results should be a 2-D Numpy array");

    auto buf = in_results.request();
    double* ptr = (double*)buf.ptr;

    size_t N = in_results.shape()[0];
    size_t M = in_results.shape()[1];

    std::vector<std::vector<double> > results;

    run_calculator(input_file, results);

    size_t pos = 0;
    for (size_t i = 0; i < results.size(); i++) {
        const std::vector<double>& line_data = results[i];
        for (size_t j = 0; j < line_data.size(); j++) {
            ptr[pos] = line_data[j];
            pos++;
        }
    }
}

PYBIND11_MODULE(calculator, m) {
    // optional module docstring
    m.doc() = "pybind11 calculator plugin";

    m.def("run_calculator", &wrapper, "Run the calculator");
}

Python side

results= np.zeros((N, M))
run_calculator(input_file, results)

This way I also do not expose classes Setup and Calculator to the python side.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.