1

A sensor provides a stream of frames containing object coordinates, which are stored in ProtoBuf format in a gzipped file. I would like to read this file in Julia.

Using protoc, I have generated the Protobuf files for both Python and Julia, coordinate_push.py and coordinate_push.jl

My Python code is as follows:

frameList = []

with gzip.open(filePath) as f:
    data = f.read()
    next_pos, pos = 0, 0

    while pos < len(data):
        msg = coordinate_push.CoordinatesFrame()

        next_pos, pos = _DecodeVarint32(data, pos)
        msg.ParseFromString(data[pos:pos + next_pos])

        frameList.append(msg)

        pos += next_pos

I'd like to rewrite the above in Julia, and don't know where to start. Part of the problem is that I haven't fully understood the Python script (IO is not my strong point).

I understand that I need:

  • to open the gzip file, presumably using using GZip; file = GZip.open(file_path, "r")
  • to read in the data, along the lines of using ProtoBuf; data = readproto(iob, CoordinatesFrame())

What I don't understand is:

  • how to define iob, and especially how to link it to file (in the Julia Protobuf manual, we had iob = PipeBuffer(), but here it's a gzip-file that we'd like to read)
  • how to replicate the while-loop in Julia, and in particular the mysterious _DecodeVarint32 (I'm on Windows, if it's related to that.)
  • whether the file coordinate_push.jl has to be in the same directory as my main file, and if not, how I can properly import it (it is currently in a proto subfolder, and in Python I'd import it using from src.proto import coordinate_push)

Insight on any of the three points would be highly appreciated.

1 Answer 1

1

You should open an issue on the Gzip GitHub repo and ask this first part of your question there (I am not a Gzip expert unfortunately).

On the second point, I suggest looking here: https://github.com/JuliaIO/FileIO.jl/blob/master/README.md for lots of examples of FileIO loops which seems exactly what you need to replicate that Python loop. For the second part of this question, you best bet for that function is to try and hunt down the definition on GitHub or in the docs somewhere.

For the 3rd questions, coordinate_push.jl does not need to be in the same folder as your "main file" (I am not sure what you mean by this so perhaps it would help to add context on the structure of your files). To import that file all you need to do is add include("path/to/coordinate_push.jl") at the top of the file you want to call/run the code from. It's worth noting that the path can either be the absolute path or the relative project path (in some cases).

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.