2

I was trying to process the live camera feed from flutter so I needed to send the byte data to nativve Android for processing.

I was concatenating the 3 planes as suggested by flutterfire. I needed the data to create an InputImage for the google ml kit.

Concatenate Plane method

static Uint8List _concatenatePlanes(List<Plane> planes) {
    final WriteBuffer allBytes = WriteBuffer();
    for (Plane plane in planes) {
      allBytes.putUint8List(plane.bytes);
    }
    return allBytes.done().buffer.asUint8List();
  }

Input Image created from the image data in native Android

public void fromByteBuffer(Map<String, Object> imageData, final MethodChannel.Result result) {
  byte[] bytes = (byte[]) imageData.get("bytes");

  int rotationCompensation = ((int) imageData.get("rotation")) % 360;

//Create an input image
  InputImage inputImage = InputImage.fromByteArray(bytes,
                (int) imageData.get("width"),
                (int) imageData.get("height"),
                 rotationCompensation,
                InputImage.IMAGE_FORMAT_NV21);
}

I am not getting any results after processing a frame from camera stream, but if the same frame is captured, stored and then processed by creating the Input Image from file path the image is being processed properly.

Is there anything wrong in the way I am creating the input image. Any help is appreciated.

7
  • The camera streaming use case has been working fine with the native code on Android. I am wondering it is related to how the rotationDegree is calculated. Could you refer to this[1] link to see if you can do the same for computing a correct rotationDegree? [1]github.com/googlesamples/mlkit/blob/… Mar 23, 2021 at 19:17
  • Thank you @ChenxiSong I'll try this and let you know the results. Mar 25, 2021 at 8:36
  • Hi, the ML Kit InputImage you are using does support android.media.Image as an input. developers.google.com/ml-kit/vision/pose-detection/…. When building your flutter plugin, could you pass that directly through the method channel? Mar 31, 2021 at 16:11
  • Hi @ChenxiSong Yeah I am already using that method and it is working fine. But we also need to process the camera stream data to get live information. I tried your above suggestion for live detection unfortunately it did not give any results. If you come up with any other solution let me know. Thanks Mar 31, 2021 at 16:23
  • I see. Since the android native sdk has been working fine. I guess it is related to how flutter works. Sorry for not being able to help further. Apr 2, 2021 at 17:18

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy