1

I stumbled upon a strange problem in vuforia.When i request a camera image using CameraDevice.GetCameraImage(mypixelformat), the image returned is both flipped sideways and rotated 180 deg. Because of this, to obtain a normal image i have to first rotate the image and then flip it sideways.The approach i am using is simply iterating over pixels of the image and modifying them.This approach is very poor performance wise.Below is the code:

    Texture2D image;

    CameraDevice cameraDevice = Vuforia.CameraDevice.Instance;
    Vuforia.Image vufImage = cameraDevice.GetCameraImage(pixelFormat);
    image = new Texture2D(vufImage.Width, vufImage.Height);
    vufImage.CopyToTexture(image);

    Color32[] colors = image.GetPixels32();
    System.Array.Reverse(colors, 0, colors.Length);  //rotate 180deg
    image.SetPixels32(colors);                       //apply rotation
    image = FlipTexture(image);                      //flip sideways






                 //***** THE FLIP TEXTURE METHOD *******//

private Texture2D FlipTexture(Texture2D original, bool upSideDown = false)
{

    Texture2D flipped = new Texture2D(original.width, original.height);

    int width  = original.width;
    int height = original.height;


    for (int col = 0; col < width; col++)
    {
        for (int row = 0; row < height; row++)
        {
            if (upSideDown)
            {
                flipped.SetPixel(row, (width - 1) - col, original.GetPixel(row, col));
            }
            else
            {
                flipped.SetPixel((width - 1) - col, row, original.GetPixel(col, row));
            }
        }
    }

    flipped.Apply();

    return flipped;
}

To improve the performance i want to somehow schedule these pixel operations on the GPU, i have heard that a compute shader can be used, but i have no idea where to start.Can someone please help me write the same operations in a compute shader so that the GPU can handle them, Thankyou!.

5
  • The VuforiaConfiguration has an option to flip the camera image horizontally. might that option solve your problem?
    – Senbazuru
    Aug 7, 2018 at 12:53
  • That only flips the image when Vuforia play is enabled(In unity editor only) Aug 7, 2018 at 13:37
  • And btw it doesn't work!. Aug 7, 2018 at 13:42
  • May I ask how you want to use the Texture2D, and how to apply it?
    – Senbazuru
    Aug 8, 2018 at 6:34
  • I don't want to apply the texture onto anything, otherwise i would've just change the u v's and cancelled the flipping and rotation effect.I have a function that crops the part of the Texture2D passed to it, which comes under a RectTransform (Region of interest).Something similar to how the windows snipping tool is, except that my function crops from a texture passed to it Aug 8, 2018 at 11:59

1 Answer 1

5

The whole compute shader are new for me too, but i took the occasion to research it a little bit for myself too. The following works for flipping a texture vertically (rotating and flipping horizontally should be just a vertical flip). Someone might have a more elaborate solution for you, but maybe this is enough to get you started.

The Compute shader code:

 #pragma kernel CSMain
// Create a RenderTexture with enableRandomWrite flag and set it
// with cs.SetTexture
RWTexture2D<float4> Result;
Texture2D<float4> ImageInput;
float2 flip;
[numthreads(8,8,1)]
void CSMain (uint3 id : SV_DispatchThreadID)
{
    flip = float2(512 , 1024) - id.xy ;
    Result[id.xy] = float4(ImageInput[flip].x, ImageInput[flip].y, ImageInput[flip].z, 1.0);
}

and called from any script:

public void FlipImage()
{
    int kernelHandle = shader.FindKernel("CSMain");
    RenderTexture tex = new RenderTexture(512, 1024, 24);
    tex.enableRandomWrite = true;
    tex.Create();

    shader.SetTexture(kernelHandle, "Result", tex);
    shader.SetTexture(kernelHandle, "ImageInput", myTexture);
    shader.Dispatch(kernelHandle, 512/8 , 1024 / 8, 1);

    RenderTexture.active = tex;
    result.ReadPixels(new Rect(0, 0, tex.width, tex.height), 0, 0);
    result.Apply();
}

This takes an input Texture2D, flips it in the shader, applies it to a RenderTexture and to a Texture2D, whatever you need. Note that the image sizes are hardcoded in my instance and should be replaced by whatever size you need. (for within the shader use shader.SetInt(); )

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.