10

I've been trying to make a scene with scene kit in which an specified object is always in front of others, despite the fact that it's actually behind the other objects. A similar effect to this used in blender.

Apparently, blender uses GUI and a lot of math to transform otherwise 2D objects, but I need this effect in a SCNNode with a SCNGeometry, in other words, a 3D object currently locate in the scene.

I considered using category masks, but after reading Apple's documentation I've realized that doesn't work for the effect I'm looking for.

Does anyone know a way of doin this in SceneKit? Or better yet, is it even possible to do this?

Thank you all so much in advance, for now and all other help I've got from StackExchange!

6 Answers 6

6

As explained in my previous answer, the accepted answer isn't optimal and works properly only for billboards, huds and other generally flat objects (not necessarily entirely 2D). When using 3D objects and disabling reading from the depth buffer and objects like the one in the image in above, it won't render correctly from every angle. I.e a 3D object needs to read from the depth buffer to detect its own pixels and depth. That all said, I present the correct answer:

SCNTechnique

In short, render 2 additional passes. One for the control gizmo (DRAW_NODE), and one to mix it together with the scene by another pass (DRAW_QUAD, with a shader that uses the previous passes as inputs).

Following is the techique's scntec.plist contents:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>passes</key>
    <dict>
        <key>gizmoonly</key>
        <dict>
            <key>colorStates</key>
            <dict>
                <key>clear</key>
                <true/>
                <key>clearColor</key>
                <string>0.5 0.5 0.5 0.0</string>
            </dict>
            <key>depthStates</key>
            <dict>
                <key>clear</key>
                <true/>
            </dict>
            <key>inputs</key>
            <dict>
                <key>colorSampler</key>
                <string>COLOR</string>
            </dict>
            <key>outputs</key>
            <dict>
                <key>color</key>
                <string>gizmonode</string>
            </dict>
            <key>draw</key>
            <string>DRAW_NODE</string>
            <key>node</key>
            <string>movegizmo</string>
        </dict>
        <key>quadscene</key>
        <dict>
            <key>colorStates</key>
            <dict>
                <key>clear</key>
                <true/>
                <key>clearColor</key>
                <string>sceneBackground</string>
            </dict>
            <key>depthStates</key>
            <dict>
                <key>clear</key>
                <true/>
            </dict>
            <key>inputs</key>
            <dict>
                <key>totalSceneO</key>
                <string>COLOR</string>
                <key>a_texcoord</key>
                <string>a_texcoord-symbol</string>
                <key>gizmoNodeO</key>
                <string>gizmonode</string>
            </dict>
            <key>outputs</key>
            <dict>
                <key>color</key>
                <string>COLOR</string>
            </dict>
            <key>draw</key>
            <string>DRAW_QUAD</string>
            <key>program</key>
            <string>gizmo</string>
        </dict>
    </dict>
    <key>sequence</key>
    <array>
        <string>gizmoonly</string>
        <string>quadscene</string>
    </array>
    <key>targets</key>
    <dict>
        <key>totalscene</key>
        <dict>
            <key>type</key>
            <string>color</string>
        </dict>
        <key>gizmonode</key>
        <dict>
            <key>type</key>
            <string>color</string>
        </dict>
    </dict>
    <key>symbols</key>
    <dict>
        <key>a_texcoord-symbol</key>
        <dict>
            <key>semantic</key>
            <string>texcoord</string>
        </dict>
        <key>vertexSymbol</key>
        <dict>
            <key>semantic</key>
            <string>vertex</string>
        </dict>
    </dict>
</dict>
</plist>

Following is the vertex shader for the second pass:

attribute vec4 a_position;
varying vec2 uv;

void main() {
    gl_Position = a_position;
    uv = (a_position.xy + 1.0) * 0.5;
}

The fragment shader for the second pass:

uniform sampler2D totalSceneO;
uniform sampler2D gizmoNodeO;

varying vec2 uv;

void main() {
    vec4 t0 = texture2D(totalSceneO, uv);
    vec4 t1 = texture2D(gizmoNodeO, uv);
    gl_FragColor = (1.0 - t1.a) * t0 + t1.a * t1;
}

Swift code:

if let path = NSBundle.mainBundle().pathForResource("scntec", ofType: "plist") {
            if let dico1 = NSDictionary(contentsOfFile: path)  {
                let dico = dico1 as! [String : AnyObject]

                let technique = SCNTechnique(dictionary:dico)
                scnView.technique = technique
            }
}

Objective-C code:

NSURL *url = [[NSBundle mainBundle] URLForResource:@"scntec" withExtension:@"plist"];
SCNTechnique *technique = [SCNTechnique techniqueWithDictionary:[NSDictionary dictionaryWithContentsOfURL:url]];
    self.myView.technique = technique;

Set the name for the gizmo node:

theGizmo.name = @"movegizmo";
1
  • I know this is quite old. But can you tell me how do you use the 2 shader. Are they in 2 different files? How and when are the loaded ? Can you please help? Aug 16, 2021 at 6:05
6

So, turns out I found the answer to my question (with the help of mnuages). I just wanted to post a complete answer.

Final result
Like suggested by mnuages, I tried setting both readsFromDepthBuffer and writesToDepthBuffer to false and setting the node's renderingOrder to a high number. It worked the wrong way. Instead of being always in front, it was always behind every object. The way to get the result shown in the picture is to set only readsFromDepthBuffer to false, and set the cube's renderingOrder to -1, or else it will be impossible to draw over it.

Since the cube and the other nodes' materials have readsFromDepthBuffer and writesToDepthBuffer set to the default value of true, it will still be in front of the objects behind it, and behind the objects in front of it, in other words, it will be normal, only the arrows will behave the way we want.

As can be seen in the image, the portion of the lines in front of the cube are visible. The same can't be said for portion behind it.

Edit: This only worked for me because in my case the camera did not have to rotate around the object. It will not work if you need the gizmo to be always in front regardless of the camera angle. Check the accepted answer for an explanation on how to achieve that, additionally, simeon's answer for a Metal implementation.

6
  • That fits with Apple's documentation for writesToDepthBuffer and readsFromDepthBuffer. The main use case for writesToDepthBuffer seems to be for handling transparency. I've found that an object with transparency can cause objects behind it to vanish from some angles unless writesToDepthBuffer is set to false for the transparent object's SCNMaterial. Even then I've seen a few glitches, but this definitely helped.
    – Dave Ruske
    Jun 10, 2015 at 17:42
  • 1
    It's perhaps wandering off topic a little to add to my previous comment, but it turns out that setting renderingOrder to -1 on my transparent object fixed remaining render issues.
    – Dave Ruske
    Jun 11, 2015 at 0:48
  • That's a little senseless to me, I mean, the documentation suggests that simply setting an object's renderingOrder to a bigger number and combining it with the proper depth buffer setting would work, but apparently an object with its material's readFromDepthBuffer property set to false will only be in front of other objects if said objects have a negative renderingOrder. Jun 11, 2015 at 0:54
  • Keep in mind (as noted by Xartec) that this technique will only work in cases similar to mine in which the scene will always be visible from the same angle only. Apr 14, 2018 at 12:35
  • Thanks for this. I was using billboard constraint, and in my case, I had to set "readsFromDepthBuffer" false to all nodes, and set rendering order to be descending order: 0 for the first one to draw, and subsequent ones were -1, -2, -3, etc. May 11, 2018 at 17:50
4

@Xartec's answer is the way to go. You basically render the gizmo alone, on a transparent background, then use a simple shader to mix it with the scene. Here's the same setup for Metal

Gizmo.metal

#include <metal_stdlib>
using namespace metal;

#include <SceneKit/scn_metal>

struct custom_vertex_t
{
    float4 position [[attribute(SCNVertexSemanticPosition)]];
    float4 normal [[attribute(SCNVertexSemanticNormal)]];
};

struct out_vertex_t
{
    float4 position [[position]];
    float2 uv;
};

vertex out_vertex_t gizmo_vertex(custom_vertex_t in [[stage_in]])
{
    out_vertex_t out;
    out.position = in.position;
    out.uv = float2((in.position.x + 1.0) * 0.5 , (in.position.y + 1.0) * -0.5);

    return out;
};

constexpr sampler s = sampler(coord::normalized,
                              r_address::clamp_to_edge,
                              t_address::repeat,
                              filter::linear);

fragment half4 gizmo_fragment(out_vertex_t vert [[stage_in]],
                              texture2d<float, access::sample> totalSampler [[texture(0)]],
                              texture2d<float, access::sample> gizmoSampler [[texture(1)]])
{

    float4 t0 = totalSampler.sample(s, vert.uv);
    float4 t1 = gizmoSampler.sample(s, vert.uv);

    return half4((1.0 - t1.a) * t0 + t1.a * t1);
}

GizmoTechnique.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>passes</key>
    <dict>
        <key>pass_scene</key>
        <dict>
            <key>draw</key>
            <string>DRAW_SCENE</string>
            <key>inputs</key>
            <dict/>
            <key>outputs</key>
            <dict>
                <key>color</key>
                <string>color_scene</string>
            </dict>
            <key>colorStates</key>
            <dict>
                <key>clear</key>
                <true/>
                <key>clearColor</key>
                <string>sceneBackground</string>
            </dict>
        </dict>
        <key>pass_gizmo</key>
        <dict>
            <key>colorStates</key>
            <dict>
                <key>clear</key>
                <true/>
                <key>clearColor</key>
                <string>0 0 0 0</string>
            </dict>
            <key>depthStates</key>
            <dict>
                <key>clear</key>
                <true/>
            </dict>
            <key>outputs</key>
            <dict>
                <key>color</key>
                <string>color_gizmo</string>
            </dict>
            <key>inputs</key>
            <dict/>
            <key>draw</key>
            <string>DRAW_NODE</string>
            <key>node</key>
            <string>transformGizmo</string>
        </dict>
        <key>mix</key>
        <dict>
            <key>colorStates</key>
            <dict>
                <key>clear</key>
                <true/>
            </dict>
            <key>depthStates</key>
            <dict>
                <key>clear</key>
                <true/>
            </dict>
            <key>inputs</key>
            <dict>
                <key>totalSampler</key>
                <string>COLOR</string>
                <key>gizmoSampler</key>
                <string>color_gizmo</string>
            </dict>
            <key>outputs</key>
            <dict>
                <key>color</key>
                <string>COLOR</string>
            </dict>
            <key>draw</key>
            <string>DRAW_QUAD</string>
            <key>program</key>
            <string>doesntexist</string>
            <key>metalFragmentShader</key>
            <string>gizmo_fragment</string>
            <key>metalVertexShader</key>
            <string>gizmo_vertex</string>
        </dict>
    </dict>
    <key>sequence</key>
    <array>
        <string>pass_gizmo</string>
        <string>mix</string>
    </array>
    <key>targets</key>
    <dict>
        <key>color_gizmo</key>
        <dict>
            <key>type</key>
            <string>color</string>
        </dict>
    </dict>
    <key>symbols</key>
    <dict>
        <key>vertexSymbol</key>
        <dict>
            <key>semantic</key>
            <string>vertex</string>
        </dict>
    </dict>
</dict>
</plist>
2
  • 2
    Your answer seems simpler, but it is derived from Xartec's original answer. So I'm accepting his, and upvoting yours. Feb 5, 2020 at 22:20
  • Works like a charm. Since I'm using this inside of a framework, I had to add this line of code in order to correctly load the Metal library: scnView.technique!.library = try! view.device!.makeDefaultLibrary(bundle: frameworkBundle).
    – Nickkk
    Aug 8, 2021 at 20:42
2

The readsFromDepthBuffer answer may work for the angle in the the image shown but if you were to rotate and look at different angles you will see the gizmo's axis overlap incorrectly from some angles. In other words some axis will be behind another when it should be in front of it, depending on which order the faces are rendered. This is obviously because the object (material) doesnt read the depth buffer, including for parts it already rendered of itself.

In general an easy solution to render a SCNNode in front of the main scene is to use an SK3DNode (with its own scene and depthbuffer) in the scene's overlaySKScene. For the scenario shown in the picture in the answer this would require additional code to rotate and position the SK3DNode accurately so it's probably not what the OP was looking for but this may be of help to others who find this question.

1

SCNMaterial exposes writesToDepthBuffer and readsFromDepthBuffer that will let you do that (in combination with SCNNode's renderingOrder if necessary)

1
  • 1
    I've finally managed to check out what you said. In fact, setting the options you said from it's default True value to False, seems to be the solution, as I've managed to get my obect to be drawn always in front of some others, but not all. I have yet to figure out how to work the rendering order, because there is something I am doing wrong. But thank you :) Jun 9, 2015 at 19:21
1

I am not sure if it is of any help, but I happily just found out, that the method Apple described in their documentation actually works out nicely. I have an object which is supposed to be shown permanently and in front of all the others. Setting its rendering order (node inspector) to higher values (yet all the same for said nodes) and unchecking the ‚reads depth‘ checkbox (Material inspector) for all used materials leads to the desired result. The rendering order of all other objects is kept at zero, no negative order applied. I understood you need this in code, but maybe you can make something out of this?

1
  • Thank you! But five years ago when I did try that it didn't work. I found an unideal solution which didn't quite work if the camera rotated, but back then I didn't need the camera to rotate around it. These days I can't even verify your suggestion since I no longer own a Mac. I'll upvote you answer and hope it can help others. Mar 3, 2020 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.