3

Edit6:

So, I believe this is a threading problem. Some kind of concurrency or race condition that happens arbitrarily. My current setup creates a variable, let image = self.videoImage and somewhere between threads either image or self.videoImage is freeing memory somewhere and other threads access is causing issues. The goal is to utilize a different technique will deploying threads and ensure data is prepared.

I am rendering a CIImage using

if(ciImage == nil) { print("CIImage nil") }
if(currentDrawable == nil) { print("Draw nil") }
if(currentDrawable.texture == nil) { print("Text nil") }
if(commandBuffer == nil) { print("CB nil") }
if(ciImage.extent == nil) { print("Extent nil") }
if(ciContext == nil) { print("Context nil") }
if(colorSpace == nil) { print("CS nil") }
ciContext.render(ciImage, to: currentDrawable.texture, commandBuffer: commandBuffer, bounds: ciImage.extent, colorSpace: colorSpace)

I am printing out nil because I was trying to determine the cause of my EXC_BAD_ACCESS error which I believe generally stems from accessing bad memory, or a variable that is not there anymore. Yet, I am still getting the error without obtaining a nil message. Why would I be getting a EXC_BAD_ACCESS error during a CIContext.render() call?

excbadaccess

Edit: Here is the entire method. I showed the picture to illustrate where the EXC_BAD_ACCESS is. For @Shauket's comment, if the CIImage was nil prior, it would have given an EXC_BAD_ACCESS prior to the call.

Note: This happens what seems randomly; obviously there is a reason, but it is very sporadic. It could happen the first time or the millionth time I call this method.

    func update(with ciImage: CIImage) {
        // Wait to ensure only kMaxBuffersInFlight are getting proccessed by any stage in the Metal
        // pipeline (App, Metal, Drivers, GPU, etc)
        let _ = inFlightSemaphore.wait(timeout: .distantFuture)
        guard
            let commandBuffer = commandQueue.makeCommandBuffer(),
            let currentDrawable = renderDestination.currentDrawable
            else {
                inFlightSemaphore.signal()
                return
        }
        commandBuffer.label = "MyCommand"

        commandBuffer.addCompletedHandler{ [weak self] commandBuffer in
            if let strongSelf = self {
                strongSelf.inFlightSemaphore.signal()
            }
        }
        //Won't ever get here if simulator because I also check this inside RealTimeDepthViewController but need to check so the pod will compile
        #if targetEnvironment(simulator)
        #else
        if(ciImage == nil) { print("CIImage nil") }
        if(currentDrawable == nil) { print("Draw nil") }
        if(currentDrawable.texture == nil) { print("Text nil") }
        if(commandBuffer == nil) { print("CB nil") }
        if(ciImage.extent == nil) { print("Extent nil") }
        if(ciContext == nil) { print("Context nil") }
        if(colorSpace == nil) { print("CS nil") }
        ciContext.render(ciImage, to: currentDrawable.texture, commandBuffer: commandBuffer, bounds: ciImage.extent, colorSpace: colorSpace)

        commandBuffer.present(currentDrawable)
        commandBuffer.commit()
        #endif
    }

Additionally, this is the backtrace.

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x13b00021d)
    frame #0: 0x0000000196fce0a0 CoreImage`CI::ColorKernelImage::child_count() const + 100
    frame #1: 0x0000000196fff704 CoreImage`CI::GraphObject::traverse_unique(CI::GraphObject*, void (CI::GraphObject*, CI::GraphObject*, int, int, int) block_pointer) + 244
    frame #2: 0x0000000197003b30 CoreImage`CI::prepare_initial_graph(CI::Context*, char const*, CI::Image*, CI::RenderDestination const*, CGRect, CGColorSpace*, CI::PixelFormat, CI::swizzle_info, CI::Affine const&, bool, CI::TextureDescriptor*) + 300
    frame #3: 0x0000000197005750 CoreImage`CI::image_render(char const*, CI::Context*, CI::Image*, CI::RenderDestination const*) + 196
    frame #4: 0x00000001970031a0 CoreImage`CI::RenderToTexture<CI::Texture>::render_to_texture(CI::Image*, CI::Context*, CGSize const&, CI::Texture) const + 88
    frame #5: 0x00000001970034d8 CoreImage`CI::RenderToMTLTexture::render(CI::Image*, CI::Context*) const + 236
    frame #6: 0x0000000196f0f9d4 CoreImage`-[CIRenderDestination _render:withContext:] + 72
    frame #7: 0x0000000196f1354c CoreImage`-[CIContext(CIRenderDestination) _startTaskToRender:toDestination:forPrepareRender:forClear:error:] + 2824
    frame #8: 0x0000000196f12a28 CoreImage`-[CIContext(CIRenderDestination) startTaskToRender:fromRect:toDestination:atPoint:error:] + 144
    frame #9: 0x0000000196e2b398 CoreImage`-[CIContext render:toMTLTexture:commandBuffer:bounds:colorSpace:] + 472
  * frame #10: 0x0000000104b76024 DepthCamera`MetalRenderer.update(ciImage=0x000000010994d810, self=<unavailable>) at MetalRenderer.swift:81:19 [opt]
    frame #11: 0x0000000104b8ea2c DepthCamera`specialized RealtimeDepthMaskViewController.draw(self=0x0000000109c17c80) at RealTimeDepthViewController.swift:543:26 [opt]
    frame #12: 0x0000000104b843d0 DepthCamera`merged @objc DepthCamera.RealtimeDepthMaskViewController.buttonClicked(sender: __C.UIButton) -> () + 60
    frame #13: 0x00000001af4d2380 MetalKit`-[MTKView draw] + 184
    frame #14: 0x00000001af4ce8d0 MetalKit`-[MTKViewDisplayLinkTarget draw] + 40
    frame #15: 0x000000010524c208 GPUToolsCore`-[DYDisplayLinkInterposer forwardDisplayLinkCallback:] + 172
    frame #16: 0x000000019c051f24 QuartzCore`CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 628
    frame #17: 0x000000019c120608 QuartzCore`display_timer_callback(__CFMachPort*, void*, long, void*) + 268
    frame #18: 0x000000019551ddac CoreFoundation`__CFMachPortPerform + 176
    frame #19: 0x00000001955487c4 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 60
    frame #20: 0x0000000195547e90 CoreFoundation`__CFRunLoopDoSource1 + 448
    frame #21: 0x0000000195542ac8 CoreFoundation`__CFRunLoopRun + 2144
    frame #22: 0x0000000195541f40 CoreFoundation`CFRunLoopRunSpecific + 480
    frame #23: 0x000000019f7d2534 GraphicsServices`GSEventRunModal + 108
    frame #24: 0x00000001996cd580 UIKitCore`UIApplicationMain + 1940
    frame #25: 0x0000000102a78d38 ARMail`main at AppDelegate.swift:13:7 [opt]
    frame #26: 0x00000001953c0e18 libdyld.dylib`start + 4

Edit2:

I have now figured out that the issue is currentDrawable. The below image illustrates that the variable currentDrawable is not nil right before the cicontext.render but it IS nil (address 0x0) at the time of the call.

Documentation states

If all drawable objects are in use, the value of this property is nil. Your app should check that currentDrawable isn’t nil before attempting to draw. The view changes the value of this property only after returning from a drawing function, either draw(_:) from a subclassed instance of the view, or draw(in:) from the view’s delegate. In my case, this method is only called within my draw method.

extension RealtimeDepthMaskViewController: MTKViewDelegate {
    public func draw(in view: MTKView) {
        if(renderer.checkRenderCurrentDrawable()) { 
            renderer.update(with: image)
            if(self.finalImage != nil) {
                self.finalImage = image
            }
        }
        else {
            print("Draw nil")
        }
    }
}

func checkRenderCurrentDrawable() -> Bool {
    return self.renderDestination.currentDrawable != nil
}

So, I check if it is nil, it is not at the time, then at the time of cicontext.render it becomes nil...

Edit3:

Still looking into it, but obtained this on the last test...

020-03-12 02:20:51.289750-0400 ARMail[6169:1212379] Compiler failed to build request
2020-03-12 02:20:51.289963-0400 ARMail[6169:1212379] Metal library creation failed: Error Domain=MTLLibraryErrorDomain Code=3 "Function _blendWithMaskB0 argument  marked as info type Output for function composition has different type <4 x float> than the source function _ci_early_out return type i1" UserInfo={NSLocalizedDescription=Function _blendWithMaskB0 argument  marked as info type Output for function composition has different type <4 x float> than the source function _ci_early_out return type

Edit4:

Similar backtrace below; after much testing, it seems that sometimes self is the bad address. Or currentDrawable.texture is the bad address. Or ciImage is the bad address. However, in the end all be all, more than the rest, ciImage is the culprit which provides the bad address. Below the backtrace is a snippet indicating that the address for ciImage cannot be obtained, which seems like it is the bad address. However, I'm still not sure why, because ciImage is not nil 0.125 secs before when I print out ciImage.

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa9601760910)
    frame #0: 0x0000000196fce0a0 CoreImage`CI::ColorKernelImage::child_count() const + 100
    frame #1: 0x0000000196fff704 CoreImage`CI::GraphObject::traverse_unique(CI::GraphObject*, void (CI::GraphObject*, CI::GraphObject*, int, int, int) block_pointer) + 244
    frame #2: 0x0000000197003b30 CoreImage`CI::prepare_initial_graph(CI::Context*, char const*, CI::Image*, CI::RenderDestination const*, CGRect, CGColorSpace*, CI::PixelFormat, CI::swizzle_info, CI::Affine const&, bool, CI::TextureDescriptor*) + 300
    frame #3: 0x0000000197005750 CoreImage`CI::image_render(char const*, CI::Context*, CI::Image*, CI::RenderDestination const*) + 196
    frame #4: 0x00000001970031a0 CoreImage`CI::RenderToTexture<CI::Texture>::render_to_texture(CI::Image*, CI::Context*, CGSize const&, CI::Texture) const + 88
    frame #5: 0x00000001970034d8 CoreImage`CI::RenderToMTLTexture::render(CI::Image*, CI::Context*) const + 236
    frame #6: 0x0000000196f0f9d4 CoreImage`-[CIRenderDestination _render:withContext:] + 72
    frame #7: 0x0000000196f1354c CoreImage`-[CIContext(CIRenderDestination) _startTaskToRender:toDestination:forPrepareRender:forClear:error:] + 2824
    frame #8: 0x0000000196f12a28 CoreImage`-[CIContext(CIRenderDestination) startTaskToRender:fromRect:toDestination:atPoint:error:] + 144
    frame #9: 0x0000000196e2b398 CoreImage`-[CIContext render:toMTLTexture:commandBuffer:bounds:colorSpace:] + 472
  * frame #10: 0x0000000105e00c1c DepthCamera`MetalRenderer.update(ciImage=<unavailable>, self=0x000000028105c320) at MetalRenderer.swift:86:19 [opt]
    frame #11: 0x0000000105e0d3c8 DepthCamera`specialized RealtimeDepthMaskViewController.draw(self=0x000000010983ba00) at RealTimeDepthViewController.swift:556:30 [opt]
    frame #12: 0x0000000105e07cf8 DepthCamera`@objc RealtimeDepthMaskViewController.draw(in:) [inlined] DepthCamera.RealtimeDepthMaskViewController.draw(in: __C.MTKView) -> () at <compiler-generated>:0 [opt]
    frame #13: 0x0000000105e07cf4 DepthCamera`@objc RealtimeDepthMaskViewController.draw(in:) at <compiler-generated>:521 [opt]
    frame #14: 0x00000001af4d2380 MetalKit`-[MTKView draw] + 184
    frame #15: 0x00000001af4ce8d0 MetalKit`-[MTKViewDisplayLinkTarget draw] + 40
    frame #16: 0x0000000106368208 GPUToolsCore`-[DYDisplayLinkInterposer forwardDisplayLinkCallback:] + 172
    frame #17: 0x000000019c051f24 QuartzCore`CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 628
    frame #18: 0x000000019c120608 QuartzCore`display_timer_callback(__CFMachPort*, void*, long, void*) + 268
    frame #19: 0x000000019551ddac CoreFoundation`__CFMachPortPerform + 176
    frame #20: 0x00000001955487c4 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 60
    frame #21: 0x0000000195547e90 CoreFoundation`__CFRunLoopDoSource1 + 448
    frame #22: 0x0000000195542ac8 CoreFoundation`__CFRunLoopRun + 2144
    frame #23: 0x0000000195541f40 CoreFoundation`CFRunLoopRunSpecific + 480
    frame #24: 0x000000019f7d2534 GraphicsServices`GSEventRunModal + 108
    frame #25: 0x00000001996cd580 UIKitCore`UIApplicationMain + 1940
    frame #26: 0x0000000104b8e5b4 ARMail`main at AppDelegate.swift:13:7 [opt]
    frame #27: 0x00000001953c0e18 libdyld.dylib`start + 4

enter image description here

enter image description here

Provided all of the backtraces, it seems the final question lies with

the CoreImage library is crashing on

thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa9601760910)
frame #0: 0x0000000196fce0a0 CoreImageCI::ColorKernelImage::child_count() const + 100

but I don't have access to this.

It seems that I am sending in the ciImage at its current state into ciContext.render() then something happens as it goes through, and the memory gets corrupted or something. That's my current best guess.

Why it is not a dangling pointer:

It shouldn't be a dangling pointer because I create a new object of the current image called image below. So, although self.videoImage might change, the image is set for this method and the address is different than that of self.videoImage. Nothing else is changing image.

//1) Capture CVPixelBuffer
self.videoImage = ...

//2) Update the MTKView
public func draw(in view: MTKView) {
    if(self.videoImage != nil) {
    let image:CIImage = self.videoImage! //passByValue (to my understanding at least)
    let outputImage = image.applyingFilter("CIBlendWithMask", parameters: parameters) 
    if(renderer.checkRenderCurrentDrawable()) {
        print("Renderer: \(renderer)")
        **//How would there be a dangling pointer for outputImage?
        renderer.update(with: outputImage)
        if(self.finalImage != nil) {
            self.finalImage = outputImage
        }
    else {
        print("Draw nil")
    }
    else {
        print("SVI nil")
}

//3) Render the new image via CIImage
func update(with ciImage: CIImage) {
        // Wait to ensure only kMaxBuffersInFlight are getting proccessed by any stage in the Metal
        // pipeline (App, Metal, Drivers, GPU, etc)
        let _ = inFlightSemaphore.wait(timeout: .distantFuture)
        guard
            let commandBuffer = commandQueue.makeCommandBuffer(),
            let currentDrawable = renderDestination.currentDrawable
            else {
                inFlightSemaphore.signal()
                return
        }
        commandBuffer.label = "MyCommand"

        commandBuffer.addCompletedHandler{ [weak self] commandBuffer in
            if let strongSelf = self {
                strongSelf.inFlightSemaphore.signal()
            }
        }
        //Won't ever get here if simulator because I also check this inside RealTimeDepthViewController but neeed to check so the pod will compile
        #if targetEnvironment(simulator)
        #else
        ciContext.render(ciImage, to: currentDrawable.texture, commandBuffer: commandBuffer, bounds: ciImage.extent, colorSpace: colorSpace)

        commandBuffer.present(currentDrawable)
        commandBuffer.commit()
        #endif
    }

Edit5:

I have removed image.applyingfilter() and magically the errors are a little more narrow.

I am now receiving a -[CIImage release]: message sent to deallocated instance 0x15f2a8ca0 and the error is recognizing my ciImage as Dead which all point to my ciImage memory is being freed. However, I'm not sure why. How do I debug a "message sent to deallocated instance" when "zombies are enabled" is not catching the exact error?

 thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BREAKPOINT (code=1, subcode=0x19562c250)
    frame #0: 0x000000019562c250 CoreFoundation`___forwarding___.cold.2 + 92
    frame #1: 0x00000001955d0f78 CoreFoundation`___forwarding___ + 1120
    frame #2: 0x00000001955d33a0 CoreFoundation`_CF_forwarding_prep_0 + 96
    frame #3: 0x00000001952ff3f8 libobjc.A.dylib`objc_release + 136
    frame #4: 0x0000000101b91574 DepthCamera`specialized RealtimeDepthMaskViewController.draw(self=0x0000000108018c00) at <compiler-generated>:0 [opt]
    frame #5: 0x0000000101b8bbe4 DepthCamera`@objc RealtimeDepthMaskViewController.draw(in:) [inlined] DepthCamera.RealtimeDepthMaskViewController.draw(in: __C.MTKView) -> () at <compiler-generated>:0 [opt]
  * frame #6: 0x0000000101b8bbe0 DepthCamera`@objc RealtimeDepthMaskViewController.draw(in:) at <compiler-generated>:530 [opt]
    frame #7: 0x00000001af4d2380 MetalKit`-[MTKView draw] + 184
    frame #8: 0x00000001af4ce8d0 MetalKit`-[MTKViewDisplayLinkTarget draw] + 40
    frame #9: 0x00000001020b0208 GPUToolsCore`-[DYDisplayLinkInterposer forwardDisplayLinkCallback:] + 172
    frame #10: 0x000000019c051f24 QuartzCore`CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 628
    frame #11: 0x000000019c120608 QuartzCore`display_timer_callback(__CFMachPort*, void*, long, void*) + 268
    frame #12: 0x000000019551ddac CoreFoundation`__CFMachPortPerform + 176
    frame #13: 0x00000001955487c4 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 60
    frame #14: 0x0000000195547e90 CoreFoundation`__CFRunLoopDoSource1 + 448
    frame #15: 0x0000000195542ac8 CoreFoundation`__CFRunLoopRun + 2144
    frame #16: 0x0000000195541f40 CoreFoundation`CFRunLoopRunSpecific + 480
    frame #17: 0x000000019f7d2534 GraphicsServices`GSEventRunModal + 108
    frame #18: 0x00000001996cd580 UIKitCore`UIApplicationMain + 1940
    frame #19: 0x0000000100b2a5b4 ARMail`main at AppDelegate.swift:13:7 [opt]
    frame #20: 0x00000001953c0e18 libdyld.dylib`start + 4
19
  • how you are creating CIImage object? Mar 12, 2020 at 3:06
  • I don't know this language, but instead of all the ifs can you print each of those (is. Command buffer, currentdrawable, etc) to potentially see which one is MIA?
    – Sydney Y
    Mar 12, 2020 at 3:10
  • 1
    Added an edit - check it out. @Sydney - I'm just checking the nil or null of them. If they were MIA, it would print one of them. I'm not getting a single print statement. Mar 12, 2020 at 3:30
  • Are you calling this method async on a DispatchQue?
    – inokey
    Mar 12, 2020 at 6:07
  • 1
    So, I believe this is a threading problem. Some kind of concurrency or race condition that happens arbitrarily. My current setup creates a variable, let image = self.videoImage and somewhere between threads either image or self.videoImage is freeing memory somewhere and other threads access is causing issues. Mar 16, 2020 at 3:21

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.