Edit6:
So, I believe this is a threading problem. Some kind of concurrency or race condition that happens arbitrarily. My current setup creates a variable, let image = self.videoImage and somewhere between threads either image or self.videoImage is freeing memory somewhere and other threads access is causing issues. The goal is to utilize a different technique will deploying threads and ensure data is prepared.
I am rendering a CIImage
using
if(ciImage == nil) { print("CIImage nil") }
if(currentDrawable == nil) { print("Draw nil") }
if(currentDrawable.texture == nil) { print("Text nil") }
if(commandBuffer == nil) { print("CB nil") }
if(ciImage.extent == nil) { print("Extent nil") }
if(ciContext == nil) { print("Context nil") }
if(colorSpace == nil) { print("CS nil") }
ciContext.render(ciImage, to: currentDrawable.texture, commandBuffer: commandBuffer, bounds: ciImage.extent, colorSpace: colorSpace)
I am printing out nil
because I was trying to determine the cause of my EXC_BAD_ACCESS
error which I believe generally stems from accessing bad memory, or a variable that is not there anymore. Yet, I am still getting the error without obtaining a nil
message. Why would I be getting a EXC_BAD_ACCESS
error during a CIContext.render()
call?
Edit: Here is the entire method. I showed the picture to illustrate where the EXC_BAD_ACCESS is. For @Shauket's comment, if the CIImage
was nil prior, it would have given an EXC_BAD_ACCESS
prior to the call.
Note: This happens what seems randomly; obviously there is a reason, but it is very sporadic. It could happen the first time or the millionth time I call this method.
func update(with ciImage: CIImage) {
// Wait to ensure only kMaxBuffersInFlight are getting proccessed by any stage in the Metal
// pipeline (App, Metal, Drivers, GPU, etc)
let _ = inFlightSemaphore.wait(timeout: .distantFuture)
guard
let commandBuffer = commandQueue.makeCommandBuffer(),
let currentDrawable = renderDestination.currentDrawable
else {
inFlightSemaphore.signal()
return
}
commandBuffer.label = "MyCommand"
commandBuffer.addCompletedHandler{ [weak self] commandBuffer in
if let strongSelf = self {
strongSelf.inFlightSemaphore.signal()
}
}
//Won't ever get here if simulator because I also check this inside RealTimeDepthViewController but need to check so the pod will compile
#if targetEnvironment(simulator)
#else
if(ciImage == nil) { print("CIImage nil") }
if(currentDrawable == nil) { print("Draw nil") }
if(currentDrawable.texture == nil) { print("Text nil") }
if(commandBuffer == nil) { print("CB nil") }
if(ciImage.extent == nil) { print("Extent nil") }
if(ciContext == nil) { print("Context nil") }
if(colorSpace == nil) { print("CS nil") }
ciContext.render(ciImage, to: currentDrawable.texture, commandBuffer: commandBuffer, bounds: ciImage.extent, colorSpace: colorSpace)
commandBuffer.present(currentDrawable)
commandBuffer.commit()
#endif
}
Additionally, this is the backtrace.
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x13b00021d)
frame #0: 0x0000000196fce0a0 CoreImage`CI::ColorKernelImage::child_count() const + 100
frame #1: 0x0000000196fff704 CoreImage`CI::GraphObject::traverse_unique(CI::GraphObject*, void (CI::GraphObject*, CI::GraphObject*, int, int, int) block_pointer) + 244
frame #2: 0x0000000197003b30 CoreImage`CI::prepare_initial_graph(CI::Context*, char const*, CI::Image*, CI::RenderDestination const*, CGRect, CGColorSpace*, CI::PixelFormat, CI::swizzle_info, CI::Affine const&, bool, CI::TextureDescriptor*) + 300
frame #3: 0x0000000197005750 CoreImage`CI::image_render(char const*, CI::Context*, CI::Image*, CI::RenderDestination const*) + 196
frame #4: 0x00000001970031a0 CoreImage`CI::RenderToTexture<CI::Texture>::render_to_texture(CI::Image*, CI::Context*, CGSize const&, CI::Texture) const + 88
frame #5: 0x00000001970034d8 CoreImage`CI::RenderToMTLTexture::render(CI::Image*, CI::Context*) const + 236
frame #6: 0x0000000196f0f9d4 CoreImage`-[CIRenderDestination _render:withContext:] + 72
frame #7: 0x0000000196f1354c CoreImage`-[CIContext(CIRenderDestination) _startTaskToRender:toDestination:forPrepareRender:forClear:error:] + 2824
frame #8: 0x0000000196f12a28 CoreImage`-[CIContext(CIRenderDestination) startTaskToRender:fromRect:toDestination:atPoint:error:] + 144
frame #9: 0x0000000196e2b398 CoreImage`-[CIContext render:toMTLTexture:commandBuffer:bounds:colorSpace:] + 472
* frame #10: 0x0000000104b76024 DepthCamera`MetalRenderer.update(ciImage=0x000000010994d810, self=<unavailable>) at MetalRenderer.swift:81:19 [opt]
frame #11: 0x0000000104b8ea2c DepthCamera`specialized RealtimeDepthMaskViewController.draw(self=0x0000000109c17c80) at RealTimeDepthViewController.swift:543:26 [opt]
frame #12: 0x0000000104b843d0 DepthCamera`merged @objc DepthCamera.RealtimeDepthMaskViewController.buttonClicked(sender: __C.UIButton) -> () + 60
frame #13: 0x00000001af4d2380 MetalKit`-[MTKView draw] + 184
frame #14: 0x00000001af4ce8d0 MetalKit`-[MTKViewDisplayLinkTarget draw] + 40
frame #15: 0x000000010524c208 GPUToolsCore`-[DYDisplayLinkInterposer forwardDisplayLinkCallback:] + 172
frame #16: 0x000000019c051f24 QuartzCore`CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 628
frame #17: 0x000000019c120608 QuartzCore`display_timer_callback(__CFMachPort*, void*, long, void*) + 268
frame #18: 0x000000019551ddac CoreFoundation`__CFMachPortPerform + 176
frame #19: 0x00000001955487c4 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 60
frame #20: 0x0000000195547e90 CoreFoundation`__CFRunLoopDoSource1 + 448
frame #21: 0x0000000195542ac8 CoreFoundation`__CFRunLoopRun + 2144
frame #22: 0x0000000195541f40 CoreFoundation`CFRunLoopRunSpecific + 480
frame #23: 0x000000019f7d2534 GraphicsServices`GSEventRunModal + 108
frame #24: 0x00000001996cd580 UIKitCore`UIApplicationMain + 1940
frame #25: 0x0000000102a78d38 ARMail`main at AppDelegate.swift:13:7 [opt]
frame #26: 0x00000001953c0e18 libdyld.dylib`start + 4
Edit2:
I have now figured out that the issue is currentDrawable
. The below image illustrates that the variable currentDrawable
is not nil right before the cicontext.render
but it IS nil (address 0x0) at the time of the call.
Documentation states
If all drawable objects are in use, the value of this property is nil. Your app should check that currentDrawable isn’t nil before attempting to draw. The view changes the value of this property only after returning from a drawing function, either draw(_:) from a subclassed instance of the view, or draw(in:) from the view’s delegate. In my case, this method is only called within my draw method.
extension RealtimeDepthMaskViewController: MTKViewDelegate {
public func draw(in view: MTKView) {
if(renderer.checkRenderCurrentDrawable()) {
renderer.update(with: image)
if(self.finalImage != nil) {
self.finalImage = image
}
}
else {
print("Draw nil")
}
}
}
func checkRenderCurrentDrawable() -> Bool {
return self.renderDestination.currentDrawable != nil
}
So, I check if it is nil
, it is not at the time, then at the time of cicontext.render
it becomes nil...
Edit3:
Still looking into it, but obtained this on the last test...
020-03-12 02:20:51.289750-0400 ARMail[6169:1212379] Compiler failed to build request
2020-03-12 02:20:51.289963-0400 ARMail[6169:1212379] Metal library creation failed: Error Domain=MTLLibraryErrorDomain Code=3 "Function _blendWithMaskB0 argument marked as info type Output for function composition has different type <4 x float> than the source function _ci_early_out return type i1" UserInfo={NSLocalizedDescription=Function _blendWithMaskB0 argument marked as info type Output for function composition has different type <4 x float> than the source function _ci_early_out return type
Edit4:
Similar backtrace below; after much testing, it seems that sometimes self
is the bad address. Or currentDrawable.texture
is the bad address. Or ciImage
is the bad address. However, in the end all be all, more than the rest, ciImage
is the culprit which provides the bad address. Below the backtrace is a snippet indicating that the address for ciImage
cannot be obtained, which seems like it is the bad address. However, I'm still not sure why, because ciImage
is not nil 0.125 secs before when I print out ciImage
.
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa9601760910)
frame #0: 0x0000000196fce0a0 CoreImage`CI::ColorKernelImage::child_count() const + 100
frame #1: 0x0000000196fff704 CoreImage`CI::GraphObject::traverse_unique(CI::GraphObject*, void (CI::GraphObject*, CI::GraphObject*, int, int, int) block_pointer) + 244
frame #2: 0x0000000197003b30 CoreImage`CI::prepare_initial_graph(CI::Context*, char const*, CI::Image*, CI::RenderDestination const*, CGRect, CGColorSpace*, CI::PixelFormat, CI::swizzle_info, CI::Affine const&, bool, CI::TextureDescriptor*) + 300
frame #3: 0x0000000197005750 CoreImage`CI::image_render(char const*, CI::Context*, CI::Image*, CI::RenderDestination const*) + 196
frame #4: 0x00000001970031a0 CoreImage`CI::RenderToTexture<CI::Texture>::render_to_texture(CI::Image*, CI::Context*, CGSize const&, CI::Texture) const + 88
frame #5: 0x00000001970034d8 CoreImage`CI::RenderToMTLTexture::render(CI::Image*, CI::Context*) const + 236
frame #6: 0x0000000196f0f9d4 CoreImage`-[CIRenderDestination _render:withContext:] + 72
frame #7: 0x0000000196f1354c CoreImage`-[CIContext(CIRenderDestination) _startTaskToRender:toDestination:forPrepareRender:forClear:error:] + 2824
frame #8: 0x0000000196f12a28 CoreImage`-[CIContext(CIRenderDestination) startTaskToRender:fromRect:toDestination:atPoint:error:] + 144
frame #9: 0x0000000196e2b398 CoreImage`-[CIContext render:toMTLTexture:commandBuffer:bounds:colorSpace:] + 472
* frame #10: 0x0000000105e00c1c DepthCamera`MetalRenderer.update(ciImage=<unavailable>, self=0x000000028105c320) at MetalRenderer.swift:86:19 [opt]
frame #11: 0x0000000105e0d3c8 DepthCamera`specialized RealtimeDepthMaskViewController.draw(self=0x000000010983ba00) at RealTimeDepthViewController.swift:556:30 [opt]
frame #12: 0x0000000105e07cf8 DepthCamera`@objc RealtimeDepthMaskViewController.draw(in:) [inlined] DepthCamera.RealtimeDepthMaskViewController.draw(in: __C.MTKView) -> () at <compiler-generated>:0 [opt]
frame #13: 0x0000000105e07cf4 DepthCamera`@objc RealtimeDepthMaskViewController.draw(in:) at <compiler-generated>:521 [opt]
frame #14: 0x00000001af4d2380 MetalKit`-[MTKView draw] + 184
frame #15: 0x00000001af4ce8d0 MetalKit`-[MTKViewDisplayLinkTarget draw] + 40
frame #16: 0x0000000106368208 GPUToolsCore`-[DYDisplayLinkInterposer forwardDisplayLinkCallback:] + 172
frame #17: 0x000000019c051f24 QuartzCore`CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 628
frame #18: 0x000000019c120608 QuartzCore`display_timer_callback(__CFMachPort*, void*, long, void*) + 268
frame #19: 0x000000019551ddac CoreFoundation`__CFMachPortPerform + 176
frame #20: 0x00000001955487c4 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 60
frame #21: 0x0000000195547e90 CoreFoundation`__CFRunLoopDoSource1 + 448
frame #22: 0x0000000195542ac8 CoreFoundation`__CFRunLoopRun + 2144
frame #23: 0x0000000195541f40 CoreFoundation`CFRunLoopRunSpecific + 480
frame #24: 0x000000019f7d2534 GraphicsServices`GSEventRunModal + 108
frame #25: 0x00000001996cd580 UIKitCore`UIApplicationMain + 1940
frame #26: 0x0000000104b8e5b4 ARMail`main at AppDelegate.swift:13:7 [opt]
frame #27: 0x00000001953c0e18 libdyld.dylib`start + 4
Provided all of the backtraces, it seems the final question lies with
the CoreImage library is crashing on
thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa9601760910)
frame #0: 0x0000000196fce0a0 CoreImageCI::ColorKernelImage::child_count() const + 100
but I don't have access to this.
It seems that I am sending in the ciImage
at its current state into ciContext.render()
then something happens as it goes through, and the memory gets corrupted or something. That's my current best guess.
Why it is not a dangling pointer:
It shouldn't be a dangling pointer because I create a new object of the current image called image
below. So, although self.videoImage
might change, the image
is set for this method and the address is different than that of self.videoImage
. Nothing else is changing image
.
//1) Capture CVPixelBuffer
self.videoImage = ...
//2) Update the MTKView
public func draw(in view: MTKView) {
if(self.videoImage != nil) {
let image:CIImage = self.videoImage! //passByValue (to my understanding at least)
let outputImage = image.applyingFilter("CIBlendWithMask", parameters: parameters)
if(renderer.checkRenderCurrentDrawable()) {
print("Renderer: \(renderer)")
**//How would there be a dangling pointer for outputImage?
renderer.update(with: outputImage)
if(self.finalImage != nil) {
self.finalImage = outputImage
}
else {
print("Draw nil")
}
else {
print("SVI nil")
}
//3) Render the new image via CIImage
func update(with ciImage: CIImage) {
// Wait to ensure only kMaxBuffersInFlight are getting proccessed by any stage in the Metal
// pipeline (App, Metal, Drivers, GPU, etc)
let _ = inFlightSemaphore.wait(timeout: .distantFuture)
guard
let commandBuffer = commandQueue.makeCommandBuffer(),
let currentDrawable = renderDestination.currentDrawable
else {
inFlightSemaphore.signal()
return
}
commandBuffer.label = "MyCommand"
commandBuffer.addCompletedHandler{ [weak self] commandBuffer in
if let strongSelf = self {
strongSelf.inFlightSemaphore.signal()
}
}
//Won't ever get here if simulator because I also check this inside RealTimeDepthViewController but neeed to check so the pod will compile
#if targetEnvironment(simulator)
#else
ciContext.render(ciImage, to: currentDrawable.texture, commandBuffer: commandBuffer, bounds: ciImage.extent, colorSpace: colorSpace)
commandBuffer.present(currentDrawable)
commandBuffer.commit()
#endif
}
Edit5:
I have removed image.applyingfilter()
and magically the errors are a little more narrow.
I am now receiving a -[CIImage release]: message sent to deallocated instance 0x15f2a8ca0
and the error is recognizing my ciImage
as Dead
which all point to my ciImage
memory is being freed. However, I'm not sure why. How do I debug a "message sent to deallocated instance" when "zombies are enabled" is not catching the exact error?
thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BREAKPOINT (code=1, subcode=0x19562c250)
frame #0: 0x000000019562c250 CoreFoundation`___forwarding___.cold.2 + 92
frame #1: 0x00000001955d0f78 CoreFoundation`___forwarding___ + 1120
frame #2: 0x00000001955d33a0 CoreFoundation`_CF_forwarding_prep_0 + 96
frame #3: 0x00000001952ff3f8 libobjc.A.dylib`objc_release + 136
frame #4: 0x0000000101b91574 DepthCamera`specialized RealtimeDepthMaskViewController.draw(self=0x0000000108018c00) at <compiler-generated>:0 [opt]
frame #5: 0x0000000101b8bbe4 DepthCamera`@objc RealtimeDepthMaskViewController.draw(in:) [inlined] DepthCamera.RealtimeDepthMaskViewController.draw(in: __C.MTKView) -> () at <compiler-generated>:0 [opt]
* frame #6: 0x0000000101b8bbe0 DepthCamera`@objc RealtimeDepthMaskViewController.draw(in:) at <compiler-generated>:530 [opt]
frame #7: 0x00000001af4d2380 MetalKit`-[MTKView draw] + 184
frame #8: 0x00000001af4ce8d0 MetalKit`-[MTKViewDisplayLinkTarget draw] + 40
frame #9: 0x00000001020b0208 GPUToolsCore`-[DYDisplayLinkInterposer forwardDisplayLinkCallback:] + 172
frame #10: 0x000000019c051f24 QuartzCore`CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 628
frame #11: 0x000000019c120608 QuartzCore`display_timer_callback(__CFMachPort*, void*, long, void*) + 268
frame #12: 0x000000019551ddac CoreFoundation`__CFMachPortPerform + 176
frame #13: 0x00000001955487c4 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 60
frame #14: 0x0000000195547e90 CoreFoundation`__CFRunLoopDoSource1 + 448
frame #15: 0x0000000195542ac8 CoreFoundation`__CFRunLoopRun + 2144
frame #16: 0x0000000195541f40 CoreFoundation`CFRunLoopRunSpecific + 480
frame #17: 0x000000019f7d2534 GraphicsServices`GSEventRunModal + 108
frame #18: 0x00000001996cd580 UIKitCore`UIApplicationMain + 1940
frame #19: 0x0000000100b2a5b4 ARMail`main at AppDelegate.swift:13:7 [opt]
frame #20: 0x00000001953c0e18 libdyld.dylib`start + 4
if
s can you print each of those (is. Command buffer, currentdrawable, etc) to potentially see which one is MIA?nil
ornull
of them. If they were MIA, it would print one of them. I'm not getting a single print statement.let image = self.videoImage
and somewhere between threads eitherimage
orself.videoImage
is freeing memory somewhere and other threads access is causing issues.