8

I have researched all over in order to find a kernel that performs adaptive thresholding on iOS. Unfortunately I do not understand the kernel language or the logic behind it. Below, I have found a routine that performs thresholding (https://gist.github.com/xhruso00/a3f8a9c8ae7e33b8b23d)

static NSString * const kKernelSource = @"kernel vec4 thresholdKernel(sampler image)\n"
"{\n"
"  float inputThreshold = 0.05;\n"
"  float pass = 1.0;\n"
"  float fail = 0.0;\n"
"  const vec4   vec_Y = vec4( 0.299, 0.587, 0.114, 0.0 );\n"
"  vec4  src = unpremultiply( sample(image, samplerCoord(image)) );\n"
"  float Y = dot( src, vec_Y );\n"
"  src.rgb = vec3( compare( Y - inputThreshold, fail, pass));\n"
"  return premultiply(src);\n"
"}";

Is it possible to rewrite this into an adaptive thresholding kernel? The image I am supplying to it has been turned into B&W and has already been blurred. Are there any resources you could point me to? I would like to stick with CoreImage as my whole stack is built around it.

Edit: The best example / reference from what I am trying to achieve has been implemented in GPUImage's GPUImageAdaptiveThresholdFilter - https://github.com/BradLarson/GPUImage/blob/c5f0914152419437869c35e29858773b1a06083c/framework/Source/GPUImageAdaptiveThresholdFilter.m

1
  • I updated my code with description, metal and modern approach gist.github.com/xhruso00/a3f8a9c8ae7e33b8b23d I inspired myself with below code but the code below doesn't respect alpha premultiplication (transparent areas will become black or white).
    – Marek H
    Oct 17, 2019 at 21:18

3 Answers 3

6

How does this look: I've used the CoreImage CIBoxBlur (although the dedicated convolution filters may be faster) and passed the output of that into my existing threshold filter.

class AdaptiveThresholdFilter: CIFilter
{
    var inputImage : CIImage?


    var thresholdKernel =  CIColorKernel(string:
    "kernel vec4 thresholdFilter(__sample image, __sample threshold)" +
    "{" +
    "   float imageLuma = dot(image.rgb, vec3(0.2126, 0.7152, 0.0722));" +
    "   float thresholdLuma = dot(threshold.rgb, vec3(0.2126, 0.7152, 0.0722));" +

    "   return vec4(vec3(step(imageLuma, thresholdLuma)), 1.0);" +
    "}"
    )


    override var outputImage: CIImage!
    {
        guard let inputImage = inputImage,
            thresholdKernel = thresholdKernel else
        {
            return nil
        }

        let blurred = inputImage.imageByApplyingFilter("CIBoxBlur",
               withInputParameters: [kCIInputRadiusKey: 9])

        let extent = inputImage.extent
        let arguments = [inputImage, blurred]

        return thresholdKernel.applyWithExtent(extent, arguments: arguments)
    }
}

I found this image of a shaded page and with this code:

let page = CIImage(image: UIImage(named: "son1.gif")!)

let filter = AdaptiveThresholdFilter()

filter.inputImage = page

let final = filter.outputImage

I got this result:

enter image description here

Cheers!

Simon

1
5

Simon's Filter is the right approach to achieve the desired effect, however, you have to modify a couple of things.

First of all, switch the order of imageLuma and thresholdLuma, since we want black letters to remain black and not the other way around. Also, you should add a constant (I chose 0.01) to remove noise.

    var thresholdKernel =  CIColorKernel(string:
    "kernel vec4 thresholdFilter(__sample image, __sample threshold)" +
        "{" +
        "   float imageLuma = dot(image.rgb, vec3(0.2126, 0.7152, 0.0722));" +
        "   float thresholdLuma = dot(threshold.rgb, vec3(0.2126, 0.7152, 0.0722));" +
        "   return vec4(vec3(step(thresholdLuma, imageLuma+0.001)), 1);"     
    "}"

override var outputImage: CIImage! {
    guard let inputImage = inputImage,
        let thresholdKernel = thresholdKernel else {
        return nil
    }
    let blurred = inputImage.applyingFilter("CIBoxBlur", withInputParameters: [kCIInputRadiusKey: 5]) // block size
    let extent = inputImage.extent
    let arguments = [inputImage, blurred]
    return thresholdKernel.apply(withExtent: extent, arguments: arguments)
}

And this is, what you get Only using Apple's Core Image, without having to install any external libraries :)

enter image description here

Of course, you can play around a little with the values of constant and block size.

3
  • Pascal, thank you for your answer. Any idea on how to implement this in CoreImage too? github.com/ctodobom/OpenNoteScanner/blob/master/app/src/main/…
    – mmackh
    Jun 2, 2017 at 18:23
  • You mentioned a constant of 0.01 but you have 0.001, is this a mistake? Also, do you apply this constant to the imageLuma or the threshold luma (assuming i did want white letters with black background.
    – Pochi
    Jul 28, 2017 at 3:29
  • 0.01 and 0.001 doesn't seem to make too much of a difference. You may try different values to fit your need.
    – triiiiista
    Dec 11, 2017 at 5:12
4

You can use CIColorThresholdOtsu core image filter

1
  • 1
    This more recent answer is quite useful, too! If a developer writes a thresholding algorithm and just wants to apply the threshold, then CIColorThreshold is also worth a look: cifilter.io/CIColorThreshold
    – Rethunk
    Mar 29, 2021 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.