4

I have some numbers coming in as strings using non-ASCII digits unfortunately. I need to convert them to regular Ruby numbers to do some math on them. So for example if the number-as-a-string "۱۹" comes in, which is 19 but as the characters "extended arabic indic digit one" followed by "extended arabic indic digit nine", I need a way to convert that to the Ruby integer Fixnum 19.

The problem is, according to this, there are 55 groups of 0-9 of these extended digits, i.e. 550 total codepoints I need to handle.

I already know that for a given group, the codepoints for consecutive digits are contiguous, so for example extended arabic indic digit 0 is U+06F0 and extended arabic indic digit 9 is U+06F9, so I can test each digit to see which range it's in and then subtract the zero codepoint as an integer from the codepoint of the character I'm looking at, to give me the regular Ruby integer. For example, 6F9 - 6F0 = 9 (in rough terms, once they're converted to their integer code points).

But to do this, I need to create a giant lookup hash for these 55 ranges and that's a lot of typing. I suppose I could translate the HTML table at the link above into a ruby map, but that feels hacky.

I already know that

"۱۹" =~ /[[:digit:]]+/

will be a match, but the question is "How to turn those Unicode digits back into regular Ruby integers?"

There has to be a better way! Any ideas?

Thanks!

1 Answer 1

6

This is relatively painless.

class DecimalToIntegerConverter
  altzeros = [0x06f0, 0xff10] # ... need all zeroes here
  @@digits = altzeros.flat_map { |z| ((z.chr(Encoding::UTF_8))..((z+9).chr(Encoding::UTF_8))).to_a }.join('')
  @@replacements = "0123456789" * altzeros.size
  def self.convert(str)
    str.tr(@@digits, @@replacements).to_i
  end
end

str = "۱۹ and 25?"
str.scan(/[[:digit:]]+/).map do |s|
  DecimalToIntegerConverter.convert(s)
end
# => [19, 25]
6
  • Thanks @Amadan, that looks like it would work well. I wouldn't have to type in the 55 ranges, but I'd still have to type in the 55 altzeros. Which I can do ... But is there another way where I don't even have to type in the 55 altzeros? A gem or some further magic?
    – Joel
    May 20, 2016 at 6:39
  • @CarySwoveland "0123456789" * altzeros.size ensures that both strings have the same size
    – Stefan
    May 20, 2016 at 6:52
  • @Stefan, oh, I see. Thanks. May 20, 2016 at 7:00
  • You could also create a replacement hash hash = {'۱' => '1', ..., '۹' => 9} and substitute the digits via '۱۹'.gsub(/[[:digit:]]/, hash)
    – Stefan
    May 20, 2016 at 7:52
  • I was hoping the Twitter I18N gem twitter_cldr would have something built in for this but could not find any way to accomplish this with it. Anyone else care to verify? May 20, 2016 at 12:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.