1

I am trying to use some Rust wasm code in Bigquery as UDF, and in order to pass on Java String to Rust code the TextEncoder and TextDecoder would be needed to conveniently doing that. As it mentioned here Passing a JavaScript string to a Rust function compiled to WebAssembly

But when I try out some of my code on BigQuery, I encountered an error saying TextEncoder is not defined.

enter image description here

You can try it out as well with a query like this: https://github.com/liufuyang/rb62-wasm/blob/master/try-3-old.sql

While a working version without using TextEncoder is at https://github.com/liufuyang/rb62-wasm/blob/master/try-3.sql

1 Answer 1

2

That means the object is not defined.

As an option, bring your own TextEncoder.

For example, take your try-3-old.sql, and then add this line at the end of the JS UDF definition:

return main();
'''
OPTIONS (library="gs://fh-bigquery/js/inexorabletash.encoding.js");

And now it works:

enter image description here


(wondering, what's the goal with rb62?)

7
  • Thank you, I think this is exactly what I need.
    – Fuyang Liu
    Feb 6, 2020 at 21:24
  • About the goal with rb62: basically it is used internally at Spotify to encode/decode some gid (128bit or 16byte) to 62 based value, as in those examples show. We already have a js UDF library to do this. I am just testing out 1. whether it is possible to use some Rust/wasm function to do the same via js UDF interface. 2. Could the speed be improved or not as we normally have millions of rows to parse.
    – Fuyang Liu
    Feb 6, 2020 at 21:28
  • What I find so far is that, as it is not convenient to pass JS string (utf16) values to the wasm Rust code environment (where the string is in utf8, also this seems can only be done via copy stuff through the wasm memory), it seems not gaining any performance benefit, specificly when trying to replace some JS string encoding lib to a Rust string encoding lib?
    – Fuyang Liu
    Feb 6, 2020 at 21:32
  • Please accept answer if useful! And let's start a new question to find a way to improve performance as desired :) Feb 6, 2020 at 21:43
  • Thank you. I will try out some stuff then ask if necessary :)
    – Fuyang Liu
    Feb 7, 2020 at 8:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.