458

Is there a commonly accepted technique for efficiently converting JavaScript strings to ArrayBuffers and vice-versa? Specifically, I'd like to be able to write the contents of an ArrayBuffer to localStorage and then read it back.

9
  • 1
    I don't have any experience in this, but judging from the API documentation (khronos.org/registry/typedarray/specs/latest) if you build an Int8Array ArrayBufferView it might be possible to simply use the bracket notation to copy chars string[i] = buffer[i] and vice versa.
    – FK82
    Aug 6, 2011 at 13:30
  • 2
    @FK82, that looks like a reasonable approach (using Uint16Arrays for JS's 16-bit characters), but JavaScript strings are immutable so you can't assign directly to a character position. I would still need to copy String.fromCharCode(x) of each value in the Uint16Array to a normal Array and then call .join() on the Array.
    – kpozin
    Aug 6, 2011 at 15:11
  • 7
    @kpozin It turns out that most modern JS engines have optimized string concatenation to the point where it's cheaper to just use string += String.fromCharCode(buffer[i]);. It does seem odd that there wouldn't be built-in methods for converting between strings and typed arrays. They had to know something like this would come up.
    – Erin
    Oct 21, 2011 at 18:48
  • 1
    arrayBuffer.toString() is working well for me. Feb 16, 2015 at 23:24
  • 1
    @citizen conn - I don't know what browser you're using, but on Chrome arrayBuffer.toString() just gives "[object ArrayBuffer]". Not very helpful.
    – mrec
    Jun 2, 2015 at 14:00

29 Answers 29

309

Update 2016 - five years on there are now new methods in the specs (see support below) to convert between strings and typed arrays using proper encoding.

TextEncoder

The TextEncoder represents:

The TextEncoder interface represents an encoder for a specific method, that is a specific character encoding, like utf-8, iso-8859-2, koi8, cp1261, gbk, ... An encoder takes a stream of code points as input and emits a stream of bytes.

Change note since the above was written: (ibid.)

Note: Firefox, Chrome and Opera used to have support for encoding types other than utf-8 (such as utf-16, iso-8859-2, koi8, cp1261, and gbk). As of Firefox 48 [...], Chrome 54 [...] and Opera 41, no other encoding types are available other than utf-8, in order to match the spec.*

*) Updated specs (W3) and here (whatwg).

After creating an instance of the TextEncoder it will take a string and encode it using a given encoding parameter:

if (!("TextEncoder" in window)) 
  alert("Sorry, this browser does not support TextEncoder...");

var enc = new TextEncoder(); // always utf-8
console.log(enc.encode("This is a string converted to a Uint8Array"));

You then of course use the .buffer parameter on the resulting Uint8Array to convert the underlaying ArrayBuffer to a different view if needed.

Just make sure that the characters in the string adhere to the encoding schema, for example, if you use characters outside the UTF-8 range in the example they will be encoded to two bytes instead of one.

For general use you would use UTF-16 encoding for things like localStorage.

TextDecoder

Likewise, the opposite process uses the TextDecoder:

The TextDecoder interface represents a decoder for a specific method, that is a specific character encoding, like utf-8, iso-8859-2, koi8, cp1261, gbk, ... A decoder takes a stream of bytes as input and emits a stream of code points.

All available decoding types can be found here.

if (!("TextDecoder" in window))
  alert("Sorry, this browser does not support TextDecoder...");

var enc = new TextDecoder("utf-8");
var arr = new Uint8Array([84,104,105,115,32,105,115,32,97,32,85,105,110,116,
                          56,65,114,114,97,121,32,99,111,110,118,101,114,116,
                          101,100,32,116,111,32,97,32,115,116,114,105,110,103]);
console.log(enc.decode(arr));

The MDN StringView library

An alternative to these is to use the StringView library (licensed as lgpl-3.0) which goal is:

  • to create a C-like interface for strings (i.e., an array of character codes — an ArrayBufferView in JavaScript) based upon the JavaScript ArrayBuffer interface
  • to create a highly extensible library that anyone can extend by adding methods to the object StringView.prototype
  • to create a collection of methods for such string-like objects (since now: stringViews) which work strictly on arrays of numbers rather than on creating new immutable JavaScript strings
  • to work with Unicode encodings other than JavaScript's default UTF-16 DOMStrings

giving much more flexibility. However, it would require us to link to or embed this library while TextEncoder/TextDecoder is being built-in in modern browsers.

Support

As of July/2018:

TextEncoder (Experimental, On Standard Track)

 Chrome    | Edge      | Firefox   | IE        | Opera     | Safari
 ----------|-----------|-----------|-----------|-----------|-----------
     38    |     ?     |    19°    |     -     |     25    |     -

 Chrome/A  | Edge/mob  | Firefox/A | Opera/A   |Safari/iOS | Webview/A
 ----------|-----------|-----------|-----------|-----------|-----------
     38    |     ?     |    19°    |     ?     |     -     |     38

°) 18: Firefox 18 implemented an earlier and slightly different version
of the specification.

WEB WORKER SUPPORT:

Experimental, On Standard Track

 Chrome    | Edge      | Firefox   | IE        | Opera     | Safari
 ----------|-----------|-----------|-----------|-----------|-----------
     38    |     ?     |     20    |     -     |     25    |     -

 Chrome/A  | Edge/mob  | Firefox/A | Opera/A   |Safari/iOS | Webview/A
 ----------|-----------|-----------|-----------|-----------|-----------
     38    |     ?     |     20    |     ?     |     -     |     38

Data from MDN - `npm i -g mdncomp` by epistemex
10
  • 3
    No support for TextDecoder from IE & Edge: caniuse.com/#search=TextDecoder Feb 23, 2018 at 13:14
  • 1
    According to MS it's in development: developer.microsoft.com/en-us/microsoft-edge/platform/status/… Mar 29, 2018 at 8:12
  • 1
    One-liner: var encoder = 'TextEncoder' in window ? new TextEncoder() : {encode: function(str){return Uint8Array.from(str, function(c){return c.codePointAt(0);});}}; so you can just var array = encoder.encode('hello');
    – Yeti
    Jul 10, 2018 at 20:35
  • 5
    The thing with TextEncoder is that if you have binary data in a string (like, image), you don't want to use TextEncoder (apparently). Characters with code points greater than 127 produce two bytes. Why do I have binary data in a string? cy.fixture(NAME, 'binary') (cypress) produces a string.
    – x-yuri
    Nov 22, 2019 at 16:21
  • 1
    You should wrap new TextDecoder("encoding") in a try/catch because you might have support for TextDecoder but not the character encoding you want to use.
    – PHP Guru
    Feb 13, 2021 at 1:42
200

Although Dennis and gengkev solutions of using Blob/FileReader work, I wouldn't suggest taking that approach. It is an async approach to a simple problem, and it is much slower than a direct solution. I've made a post in html5rocks with a simpler and (much faster) solution: http://updates.html5rocks.com/2012/06/How-to-convert-ArrayBuffer-to-and-from-String

And the solution is:

function ab2str(buf) {
  return String.fromCharCode.apply(null, new Uint16Array(buf));
}

function str2ab(str) {
  var buf = new ArrayBuffer(str.length*2); // 2 bytes for each char
  var bufView = new Uint16Array(buf);
  for (var i=0, strLen=str.length; i<strLen; i++) {
    bufView[i] = str.charCodeAt(i);
  }
  return buf;
}

EDIT:

The Encoding API helps solving the string conversion problem. Check out the response from Jeff Posnik on Html5Rocks.com to the above original article.

Excerpt:

The Encoding API makes it simple to translate between raw bytes and native JavaScript strings, regardless of which of the many standard encodings you need to work with.

<pre id="results"></pre>

<script>
  if ('TextDecoder' in window) {
    // The local files to be fetched, mapped to the encoding that they're using.
    var filesToEncoding = {
      'utf8.bin': 'utf-8',
      'utf16le.bin': 'utf-16le',
      'macintosh.bin': 'macintosh'
    };

    Object.keys(filesToEncoding).forEach(function(file) {
      fetchAndDecode(file, filesToEncoding[file]);
    });
  } else {
    document.querySelector('#results').textContent = 'Your browser does not support the Encoding API.'
  }

  // Use XHR to fetch `file` and interpret its contents as being encoded with `encoding`.
  function fetchAndDecode(file, encoding) {
    var xhr = new XMLHttpRequest();
    xhr.open('GET', file);
    // Using 'arraybuffer' as the responseType ensures that the raw data is returned,
    // rather than letting XMLHttpRequest decode the data first.
    xhr.responseType = 'arraybuffer';
    xhr.onload = function() {
      if (this.status == 200) {
        // The decode() method takes a DataView as a parameter, which is a wrapper on top of the ArrayBuffer.
        var dataView = new DataView(this.response);
        // The TextDecoder interface is documented at http://encoding.spec.whatwg.org/#interface-textdecoder
        var decoder = new TextDecoder(encoding);
        var decodedString = decoder.decode(dataView);
        // Add the decoded file's text to the <pre> element on the page.
        document.querySelector('#results').textContent += decodedString + '\n';
      } else {
        console.error('Error while requesting', file, this);
      }
    };
    xhr.send();
  }
</script>
28
  • 19
    Unfortunately my comment on html5rocks is not approved yet. Therefore a short answer here. I still think, this isn't the right way, because you miss plenty of characters, especially because most pages are in UTF-8 encoding today. On one side, for more special characters (let's say Asian), the charCodeAt function returns a 4-Byte value, so they will be chopped. On the other side, simple English characters will grow the ArrayBuffer twice (you're using 2 Byte for every 1-Byte character). Imagine sending an English text over a WebSocket, it will need twice time (not good in real time environment).
    – Dennis
    Jun 21, 2012 at 9:12
  • 10
    Three examples: (1) This is a cool text! 20 Byte in UTF8 -- 40 Byte in Unicode. (2) ÄÖÜ 6 Bytes in UTF8 -- 6 Bytes in Unicode. (3) ☐☑☒ 9 Bytes in UTF8 -- 6 Bytes in Unicode. If you want to store the string as UTF8-file (via Blob and File Writer API), you cannot use this 2 methods, because the ArrayBuffer will be in Unicode and not in UTF8.
    – Dennis
    Sep 6, 2012 at 11:19
  • 4
    I get an error: Uncaught RangeError: Maximum call stack size exceeded. What could be the problem?
    – Jacob
    Feb 19, 2013 at 8:29
  • 8
    @Dennis - JS strings use UCS2, not UTF8 (or even UTF16) - meaning charCodeAt() always returns values 0 -> 65535. Any UTF-8 code point that requires 4 bytes ends will be represented with surrogate pairs (see en.wikipedia.org/wiki/… ) - i.e. two separate 16-bit UCS2 values.
    – broofa
    Mar 4, 2013 at 22:29
  • 7
    @jacob - I believe the error is because there's a limit on the length of the array that can be passed to the apply() method. E.g. String.fromCharCode.apply(null, new Uint16Array(new ArrayBuffer(246300))).length works for me in Chrome, but if you use 246301 instead, I get your RangeError exception
    – broofa
    Mar 4, 2013 at 22:47
113

You can use TextEncoder and TextDecoder from the Encoding standard, which is polyfilled by the stringencoding library, to convert string to and from ArrayBuffers:

var uint8array = new TextEncoder().encode(string);
var string = new TextDecoder(encoding).decode(uint8array);
7
  • 2
    By the way, this is available in Firefox by default: developer.mozilla.org/en-US/docs/Web/API/TextDecoder.decode
    – Joel
    Mar 12, 2014 at 22:37
  • 2
    Thumbs up for new APIs which are much better than weird workarounds! Oct 21, 2015 at 14:16
  • 3
    This won't work with all types of characters out there.
    – David
    Mar 30, 2016 at 11:45
  • 6
    npm install text-encoding, var textEncoding = require('text-encoding'); var TextDecoder = textEncoding.TextDecoder;. No thanks.
    – Evan Hu
    Nov 29, 2016 at 6:13
  • grumble ... if I have an existing arraybuffer I want to write a string into I guess I have to take the uint8array and copy it a 2nd time??
    – shaunc
    Dec 5, 2017 at 22:35
48

Blob is much slower than String.fromCharCode(null,array);

but that fails if the array buffer gets too big. The best solution I have found is to use String.fromCharCode(null,array); and split it up into operations that won't blow the stack, but are faster than a single char at a time.

The best solution for large array buffer is:

function arrayBufferToString(buffer){

    var bufView = new Uint16Array(buffer);
    var length = bufView.length;
    var result = '';
    var addition = Math.pow(2,16)-1;

    for(var i = 0;i<length;i+=addition){

        if(i + addition > length){
            addition = length - i;
        }
        result += String.fromCharCode.apply(null, bufView.subarray(i,i+addition));
    }

    return result;

}

I found this to be about 20 times faster than using blob. It also works for large strings of over 100mb.

2
  • 4
    We should go with this solution. As this solves one more use case than the accepted one
    – sam
    Jan 4, 2016 at 8:10
  • 1
    I get: "Uncaught INVALID: json decode: this isn't json!"
    – dicroce
    Jul 22, 2020 at 20:35
31

In case you have binary data in a string (obtained from nodejs + readFile(..., 'binary'), or cypress + cy.fixture(..., 'binary'), etc), you can't use TextEncoder. It supports only utf8. Bytes with values >= 128 are each turned into 2 bytes.

ES2015:

a = Uint8Array.from(s, x => x.charCodeAt(0))

Uint8Array(33) [2, 134, 140, 186, 82, 70, 108, 182, 233, 40, 143, 247, 29, 76, 245, 206, 29, 87, 48, 160, 78, 225, 242, 56, 236, 201, 80, 80, 152, 118, 92, 144, 48

s = String.fromCharCode.apply(null, a)

"ºRFl¶é(÷LõÎW0 Náò8ìÉPPv\0"

4
  • Thank you for the simplest solution! Nov 30, 2020 at 23:08
  • Any way to make the fromCharCode TypeScript ready? It complains that a is not a number[]
    – philk
    Oct 14, 2021 at 23:50
  • This should be the accepted answer! Oct 25, 2021 at 17:11
  • That's the only answer that worked for me when using pdf.js. Thanks! Apr 5 at 0:38
25

Based on the answer of gengkev, I created functions for both ways, because BlobBuilder can handle String and ArrayBuffer:

function string2ArrayBuffer(string, callback) {
    var bb = new BlobBuilder();
    bb.append(string);
    var f = new FileReader();
    f.onload = function(e) {
        callback(e.target.result);
    }
    f.readAsArrayBuffer(bb.getBlob());
}

and

function arrayBuffer2String(buf, callback) {
    var bb = new BlobBuilder();
    bb.append(buf);
    var f = new FileReader();
    f.onload = function(e) {
        callback(e.target.result)
    }
    f.readAsText(bb.getBlob());
}

A simple test:

string2ArrayBuffer("abc",
    function (buf) {
        var uInt8 = new Uint8Array(buf);
        console.log(uInt8); // Returns `Uint8Array { 0=97, 1=98, 2=99}`

        arrayBuffer2String(buf, 
            function (string) {
                console.log(string); // returns "abc"
            }
        )
    }
)
8
  • In arrayBuffer2String(), did you mean to call callback(...) instead of console.log()? Otherwise the callback argument goes unused. Mar 14, 2012 at 16:09
  • This looks like the way to go -- thanks genkev and Dennis. Seems kind of silly that there's no synchronous way to accomplish this, but what can you do...
    – kpozin
    Mar 30, 2012 at 3:12
  • JavaScript is single threaded. Therefore the FileReader is asynchronous for two reasons: (1) it will not block the execution of other JavaScript while loading a (huge) file (imagine a more complex application) and (2) it will not block the UI/Browser (common problem with long executing JS code). Plenty of APIs are asynchronous. Even in XMLHttpRequest 2 the synchronous is removed.
    – Dennis
    Apr 2, 2012 at 16:09
  • I was really hoping this would work for me, but the conversion from string to ArrayBuffer is not working reliably. I am making an ArrayBuffer with 256 values, and can turn that into a string with length 256. But then if I try to convert that back to an ArrayBuffer - depending on the contents of my initial ArrayBuffer - I'm getting out 376 elements. If you want to try to reproduce my problem, I'm treating my ArrayBuffer as a 16x16 grid in a Uint8Array, with values calculated as a[y * w + x] = (x + y) / 2 * 16; I've tried getBlob("x"), with many different mimetypes - no luck. May 6, 2012 at 13:40
  • 18
    BlobBuilder is deprecated in newer browsers. Change new BlobBuilder(); bb.append(buf); to new Blob([buf]), cast the ArrayBuffer in the second function to a UintArray via new UintArray(buf) (or whatever's appropriate for the underlying data type), and then get rid of the getBlob() calls. Finally, for cleanliness, rename bb to blob because it's not a BlobBuilder anymore.
    – sowbug
    Jun 12, 2012 at 21:56
21

Just

const buffer = thisReturnsBuffers();

const blob = new Blob([buffer], {type: 'text/plain; charset=utf-8'});

blob.text().then(text => console.log(text));

Or

const stringVal = "string here";

const blob = new Blob([stringVal], {type: 'text/plain; charset=utf-8'});

blob.arrayBuffer().then(buffer => console.log(buffer));

Why are you all making this so complicated?

4
  • I think this won't work in the case of string containing binary data? I think you'll need to put it in a uint8array first Sep 9, 2021 at 21:09
  • Ah, in the browser that would have already been stripped. Strings can't hold arbitrary bytes there. Is it different on Node? Sep 10, 2021 at 2:26
  • I'm not an expert but an example in the browser is the USVString you can receive from a blob. developer.mozilla.org/en-US/docs/Web/API/Blob/text Sep 22, 2021 at 22:54
  • 1
    If you're starting from a blob you probably shouldn't convert it to a string to then make a buffer, since blob.arrayBuffer is more direct. Sep 23, 2021 at 6:48
19

All the following is about getting binary strings from array buffers

I'd recommend not to use

var binaryString = String.fromCharCode.apply(null, new Uint8Array(arrayBuffer));

because it

  1. crashes on big buffers (somebody wrote about "magic" size of 246300 but I got Maximum call stack size exceeded error on 120000 bytes buffer (Chrome 29))
  2. it has really poor performance (see below)

If you exactly need synchronous solution use something like

var
  binaryString = '',
  bytes = new Uint8Array(arrayBuffer),
  length = bytes.length;
for (var i = 0; i < length; i++) {
  binaryString += String.fromCharCode(bytes[i]);
}

it is as slow as the previous one but works correctly. It seems that at the moment of writing this there is no quite fast synchronous solution for that problem (all libraries mentioned in this topic uses the same approach for their synchronous features).

But what I really recommend is using Blob + FileReader approach

function readBinaryStringFromArrayBuffer (arrayBuffer, onSuccess, onFail) {
  var reader = new FileReader();
  reader.onload = function (event) {
    onSuccess(event.target.result);
  };
  reader.onerror = function (event) {
    onFail(event.target.error);
  };
  reader.readAsBinaryString(new Blob([ arrayBuffer ],
    { type: 'application/octet-stream' }));
}

the only disadvantage (not for all) is that it is asynchronous. And it is about 8-10 times faster then previous solutions! (Some details: synchronous solution on my environment took 950-1050 ms for 2.4Mb buffer but solution with FileReader had times about 100-120 ms for the same amount of data. And I have tested both synchronous solutions on 100Kb buffer and they have taken almost the same time, so loop is not much slower the using 'apply'.)

BTW here: How to convert ArrayBuffer to and from String author compares two approaches like me and get completely opposite results (his test code is here) Why so different results? Probably because of his test string that is 1Kb long (he called it "veryLongStr"). My buffer was a really big JPEG image of size 2.4Mb.

16

(Update Please see the 2nd half of this answer, where I have (hopefully) provided a more complete solution.)

I also ran into this issue, the following works for me in FF 6 (for one direction):

var buf = new ArrayBuffer( 10 );
var view = new Uint8Array( buf );
view[ 3 ] = 4;
alert(Array.prototype.slice.call(view).join(""));

Unfortunately, of course, you end up with ASCII text representations of the values in the array, rather than characters. It still (should be) much more efficient than a loop, though. eg. For the example above, the result is 0004000000, rather than several null chars & a chr(4).

Edit:

After looking on MDC here, you may create an ArrayBuffer from an Array as follows:

var arr = new Array(23);
// New Uint8Array() converts the Array elements
//  to Uint8s & creates a new ArrayBuffer
//  to store them in & a corresponding view.
//  To get at the generated ArrayBuffer,
//  you can then access it as below, with the .buffer property
var buf = new Uint8Array( arr ).buffer;

To answer your original question, this allows you to convert ArrayBuffer <-> String as follows:

var buf, view, str;
buf = new ArrayBuffer( 256 );
view = new Uint8Array( buf );

view[ 0 ] = 7; // Some dummy values
view[ 2 ] = 4;

// ...

// 1. Buffer -> String (as byte array "list")
str = bufferToString(buf);
alert(str); // Alerts "7,0,4,..."

// 1. String (as byte array) -> Buffer    
buf = stringToBuffer(str);
alert(new Uint8Array( buf )[ 2 ]); // Alerts "4"

// Converts any ArrayBuffer to a string
//  (a comma-separated list of ASCII ordinals,
//  NOT a string of characters from the ordinals
//  in the buffer elements)
function bufferToString( buf ) {
    var view = new Uint8Array( buf );
    return Array.prototype.join.call(view, ",");
}
// Converts a comma-separated ASCII ordinal string list
//  back to an ArrayBuffer (see note for bufferToString())
function stringToBuffer( str ) {
    var arr = str.split(",")
      , view = new Uint8Array( arr );
    return view.buffer;
}

For convenience, here is a function for converting a raw Unicode String to an ArrayBuffer (will only work with ASCII/one-byte characters)

function rawStringToBuffer( str ) {
    var idx, len = str.length, arr = new Array( len );
    for ( idx = 0 ; idx < len ; ++idx ) {
        arr[ idx ] = str.charCodeAt(idx) & 0xFF;
    }
    // You may create an ArrayBuffer from a standard array (of values) as follows:
    return new Uint8Array( arr ).buffer;
}

// Alerts "97"
alert(new Uint8Array( rawStringToBuffer("abc") )[ 0 ]);

The above allow you to go from ArrayBuffer -> String & back to ArrayBuffer again, where the string may be stored in eg. .localStorage :)

Hope this helps,

Dan

2
  • 1
    I don't think this is an efficient method (in terms of time or space), and this is a very unusual way to store binary data.
    – kpozin
    Jun 19, 2012 at 17:39
  • 1
    What about using base64 encoding? Mar 21, 2015 at 11:44
16

Unlike the solutions here, I needed to convert to/from UTF-8 data. For this purpose, I coded the following two functions, using the (un)escape/(en)decodeURIComponent trick. They're pretty wasteful of memory, allocating 9 times the length of the encoded utf8-string, though those should be recovered by gc. Just don't use them for 100mb text.

function utf8AbFromStr(str) {
    var strUtf8 = unescape(encodeURIComponent(str));
    var ab = new Uint8Array(strUtf8.length);
    for (var i = 0; i < strUtf8.length; i++) {
        ab[i] = strUtf8.charCodeAt(i);
    }
    return ab;
}

function strFromUtf8Ab(ab) {
    return decodeURIComponent(escape(String.fromCharCode.apply(null, ab)));
}

Checking that it works:

strFromUtf8Ab(utf8AbFromStr('latinкирилицаαβγδεζηあいうえお'))
-> "latinкирилицаαβγδεζηあいうえお"
1
11

For node.js and also for browsers using https://github.com/feross/buffer

function ab2str(buf: Uint8Array) {
  return Buffer.from(buf).toString('base64');
}
function str2ab(str: string) {
  return new Uint8Array(Buffer.from(str, 'base64'))
}

Note: Solutions here didn't work for me. I need to support node.js and browsers and just serialize UInt8Array to a string. I could serialize it as a number[] but that occupies unnecessary space. With that solution I don't need to worry about encodings since it's base64. Just in case other people struggle with the same problem... My two cents

7

I found I had problems with this approach, basically because I was trying to write the output to a file and it was non encoded properly. Since JS seems to use UCS-2 encoding (source, source), we need to stretch this solution a step further, here's my enhanced solution that works to me.

I had no difficulties with generic text, but when it was down to Arab or Korean, the output file didn't have all the chars but instead was showing error characters

File output: ","10k unit":"",Follow:"Õ©íüY‹","Follow %{screen_name}":"%{screen_name}U“’Õ©íü",Tweet:"ĤüÈ","Tweet %{hashtag}":"%{hashtag} ’ĤüÈY‹","Tweet to %{name}":"%{name}U“xĤüÈY‹"},ko:{"%{followers_count} followers":"%{followers_count}…X \Ì","100K+":"100Ì tÁ","10k unit":"Ì è",Follow:"\°","Follow %{screen_name}":"%{screen_name} Ø \°X0",K:"œ",M:"1Ì",Tweet:"¸","Tweet %{hashtag}":"%{hashtag}

Original: ","10k unit":"万",Follow:"フォローする","Follow %{screen_name}":"%{screen_name}さんをフォロー",Tweet:"ツイート","Tweet %{hashtag}":"%{hashtag} をツイートする","Tweet to %{name}":"%{name}さんへツイートする"},ko:{"%{followers_count} followers":"%{followers_count}명의 팔로워","100K+":"100만 이상","10k unit":"만 단위",Follow:"팔로우","Follow %{screen_name}":"%{screen_name} 님 팔로우하기",K:"천",M:"백만",Tweet:"트윗","Tweet %{hashtag}":"%{hashtag}

I took the information from dennis' solution and this post I found.

Here's my code:

function encode_utf8(s) {
  return unescape(encodeURIComponent(s));
}

function decode_utf8(s) {
  return decodeURIComponent(escape(s));
}

 function ab2str(buf) {
   var s = String.fromCharCode.apply(null, new Uint8Array(buf));
   return decode_utf8(decode_utf8(s))
 }

function str2ab(str) {
   var s = encode_utf8(str)
   var buf = new ArrayBuffer(s.length); 
   var bufView = new Uint8Array(buf);
   for (var i=0, strLen=s.length; i<strLen; i++) {
     bufView[i] = s.charCodeAt(i);
   }
   return bufView;
 }

This allows me to save the content to a file without encoding problems.

How it works: It basically takes the single 8-byte chunks composing a UTF-8 character and saves them as single characters (therefore an UTF-8 character built in this way, could be composed by 1-4 of these characters). UTF-8 encodes characters in a format that variates from 1 to 4 bytes in length. What we do here is encoding the sting in an URI component and then take this component and translate it in the corresponding 8 byte character. In this way we don't lose the information given by UTF8 characters that are more than 1 byte long.

0
6

if you used huge array example arr.length=1000000 you can this code to avoid stack callback problems

function ab2str(buf) {
var bufView = new Uint16Array(buf);
var unis =""
for (var i = 0; i < bufView.length; i++) {
    unis=unis+String.fromCharCode(bufView[i]);
}
return unis
}

reverse function mangini answer from top

function str2ab(str) {
    var buf = new ArrayBuffer(str.length*2); // 2 bytes for each char
    var bufView = new Uint16Array(buf);
    for (var i=0, strLen=str.length; i<strLen; i++) {
        bufView[i] = str.charCodeAt(i);
    }
    return buf;
}
5

Well, here's a somewhat convoluted way of doing the same thing:

var string = "Blah blah blah", output;
var bb = new (window.BlobBuilder||window.WebKitBlobBuilder||window.MozBlobBuilder)();
bb.append(string);
var f = new FileReader();
f.onload = function(e) {
  // do whatever
  output = e.target.result;
}
f.readAsArrayBuffer(bb.getBlob());

Edit: BlobBuilder has long been deprecated in favor of the Blob constructor, which did not exist when I first wrote this post. Here's an updated version. (And yes, this has always been a very silly way to do the conversion, but it was just for fun!)

var string = "Blah blah blah", output;
var f = new FileReader();
f.onload = function(e) {
  // do whatever
  output = e.target.result;
};
f.readAsArrayBuffer(new Blob([string]));
5

The following is a working Typescript implementation:

bufferToString(buffer: ArrayBuffer): string {
    return String.fromCharCode.apply(null, Array.from(new Uint16Array(buffer)));
}

stringToBuffer(value: string): ArrayBuffer {
    let buffer = new ArrayBuffer(value.length * 2); // 2 bytes per char
    let view = new Uint16Array(buffer);
    for (let i = 0, length = value.length; i < length; i++) {
        view[i] = value.charCodeAt(i);
    }
    return buffer;
}

I've used this for numerous operations while working with crypto.subtle.

2
  • This was my favorite approach, however, how could you do a check if "data" is Buffer? Oct 15, 2021 at 22:48
  • If the data is a string. If it is UTF16 or a smaller encoding, it can be turned into an ArrayBuffer with this function. It doesn't mean that buffer will be useful for other things, such as changing into an image. If you want to know that, you'll have to look at what the image format is and sniff it. Most file types have metadata at the start.
    – N-ate
    Dec 17, 2021 at 19:54
5

Recently I also need to do this for one of my project so did a well research and got a result from Google's Developer community which states this in a simple manner:

For ArrayBuffer to String

function ab2str(buf) {
  return String.fromCharCode.apply(null, new Uint16Array(buf));
}
// Here Uint16 can be different like Uinit8/Uint32 depending upon your buffer value type.

For String to ArrayBuffer

function str2ab(str) {
  var buf = new ArrayBuffer(str.length*2); // 2 bytes for each char
  var bufView = new Uint16Array(buf);
  for (var i=0, strLen=str.length; i < strLen; i++) {
    bufView[i] = str.charCodeAt(i);
  }
  return buf;
}
//Same here also for the Uint16Array.

For more in detail reference you can refer this blog by Google.

1
4
  stringToArrayBuffer(byteString) {
    var byteArray = new Uint8Array(byteString.length);
    for (var i = 0; i < byteString.length; i++) {
      byteArray[i] = byteString.codePointAt(i);
    }
    return byteArray;
  }
  arrayBufferToString(buffer) {
    var byteArray = new Uint8Array(buffer);
    var byteString = '';
    for (var i = 0; i < byteArray.byteLength; i++) {
      byteString += String.fromCodePoint(byteArray[i]);
    }
    return byteString;
  }
1
  • 2
    this code is buggy if string contains unicode characters. example: arrayBufferToString(stringToArrayBuffer('🐴'))==='44'
    – xmcp
    May 20, 2020 at 8:36
4

Let's say you have an arrayBuffer binaryStr:

let text = String.fromCharCode.apply(null, new Uint8Array(binaryStr));

and then you assign the text to the state.

3

See here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays/StringView (a C-like interface for strings based upon the JavaScript ArrayBuffer interface)

1
  • 3
    That code is under the GPLv3. I think it's pretty unprofessional of Mozilla to even blend that code with their standards-compliant documentation.
    – user239558
    Oct 30, 2013 at 9:03
3

I used this and works for me.

function arrayBufferToBase64( buffer ) {
    var binary = '';
    var bytes = new Uint8Array( buffer );
    var len = bytes.byteLength;
    for (var i = 0; i < len; i++) {
        binary += String.fromCharCode( bytes[ i ] );
    }
    return window.btoa( binary );
}



function base64ToArrayBuffer(base64) {
    var binary_string =  window.atob(base64);
    var len = binary_string.length;
    var bytes = new Uint8Array( len );
    for (var i = 0; i < len; i++)        {
        bytes[i] = binary_string.charCodeAt(i);
    }
    return bytes.buffer;
}
0
3

Yes:

const encstr = (`TextEncoder` in window) ? new TextEncoder().encode(str) : Uint8Array.from(str, c => c.codePointAt(0));
3

After playing with mangini's solution for converting from ArrayBuffer to String - ab2str (which is the most elegant and useful one I have found - thanks!), I had some issues when handling large arrays. More specefivally, calling String.fromCharCode.apply(null, new Uint16Array(buf)); throws an error:

arguments array passed to Function.prototype.apply is too large.

In order to solve it (bypass) I have decided to handle the input ArrayBuffer in chunks. So the modified solution is:

function ab2str(buf) {
   var str = "";
   var ab = new Uint16Array(buf);
   var abLen = ab.length;
   var CHUNK_SIZE = Math.pow(2, 16);
   var offset, len, subab;
   for (offset = 0; offset < abLen; offset += CHUNK_SIZE) {
      len = Math.min(CHUNK_SIZE, abLen-offset);
      subab = ab.subarray(offset, offset+len);
      str += String.fromCharCode.apply(null, subab);
   }
   return str;
}

The chunk size is set to 2^16 because this was the size I have found to work in my development landscape. Setting a higher value caused the same error to reoccur. It can be altered by setting the CHUNK_SIZE variable to a different value. It is important to have an even number.

Note on performance - I did not make any performance tests for this solution. However, since it is based on the previous solution, and can handle large arrays, I see no reason why not to use it.

1
  • you can use typedarray.subarray to get a chunk at specified position and size, this is what i do to read headers off binary formats in js
    – Nikos M.
    Nov 5, 2014 at 15:23
3

ArrayBuffer -> Buffer -> String(Base64)

Change ArrayBuffer to Buffer and then to String.

Buffer.from(arrBuffer).toString("base64");
2
2

The "native" binary string that atob() returns is a 1-byte-per-character Array.

So we shouldn't store 2 byte into a character.

var arrayBufferToString = function(buffer) {
  return String.fromCharCode.apply(null, new Uint8Array(buffer));
}

var stringToArrayBuffer = function(str) {
  return (new Uint8Array([].map.call(str,function(x){return x.charCodeAt(0)}))).buffer;
}
2

For me this worked well.

  static async hash(message) {
    const data = new TextEncoder().encode(message);
    const hashBuffer = await crypto.subtle.digest('SHA-256', data)
    const hashArray = Array.from(new Uint8Array(hashBuffer))
    const hashHex = hashArray.map((b) => b.toString(16).padStart(2, '0')).join('')
    return hashHex
  }
1

I'd recommend NOT using deprecated APIs like BlobBuilder

BlobBuilder has long been deprecated by the Blob object. Compare the code in Dennis' answer — where BlobBuilder is used — with the code below:

function arrayBufferGen(str, cb) {

  var b = new Blob([str]);
  var f = new FileReader();

  f.onload = function(e) {
    cb(e.target.result);
  }

  f.readAsArrayBuffer(b);

}

Note how much cleaner and less bloated this is compared to the deprecated method... Yeah, this is definitely something to consider here.

1
  • I mean, yes, but that Blob constructor wasn't really usable back in 2012 ;)
    – gengkev
    Oct 16, 2014 at 3:31
0
var decoder = new TextDecoder ();
var string = decoder.decode (arrayBuffer);

See https://developer.mozilla.org/en-US/docs/Web/API/TextDecoder/decode

0

From emscripten:

function stringToUTF8Array(str, outU8Array, outIdx, maxBytesToWrite) {
  if (!(maxBytesToWrite > 0)) return 0;
  var startIdx = outIdx;
  var endIdx = outIdx + maxBytesToWrite - 1;
  for (var i = 0; i < str.length; ++i) {
    var u = str.charCodeAt(i);
    if (u >= 55296 && u <= 57343) {
      var u1 = str.charCodeAt(++i);
      u = 65536 + ((u & 1023) << 10) | u1 & 1023
    }
    if (u <= 127) {
      if (outIdx >= endIdx) break;
      outU8Array[outIdx++] = u
    } else if (u <= 2047) {
      if (outIdx + 1 >= endIdx) break;
      outU8Array[outIdx++] = 192 | u >> 6;
      outU8Array[outIdx++] = 128 | u & 63
    } else if (u <= 65535) {
      if (outIdx + 2 >= endIdx) break;
      outU8Array[outIdx++] = 224 | u >> 12;
      outU8Array[outIdx++] = 128 | u >> 6 & 63;
      outU8Array[outIdx++] = 128 | u & 63
    } else {
      if (outIdx + 3 >= endIdx) break;
      outU8Array[outIdx++] = 240 | u >> 18;
      outU8Array[outIdx++] = 128 | u >> 12 & 63;
      outU8Array[outIdx++] = 128 | u >> 6 & 63;
      outU8Array[outIdx++] = 128 | u & 63
    }
  }
  outU8Array[outIdx] = 0;
  return outIdx - startIdx
}

Use like:

stringToUTF8Array('abs', new Uint8Array(3), 0, 4);
0

Use splat unpacking instead of loops:

arrbuf = new Uint8Array([104, 101, 108, 108, 111])
text = String.fromCharCode(...arrbuf)
console.log(text)

For substrings arrbuf.slice() can be employed.

1
  • It won't work for non ascii charcters (utf-8 bytes)
    – crayze
    Aug 2 at 11:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.