86

I'm going to be storing a large array of byte values (most likely over a million) in Javascript. If I use a normal array with normal numbers, that will take 8 MB, because numbers are stored as IEEE doubles, but if I can store it as bytes, it will be only 1 MB.

I'd like to avoid wasting that much space for obvious reasons. Is there a way to store bytes as bytes instead of doubles? Browser compatibility isn't an issue for me, as long as it works in Chrome. This is in HTML5, if that makes a difference.

6
  • 12
    Typed arrays.
    – Pointy
    Sep 8, 2012 at 15:28
  • @Pointy: typed array for bytes? khronos.org/registry/typedarray/specs/latest/#7 could you please elaborate? i would like to understand that
    – naveen
    Sep 8, 2012 at 16:02
  • 7
    Re: "as long as it works in Chrome" -- I'm tempted to give you a -1 for that (I haven't but I'm tempted), because that's the same attitude that left us with a legacy of sites that only work in IE6. If what you want can only be done in Chrome, then so be it, but don't limit your site. There are several browsers out there, and they all work with HTML5. Write your site for as broad an audience as possible.
    – Spudley
    Sep 8, 2012 at 18:12
  • 20
    @Spudley It's not intended to be a public site. It's written for an audience of just me. Sep 8, 2012 at 18:16

4 Answers 4

90

By using typed arrays, you can store arrays of these types:

Type Value Range Size(bytes)
Int8Array -128 to 127 1
Uint8Array 0 to 255 1
Uint8ClampedArray 0 to 255 1
Int16Array -32768 to 32767 2
Uint16Array 0 to 65535 2
Int32Array -2147483648 to 2147483647 4
Uint32Array 0 to 4294967295 4
Float32Array -3.4E38 to 3.4E38 4
Float64Array -1.8E308 to 1.8E308 8
BigInt64Array -2^63 to 2^63 - 1 8
BigUint64Array 0 to 2^64 - 1 8

Demo in Stack Snippets & JSFiddle

var array = new Uint8Array(100);
array[42] = 10;
console.log(array[42]);

2
  • 11
    Yes! That cut the time generating an array of 800,000+ elements from 2.4 seconds to 0.12 seconds. Using Float32Array makes an amazing difference! Nov 5, 2013 at 19:27
  • 1
    I'm having issues converting a Blob that's returned by a fetch() promise to a C# byte[]. Do you have any wisdom on what approach I should take.
    – Reahreic
    Mar 2, 2021 at 21:05
19
var array = new Uint8Array(100);    
array[10] = 256;
array[10] === 0 // true

I verified in firefox and chrome, its really an array of bytes :

var array = new Uint8Array(1024*1024*50);  // allocates 50MBytes
2
  • How does 256 === 0? Shouldn't it be 256? Dec 31, 2015 at 22:31
  • 32
    As array is an array of uint8, 256 overflows a byte and makes the value equals to 0
    – gsempe
    Jan 12, 2016 at 7:04
2

You could store the data in an array of strings of some large fixed size. It should be efficient to access any particular character in that array of strings, and to treat that character as a byte.

It would be interesting to see the operations you want to support, perhaps expressed as an interface, to make the question more concrete.

1

I wanted a more exact and useful answer to this question. Here's the real answer (adjust accordingly if you want a byte array specifically; obviously the math will be off by a factor of 8 bits : 1 byte):

class BitArray {
  constructor(bits = 0) {
    this.uints = new Uint32Array(~~(bits / 32));
  }

  getBit(bit) {
    return (this.uints[~~(bit / 32)] & (1 << (bit % 32))) != 0 ? 1 : 0;
  }

  assignBit(bit, value) {
    if (value) {
      this.uints[~~(bit / 32)] |= (1 << (bit % 32));
    } else {
      this.uints[~~(bit / 32)] &= ~(1 << (bit % 32));
    }
  }

  get size() {
    return this.uints.length * 32;
  }

  static bitsToUints(bits) {
    return ~~(bits / 32);
  }
}

Usage:

let bits = new BitArray(500);
for (let uint = 0; uint < bits.uints.length; ++uint) {
  bits.uints[uint] = 457345834;
}
for (let bit = 0; bit < 50; ++bit) {
  bits.assignBit(bit, 1);
}
str = '';
for (let bit = bits.size - 1; bit >= 0; --bit) {
  str += bits.getBit(bit);
}
str;

Output:

"00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000111111111111111111
 11111111111111111111111111111111"

Note: This class is really slow to e.g. assign bits (i.e. ~2s per 10 million assignments) if it's created as a global variable, at least in the Firefox 76.0 Console on Linux... If, on the other hand, it's created as a variable (i.e. let bits = new BitArray(1e7);), then it's blazingly fast (i.e. ~300ms per 10 million assignments)!


For more info, see here:

Note that I used Uint32Array because there's no way to directly have a bit/byte array (that you can interact with directly) and because even though there's a BigUint64Array, JS only supports 32 bits:

Bitwise operators treat their operands as a sequence of 32 bits

...

The operands of all bitwise operators are converted to...32-bit integers

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.