Transforming Uint8Array into BigInt using Javascript

I've come across 3 different ways to convert a Uint8Array to BigInt, but each method seems to produce varying results. Can someone clarify which approach is correct and recommended?

  1. Utilizing the bigint-conversion library. The function bigintConversion.bufToBigint() can be used to convert a buffer into a BigInt value, with the implementation shown below:
export function bufToBigint (buf: ArrayBuffer|TypedArray|Buffer): bigint {
  let bits = 8n
  if (ArrayBuffer.isView(buf)) bits = BigInt(buf.BYTES_PER_ELEMENT * 8)
  else buf = new Uint8Array(buf)

  let ret = 0n
  for (const i of (buf as TypedArray|Buffer).values()) {
    const bi = BigInt(i)
    ret = (ret << bits) + bi
  }
  return ret
}
  1. Using DataView:
let view = new DataView(arr.buffer, 0);
let result = view.getBigUint64(0, true);
  1. Implementing a FOR loop:
let result = BigInt(0);
for (let i = arr.length - 1; i >= 0; i++) {
  result = result * BigInt(256) + BigInt(arr[i]);
}

I'm uncertain on the accuracy of these methods since they yield divergent outcomes, albeit providing results.

Answer №1

I am open to both BE and LE, but I would like to understand why these three methods yield different outcomes.

The discrepancies in the results stem from their usage of varying endianness.

Let's transform your code snippets into an executable format for comparison:

let source_array = new Uint8Array([
    0xff, 0xee, 0xdd, 0xcc, 0xbb, 0xaa, 0x99, 0x88, 
    0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11]);
let buffer = source_array.buffer;

function method1(buf) {
  let bits = 8n
  if (ArrayBuffer.isView(buf)) {
    bits = BigInt(buf.BYTES_PER_ELEMENT * 8)
  } else {
    buf = new Uint8Array(buf)
  }

  let ret = 0n
  for (const i of buf.values()) {
    const bi = BigInt(i)
    ret = (ret << bits) + bi
  }
  return ret
}

function method2(buf) {
  let view = new DataView(buf, 0);
  return view.getBigUint64(0, true);
}

function method3(buf) {
  let arr = new Uint8Array(buf);
  let result = BigInt(0);
  for (let i = arr.length - 1; i >= 0; i--) {
    result = result * BigInt(256) + BigInt(arr[i]);
  }
  return result;
}

console.log(method1(buffer).toString(16));
console.log(method2(buffer).toString(16));
console.log(method3(buffer).toString(16));

Please note that a bug fix has been applied to "method3": change i++) to i-- at the end of your loop statement.

Here are the outputs of each method:

"method1" output: ffeeddccbbaa998877665544332211. This method represents big-endian conversion without size limitations.

"method2" output: 8899aabbccddeeff. This method involves little-endian conversion limited to 64 bits. Changing the second argument in getBigUint64 can switch between little-endian and big-endian behavior.

"method3" output: 112233445566778899aabbccddeeff. This method depicts little-endian conversion without size restrictions. Adjusting the direction of the loop will yield big-endian behavior similar to method1.

To select the appropriate method, consider factors such as the endianness of incoming arrays, size limits on BigInts, and performance requirements.

If you have control over the entire process of converting BigInts to Uint8Arrays and vice versa, using hexadecimal strings could be a simpler and faster alternative:

function serialize(bigint) {
  return "0x" + bigint.toString(16);
}
function deserialize(serialized_bigint) {
  return BigInt(serialized_bigint);
}

Answer №2

If you are looking for a way to store large integers without being limited by base64 or 128, while also handling negative numbers, then this solution may be what you need...

function encode(n) {
  let hex, bytes

  // shifting all numbers one step to the left and performing XOR if less than 0
  n = (n << 1n) ^ (n < 0n ? -1n : 0n)

  // converting to hexadecimal
  hex = n.toString(16)
  // padding if necessary
  if (hex.length % 2) hex = '0' + hex

  // converting hex to bytes
  bytes = hex.match(/.{1,2}/g).map(byte => parseInt(byte, 16))

  return bytes
}

function decode(bytes) {
  let hex, n

  // converting bytes back into hex
  hex = bytes.map(e => e.toString(16).padStart(2, 0)).join('')

  // Converting hex to BigInt
  n = BigInt(`0x`+hex)

  // Shifting all numbers to right and XOR if the first bit was signed
  n = (n >> 1n) ^ (n & 1n ? -1n : 0n)

  return n
}

const input = document.querySelector('input')
input.oninput = () => {
  console.clear()
  const bytes = encode(BigInt(input.value))
  // TODO: Save or transmit these bytes
  // new Uint8Array(bytes)
  console.log(bytes.join(','))

  const n = decode(bytes)
  console.log(n.toString(10)+'n') // because SO cannot render bigints...
}
input.oninput()
<input type="number" value="-39287498324798237498237498273323423" style="width: 100%">

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What could be causing a functional component's child component to be using stale props?

I am currently working with Next JS, but the process is similar. I have refined the code and eliminated irrelevant parts. My goal is to create a form where new fields (child components) can be added dynamically. The default setting will be 1 field, with a ...

How can I make TypeScript properly export function names for closure-compiler?

Here is the TypeScript code I am working with: namespace CompanyName.HtmlTools.Cookie { export function eraseCookie(name:string, path:string) { createCookie(name, "", path, -1); } export function readCookie(name:string) { ...

Unsure about module loading with system.js and navigating Typescript

I am currently in the process of transitioning an ASP.Net MVC application to Angular2, and I've encountered some perplexing behavior that I can't seem to grasp. Within my Angular2 app, I have a separate Layoutview that allows me to switch betwee ...

Guide to incorporating external code in InversifyJS without direct control

I'm wondering if it's feasible to add classes that are not editable. Inversify seems to rely heavily on annotations and decorators, but I'm curious if there is an alternative method. ...

What is the best way to implement rate limiting or throttling on a Strapi API?

Our company relies on a simple strapi API implemented in node.js and hosted on Heroku. Despite our efforts, we have not been able to find a solution to implement rate limiting, as it appears that Heroku does not offer throttling add-ons and strapi lacks bu ...

Javascript's second element does not trigger a click event with similar behavior

I'm currently facing an issue with displaying and hiding notification elements based on user interaction. My goal is to have multiple popup elements appear when the page loads. Then, when a user clicks the ".alert-close" element within one of the popu ...

Do we need a peer dependency specifically for TypeScript typings or is it optional?

My TypeScript library includes a React component, and one of the optional features allows users to pass an instance of a Redux store as a prop for Redux integration. <Component reduxStore={store}></Component> Since this feature is optional, I ...

Conditional Skipping of Lines in Node Line Reader: A Step-by-Step Guide

I am currently in the process of developing a project that involves using a line reader to input credit card numbers into a validator and identifier. If I input 10 numbers from four different credit card companies, I want to filter out the numbers from thr ...

Troubleshooting TypeScript: Issues with Object.assign and inheritance

After successfully using the code within an Angular project, I decided to switch to React only to find that the code is now producing unexpected results. class A { constructor(...parts: Partial<A>[]) { Object.assign(this, ...parts); } } cla ...

Switching Div Elements Created by PHP

Utilizing MySQL, I am fetching data to dynamically generate nested div tags in a hierarchical structure. This structure consists of div tags within div tags within div tags, all uniquely identified through PHP-generated ids: <div class="holder"> ...

Troubleshooting a Node.js problem with variable scope

I'm working on a nodejs route that downloads a URL as an mp3 using npm-youtube-dl. I have a download directory being monitored with chokidar for new files, and once a file is added, I save the file link. After the download completes, a function is cal ...

One way to determine whether .ajax is using Get or POST is to check the type parameter

I have a query: $.ajax({ url: "http://twitter.com/status/user_timeline/treason.json?count=10&callback=?", success: function (data, textStatus, jqXHR) { }, error: function (jqXHR, textStatus, errorThrown ...

Encountering Errors with Angular JS Following Update from Version 1.1.0 to 1.1.1

After upgrading, I noticed that the ng-repeat function is taking significantly longer to load and is attempting to display additional content boxes without serving the actual content provided by $resource. I have pinpointed the issue to the update from ve ...

Auth0 encountering issues retrieving ID token and user metadata

Currently in the process of integrating Auth0 into a Vue.js/Node.js application, I have successfully enabled user registration and login functionality (to /callback). Although the manual addition of data to the user metadata section is functional at this s ...

Exploring the values of a JavaScript promise while iterating through a for loop

I find myself wandering in the land of possibilities and would greatly appreciate some direction. After spending 2-3 hours scouring through countless SO questions and documentation related to my current predicament, I still seem to be missing the mark. Ove ...

Please refrain from submitting the form until the slow AJAX jQuery process has finished

My form is experiencing a delay of almost 4 seconds due to the Ajax jQuery I am using, which creates fields within the form. This delay causes some users to submit the form before the necessary fields are created. I need a way to prevent the form from bein ...

What is the most effective approach for annotating TypeScript abstract classes that are dynamically loaded?

I am in the process of developing a library that allows for the integration of external implementations, and I am exploring the optimal approach to defining types for these implementations. Illustration abstract class Creature { public abstract makeN ...

How to add unique elements to an array in Angular without any duplicates

I need help with pushing elements into an array and decrementing the count of it without duplicates in angular. Any assistance would be greatly appreciated ...

Ways to remove a dynamic field with jquery

I have developed a script that allows me to add dynamic fields and delete them as needed. However, I am facing an issue where I cannot delete the first element with the "el" class in my script because it removes all elements within the "input_fields_cont ...

What is the process for entering a value into mysql based on the date?

Seeking assistance with a specific coding challenge. The task at hand involves inputting a date, such as '2018-05-08', and based on the user's input, storing the value in different columns of a database table. For instance, if the date is &a ...