Transforming Uint8Array into BigInt using Javascript

I've come across 3 different ways to convert a Uint8Array to BigInt, but each method seems to produce varying results. Can someone clarify which approach is correct and recommended?

  1. Utilizing the bigint-conversion library. The function bigintConversion.bufToBigint() can be used to convert a buffer into a BigInt value, with the implementation shown below:
export function bufToBigint (buf: ArrayBuffer|TypedArray|Buffer): bigint {
  let bits = 8n
  if (ArrayBuffer.isView(buf)) bits = BigInt(buf.BYTES_PER_ELEMENT * 8)
  else buf = new Uint8Array(buf)

  let ret = 0n
  for (const i of (buf as TypedArray|Buffer).values()) {
    const bi = BigInt(i)
    ret = (ret << bits) + bi
  }
  return ret
}
  1. Using DataView:
let view = new DataView(arr.buffer, 0);
let result = view.getBigUint64(0, true);
  1. Implementing a FOR loop:
let result = BigInt(0);
for (let i = arr.length - 1; i >= 0; i++) {
  result = result * BigInt(256) + BigInt(arr[i]);
}

I'm uncertain on the accuracy of these methods since they yield divergent outcomes, albeit providing results.

Answer №1

I am open to both BE and LE, but I would like to understand why these three methods yield different outcomes.

The discrepancies in the results stem from their usage of varying endianness.

Let's transform your code snippets into an executable format for comparison:

let source_array = new Uint8Array([
    0xff, 0xee, 0xdd, 0xcc, 0xbb, 0xaa, 0x99, 0x88, 
    0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11]);
let buffer = source_array.buffer;

function method1(buf) {
  let bits = 8n
  if (ArrayBuffer.isView(buf)) {
    bits = BigInt(buf.BYTES_PER_ELEMENT * 8)
  } else {
    buf = new Uint8Array(buf)
  }

  let ret = 0n
  for (const i of buf.values()) {
    const bi = BigInt(i)
    ret = (ret << bits) + bi
  }
  return ret
}

function method2(buf) {
  let view = new DataView(buf, 0);
  return view.getBigUint64(0, true);
}

function method3(buf) {
  let arr = new Uint8Array(buf);
  let result = BigInt(0);
  for (let i = arr.length - 1; i >= 0; i--) {
    result = result * BigInt(256) + BigInt(arr[i]);
  }
  return result;
}

console.log(method1(buffer).toString(16));
console.log(method2(buffer).toString(16));
console.log(method3(buffer).toString(16));

Please note that a bug fix has been applied to "method3": change i++) to i-- at the end of your loop statement.

Here are the outputs of each method:

"method1" output: ffeeddccbbaa998877665544332211. This method represents big-endian conversion without size limitations.

"method2" output: 8899aabbccddeeff. This method involves little-endian conversion limited to 64 bits. Changing the second argument in getBigUint64 can switch between little-endian and big-endian behavior.

"method3" output: 112233445566778899aabbccddeeff. This method depicts little-endian conversion without size restrictions. Adjusting the direction of the loop will yield big-endian behavior similar to method1.

To select the appropriate method, consider factors such as the endianness of incoming arrays, size limits on BigInts, and performance requirements.

If you have control over the entire process of converting BigInts to Uint8Arrays and vice versa, using hexadecimal strings could be a simpler and faster alternative:

function serialize(bigint) {
  return "0x" + bigint.toString(16);
}
function deserialize(serialized_bigint) {
  return BigInt(serialized_bigint);
}

Answer №2

If you are looking for a way to store large integers without being limited by base64 or 128, while also handling negative numbers, then this solution may be what you need...

function encode(n) {
  let hex, bytes

  // shifting all numbers one step to the left and performing XOR if less than 0
  n = (n << 1n) ^ (n < 0n ? -1n : 0n)

  // converting to hexadecimal
  hex = n.toString(16)
  // padding if necessary
  if (hex.length % 2) hex = '0' + hex

  // converting hex to bytes
  bytes = hex.match(/.{1,2}/g).map(byte => parseInt(byte, 16))

  return bytes
}

function decode(bytes) {
  let hex, n

  // converting bytes back into hex
  hex = bytes.map(e => e.toString(16).padStart(2, 0)).join('')

  // Converting hex to BigInt
  n = BigInt(`0x`+hex)

  // Shifting all numbers to right and XOR if the first bit was signed
  n = (n >> 1n) ^ (n & 1n ? -1n : 0n)

  return n
}

const input = document.querySelector('input')
input.oninput = () => {
  console.clear()
  const bytes = encode(BigInt(input.value))
  // TODO: Save or transmit these bytes
  // new Uint8Array(bytes)
  console.log(bytes.join(','))

  const n = decode(bytes)
  console.log(n.toString(10)+'n') // because SO cannot render bigints...
}
input.oninput()
<input type="number" value="-39287498324798237498237498273323423" style="width: 100%">

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Divs are not being organized into rows correctly due to issues with Bootstrap styling

I have implemented Bootstrap in my Angular application. The stylesheet link is included in my Index.html file as follows: <link rel="stylesheet" href="../node_modules/bootstrap/dist/css/bootstrap.css"> In addition to that, I have listed Bootstrap a ...

Using React to retrieve an object from a helper method through a GET request

I am in the process of turning a frequently used function in my application into a helper method that can be imported wherever it is needed. However, I am facing an issue where I am not getting a response from the function when calling it. I need to figure ...

Safari has trouble with AJAX cross-origin requests, while Chrome and Firefox handle them without issue

I am developing a Shopify app that utilizes script tags and requires an ajax call to our server to retrieve necessary information about the shop. While everything seemed to be functioning correctly, my colleague pointed out that it was not working on his i ...

Is it possible to manipulate a modal within a controller by utilizing a certain attribute in HTML, based on specific conditions (without relying on any bootstrap services), using AngularJS?

Within my application, I have a modal that is triggered by clicking on a button (ng-click) based on certain conditions. Here is the HTML code: <button type="button" class="btn btn-outlined" ng-click="vm.change()" data-modal-target="#add-save-all-alert ...

What's the process for creating a Java object in PHP and utilizing it in JavaScript?

I am looking to create an object on a PHP page and send it as a response through an AJAX call to be used as a JavaScript object on the response page. This type of object is what I need to generate and pass along. var areaChartData = { labels ...

What is the best method for retrieving an item from localstorage?

Seeking advice on how to retrieve an item from local storage in next.js without causing a page rerender. Here is the code snippet I am currently using: import { ThemeProvider } from "@material-ui/core"; import { FC, useEffect, useState } from "react"; i ...

Scrolling will only function properly on this page if you refresh it

I have a setup where buttons on my first page lead to specific elements on the second page. To achieve this, I pass the element IDs in the URL like so: mysite.com/secondpage/:promo1(/2/3, depending on the button clicked.) Upon landing on the second page, ...

Prevent duplicate key errors when performing bulk insert operations with MongoDB

Is there a way to perform a bulk insert and proceed even if a duplicate key error occurs? I have a collection with a unique index on the id field (not _id) and some existing data. I need to add new data to the collection while skipping any documents that ...

Utilize the power of jQuery for form validation by combining the errorPlacement and showErrors functions

I am currently attempting to implement validation using the Jquery .validate plugin. Unfortunately, I have encountered an issue where I am unable to utilize both the errorPlacement and showErrors methods simultaneously. If you'd like to see a demons ...

What are some strategies for ensuring that Plotly JS can adapt its height and width to fit the entire div dynamically, without relying on fixed pixel dimensions?

Is there a way to ensure that Plotly automatically adjusts to the size of the container #myDiv without any noticeable delay when the top button is clicked? This code snippet demonstrates a high delay: var z = [], steps = [], i; for (i = 0; i < 500; i+ ...

Having trouble getting the Google motion chart to work with asynchronous JSON requests

I have been using the code below to make a request for a JSON file and then parsing it. google.load('visualization', '1', {packages: ['controls', "motionchart", "table"]}); google.setOnLoadCallback(function(){ createTable($(& ...

Guide to capturing and playing audio on iOS6 with HTML5

Our team is currently working on a web application using HTML5 and Javascript. We are facing a challenge with implementing voice recording functionality, as the Wami Api we tried is not compatible with iPad due to its use of flash for recording. Could yo ...

Explore in MegaMenu Pop-up

At my workplace, the internal web portal features a MegaMenu with a popup menu that includes a Search input field. The issue I am encountering is that when a user starts typing in the search bar and moves the mouse off of the megamenu, it disappears. It ...

Instructions for linking a webdriver script to an existing chrome tab

Currently, I am utilizing webDriver in conjunction with JavaScript to automate the extraction of information from a website. To prevent the appearance of the login screen, it is essential for the script to operate within an existing window. Despite extens ...

Click to remove the ellipsis from the backbone

Some Background Info I am working on creating a feed similar to Twitter where each row expands on click to show more information. The data is fetched from a JSON file sent from the backend to the frontend, and I am using Backbone.js for rendering. My fee ...

NodeJS - The server returns a 404 error before ultimately displaying the requested page

I'm having trouble with my nodeJS application. When I make an asynchronous call using ajax, the server first responds with a 404 error before loading the page. The application functions properly, but I keep receiving repetitive logs stating "Can' ...

Using TypeScript with Redux for Form Validation in FieldArray

My first time implementing a FieldArray from redux-form has been quite a learning experience. The UI functions properly, but there seems to be some performance issues that I need to investigate further. Basically, the concept is to click an ADD button to i ...

Adding a Material UI Tooltip to the header name of a Material UI Datagrid - a step-by-step guide!

I've nearly completed my initial project, but the client is requesting that I include labels that appear when hovering over specific datagrid cells. Unfortunately, I haven't been able to find any solutions on Google for adding a Material UI Tool ...

Simple server using node.js and express to host an HTML file and associated resources

I am currently experimenting with frontend development and need a basic web server to quickly start projects and serve files. Specifically, I have one index.html file along with some css/js/img files. I decided to work with Node.js and Express for this pur ...

Experiencing difficulties when trying to upload images using multer in an Express application with Node.js

I have been using multer to successfully upload images into a folder within my project. Despite not encountering any errors while using multer, I am facing an issue where the upload is not functioning as expected in my project, although it works perfectly ...