Differentiating between binary and text formats based on the HTTP Content-Type header

I am currently developing code to fetch data from various web resources using HTTP/HTTPS in a Node.js environment. My goal is to return the content as a string for text data and as a Buffer for binary data.

It is evident that any data starting with text, such as text/html, should be treated as text data and returned as a string, utilizing the appropriate character encoding if specified (e.g., text/html; charset=utf-8). Additionally, the presence of an explicit charset definition indicates that the content is text rather than binary, regardless of MIME type.

Based on my analysis, most content falls under the category of binary data. Audio and video formats are typically binary, as are most image types except for image/svg+xml. Generally speaking, most application/... types are considered binary, although there are exceptions like application/json.

Does the following function effectively determine whether the content is binary? Are there any significant exceptions that I may have overlooked?

function isBinary(contentType: string): boolean {
  let $: string[];

  if (/;\s*charset\s*=/i.test(contentType))
    return false;

  // Remove anything other than MIME type.
  contentType = contentType.replace(/;.*$/, '').trim();

  if (/^text\//i.test(contentType) || /\+xml$/i.test(contentType))
    return false;
  else if (($ = /^application\/(.+)/i.exec(contentType)))
    return !/^(javascript|ecmascript|json|ld\+json|rtf)$/i.test($[1]);
  else
    return true;
}

Answer №1

Checking whether the fetched data from a URL is text or binary can be done using istextorbinary.

For instance, in a lambda function, you could do:

const fetch = require('node-fetch');
const { isText, isBinary, getEncoding } = require('istextorbinary');
module.exports.handler = async (event, context, callback) => {
.
.
        const customUrl = 'www.example.com';
        const url = `https://${customUrl}${event.path}`;

        // Configure fetch parameters based on HTTP method
        var params = {};
        if (event.httpMethod === 'GET' || event.httpMethod === 'HEAD') {
            params = {
                method: event.httpMethod,
                headers: customRequestHeader
            };
        } else {
            params = {
                method: event.httpMethod,
                headers: customRequestHeader,
                body: JSON.stringify(parsedbody)
            };
        }

        console.debug('request params: ' + JSON.stringify(params));

        // Fetch URL with specified parameters
        const response = await fetch(url, params);
        var textResponse = await response.buffer();

        var isBase64EncodedValue = false;  
        if ( isBinary(null, textResponse) ) {
            console.log('textResponse is a binary blob - setting isBase64Encoded to true on returnResponse');
            isBase64EncodedValue = true;
            console.log('isBase64EncodedValue in returnResponse is: ' + isBase64EncodedValue);
            // Convert binary data to base64 encoding
            textResponse = textResponse.toString('base64');
            console.log('When isBase64EncodedValue is true, textResponse is: ' + textResponse);
        } else {
            console.log('textResponse is not a binary blob - setting isBase64Encoded to false on returnResponse');
            isBase64EncodedValue = false;
            console.log('isBase64EncodedValue in returnResponse is: ' + isBase64EncodedValue);
            // Convert text data to UTF-8 encoding
            textResponse = textResponse.toString('utf8');
            console.log('When isBase64EncodedValue is false, textResponse is: ' + textResponse);
        }

.
.
};

Have you implemented this function successfully? Feel free to share your final code!

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

How can I ensure that a user variable stored in an Angular6 service remains defined and accessible from other components?

Currently, I am working on an Angular app and facing a challenge. After receiving a user variable from an asynchronous call to Firestore Cloud, I noticed that the variable is successfully set (verified using console.log()). However, when I navigate between ...

Sending information to components in Angular using Router

Should I pass data through the angular router to a component, or is it better to use a service? Currently, the component is receiving data in the following way: this.account = activatedRoute.snapshot.data.account ...

Using decorators in TypeScript, can we define new class attributes?

Here is an example code snippet: function Getter(target: any, key: string): void { let getter = () => this[key]; /* create "foobar" property from "_foobar" */ Object.defineProperty(target, removeUnderscores(key), { get: getter, enumerabl ...

Sorting through a list of strings by checking for a specific character within each string

After spending years dabbling in VBA, I am now delving into Typescript. I currently have an array of binary strings Each string represents a binary number My goal is to filter the array and extract all strings that contain '1' at position X I ...

Using React-Bootstrap with TypeScript in your project

I'm currently working on creating a navigation bar using react-bootstrap. I've already installed the node-module as follows: "@types/react-bootstrap": "^0.32.11",. However, when I try to use it in my hello.tsx component, I encounter a compile err ...

What is the best method for connecting a ref to a component that I am duplicating with React.cloneElement?

Hi everyone! I'm trying to pass a ref into my component so that I can access the variables on the component like state. The only problem is, I'm having trouble getting it to work. It needs to be functional for both classes and functions. Every t ...

Angular 14: A collection and schematic must be provided for execution to proceed with the process

I've recently started learning angular. After installing the latest version, I created an app called "test" using the command ng new test. Next, I opened the app in Visual Studio Code and tried to create a new component by entering the command: ng g ...

Struggling to chart out the post response in Angular 7

I am facing an issue while setting up a service on Angular version 7. The problem arises with the res.json() method, throwing an error stating Property 'json' does not exist on type 'Object'. Below is my service's code: import {In ...

The onChange event does not work as expected for Select controls

I am facing an issue with my common useForm.tsx file when handling the onChange event for select controls. The error message I encounter is displayed below. Does anyone have any suggestions on how to resolve this? Error: Type '(e: ChangeEvent<HTM ...

unexpected error encountered when using WCF Service (potentially related to character encoding issue?)

After creating a WCF service that queries a semantic database and returns dbpedia.org links based on the entities, I am encountering an issue while invoking the method that retrieves these URIs. Most of the time, the method works fine, but occasionally I r ...

Is it possible to devise a universal click handler in TypeScript that will consistently execute after all other click handlers?

In my ReactJS based application written in TypeScript, we have implemented various click handlers. Different teams contribute to the application and can add their own handlers as well. The challenge we face is ensuring that a specific global click handler ...

How to make an HTTP request only once after a certain period of time

I have an Angular 11 component with multiple dynamically included inputs in a table. After the user enters a currency value into any of these inputs, I make an API call to calculate the total. The issue is that I am calling the API multiple times, so I wa ...

Switching from callback to function in TypeScript

Currently, I am utilizing the mongodb driver to establish a connection with mongo: public listUsers(filterSurname?:string):any { if (this.connected) { debug.log(this.db); var results; this.db.collection(' ...

The type 'Observable<Response | Observable<Response>>' cannot be assigned to the type 'Observable<Response>'

My service features a basic structure: import { Injectable } from '@angular/core'; import { Http, Response } from '@angular/http'; import 'rxjs/add/observable/throw'; import 'rxjs/add/operator/catch'; import ' ...

Encountering an issue while attempting to initiate a nested array: "Cannot assign a value to an optional property access in the left-hand side of an assignment expression."

I am dealing with an object that contains nested arrays, structured like this: export class OrdenCompra { public id?: number, public insumos?: OrdenCompraInsumo[], } export class OrdenCompraInsumo { id?: number; traslados?: IImpuestoTraslado[]; } export ...

Utilizing an array of data to create a complex structure with nested

In my Next.JS React project using TSX files, I have set up a data file like this: const fieldMapping = { category:[ { title: "Category 1", Subtitle: ["Category 1", "Category 2"], SubSubTitle: ["Category ...

How to extract a value from [object object] in Angular4

In my previous question, I shared the code below: getUserRole() { const headers = new Headers(); headers.append('Authorization', `Bearer ${this.getToken()}`); console.log(this.getToken()); const options = new RequestOptions({ headers: he ...

Should an HTML canvas in Angular be classified as a Component or a Service?

I have a basic drawing application that uses an MVC framework in TypeScript, and I am looking to migrate it to Angular. The current setup includes a Model for data handling, a View for rendering shapes on the canvas, and a Controller to manage interactio ...

utilize undefined files are assigned (Typescript, Express, Multer)

I am facing an issue while trying to save image uploads to a folder named "/images". The problem lies in the fact that req.files is appearing as undefined for some reason. Below is the relevant code snippet. Feel free to ask any questions, any assistance w ...

Is there a way to modify the id parameter in the URL using Angular 2's ActivatedRoute?

How can I modify a parameter in the URL without altering the overall address? https://i.stack.imgur.com/LOd4T.png This is the TypeScript code that I currently have: onRowClicked(event: any) { let currentIdPerson = event.data.IdPerson; } I am trying ...