Combining multiple JSON documents using TypeScript

Currently, my task involves tallying the number of times a specific email has made contact with us. These contacts are stored in JSON files with the key being "email".

The challenge here is that there could be an infinite number of JSON files, so I am looking to consolidate them into a single object and then iterate through them to calculate the frequency of each email.

To clarify, I need to first read the JSON content, log it, process the message, and then convert that message into a count of logs per email used.

My initial approach might not be correct, but I believe I need to merge all the JSON files into one object that I can loop through and manipulate as required. However, I seem to be facing synchronization issues.

I am using fs to read in (I anticipate around 100 JSON files), looping through them with forEach, and trying to push each entry into an array. However, the array keeps coming back empty. I am certain it's something simple that I'm missing, possibly overlooked in the fs documentation.

const fs = require('fs');

let consumed = [];
const fConsume = () => {
  fs.readdir(testFolder, (err, files) => {
    files.forEach(file => {
      let rawData = fs.readFileSync(`${testFolder}/${file}`);
      let readable = JSON.parse(rawData);
      consumed.push(readable);
    });
  
  })
}
fConsume();
console.log(consumed);

Just for clarification, this is what each JSON object looks like, and there are multiple objects per imported file.

{
      id: 'a7294140-a453-4f3c-91b6-210819e2c43e',
      email: '<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="6f0a1b070e0141070a1d010e010b0a152f02060c1d001c00091b410c0002">[email protected]</a>',
      message: 'successfully handled skipped operation.'
    },

Answer №1

In the code snippet above, fs.readdir() is an asynchronous function, meaning your function returns before it executes the callback. If you prefer to use synchronous code in this context, consider using fs.readdirSync() instead:

const fs = require('fs');

let data = [];
const processData = () => {
  const files = fs.readdirSync(testFolder)
  files.forEach(file => {
    let rawData = fs.readFileSync(`${testFolder}/${file}`);
    let readableData = JSON.parse(rawData);
    data.push(readableData);
  });  
}
processData();
console.log(data);

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is TS's method of interpreting the intersection between types that have the same named function properties but different signatures (resulting in an error when done

When working with types in Typescript, I encountered an interesting scenario. Suppose we have a type A with two properties that are functions. Now, if we define a type B as the intersection of type A with another type that has the same function properties ...

The two-way binding does not connect the property and event halves to the same target

I am trying to create a two-way binding using reactive forms in Angular. I need to exchange data between the child component and the parent component seamlessly. This is the HTML code for my child component: <input type="text" #name class=&qu ...

What are the steps for implementing JSON in C++ with a custom object?

I'm considering utilizing JSON in C++ from https://github.com/nlohmann/json#examples. Even after reviewing the basic examples, I'm still unsure how to implement it with my own object. For instance, I have a class: class Student { public: Stu ...

What is the best way to eliminate a mistaken error in TypeScript (specifically error: TS2339)?

var out = document.getElementsByClassName('myclass')[0]; out.focus(); out.select(); out.selectionStart =1; I've encountered an issue in my TypeScript file while attempting to execute the above code. Unfortunately, it's throwing errors ...

Converting group by values into JSON in Django is a common task that can be achieved by

Currently, I am facing an issue while trying to convert my grouped data into JsonResponse in Django. The error message that keeps popping up is: AttributeError: 'dict' object has no attribute 'f_type' This function below is responsi ...

What is the best way to interpret a JSON with nested structure?

I can't seem to find any errors in the code below, but I keep receiving an error message stating "item._highlightResult.map is not a function". {items.map((item, index) => ( <li key={index}><a href={item.story_url} target="_blank& ...

Storing user credentials locally on PhoneGap for Android and sending them as JSON for notifications

Having previously developed simple apps using phonegap for android, I am now working on a project that requires status bar notifications from a background service per user. Utilizing the phonegap backgroundservice plugin, I have been successful in receivin ...

Angular progress tracker with stages

I have been exploring ways to create a progress bar with steps in Angular 12 that advances based on the percentage of progress rather than just moving directly from one step to another. This is specifically for displaying membership levels and indicating h ...

Encountered an issue while configuring the Apollo server - The type '() => void' cannot be assigned to type '() => DataSources<object>'

I need help with a TypeScript-related issue. I am struggling to implement the expected return type for the function dataSources in this scenario. Here is the code snippet: const dataSources = () => { quizzessApi: new QuizzessDataSource(); } const ...

JSON array serving as a matrix for options available in DropDownLists

In my XSLT stylesheet, I have routines that process XML files to display products on a web page (ASP.NET). The XSLT outputs HTML for presenting the product along with its variations like size, color, package quantity, and style. The variations are treated ...

Iterating through each element individually in a JsonArray

The following JSON was obtained from an API call. String jsonString="{\r\n" + " \"count\": 100,\r\n" + " \"limit\": 100,\r\n" + ...

Can someone explain how to create a Function type in Typescript that enforces specific parameters?

Encountering an issue with combineReducers not being strict enough raises uncertainty about how to approach it: interface Action { type: any; } type Reducer<S> = (state: S, action: Action) => S; const reducer: Reducer<string> = (state: ...

Exploring error handling in onApplicationBootstrap() within NestJS

I have a TowerService method marked with @Injectable that is running in the onApplicationBootstrap() lifecycle hook. @Injectable() export class TasksService implements OnApplicationBootstrap { private readonly logger = new Logger(TasksService.name) co ...

Transmitting JSON data to a Flask template

I struggled to figure out how to retrieve JSON data from my search results, but I've managed to solve that issue. Now, the challenge is updating my jinja template with the data I fetched. Despite attempting various methods, my lack of experience with ...

Encountering a 404 error when trying to access a WCF Rest service (WebGet) due to a json string in the

For my project, I need to pass a dynamic Json string to a WCF Rest service web get. Everything works fine when I pass a simple string like this: http://localhost:58014/MyService.svc/api/queries/somestring However, when I try passing the Json string as a ...

Is there a way to customize the scrollbar color based on the user's preference?

Instead of hardcoding the scrollbar color to red, I want to change it based on a color variable provided by the user. I believe there are two possible solutions to this issue: Is it possible to assign a variable to line 15 instead of a specific color lik ...

A guide to parsing a JSON file in Next.js

Currently, I am in the process of migrating a next.js-project (with TypeScript) from the old pages-routing to the new app-routing. As part of this migration, I am working on implementing a new feature that involves reading JSON data directly from a file. ...

Transform input string containing newline characters into separate paragraphs

I utilize Contentful CMS for content management and fetch the content through their API. When the content is fetched, it comes in as a JSON object. One of the keys within this object pertains to the main text block for the entry I am retrieving. This stri ...

Utilize PHP to transform JSON data into JavaScript

Hello, I am a newcomer to Stackoverflow and I need some assistance. My issue involves converting JSON with PHP to JavaScript. I am using PHP to fetch data from a database and create JSON, which I then want to convert for use in JavaScript as an object (obj ...

How can I effectively handle extensive data parsing from a file using JavaScript?

Looking to optimize data parsing in JavaScript for a large file? I'm currently using JSON parse for a 250MB file, but it's too slow. Is there a faster method to extract a high volume of data without iterating through every character? The file con ...