What strategies can be utilized to manage a sizable data set?

I'm currently tasked with downloading a large dataset from my company's database and analyzing it in Excel. To streamline this process, I am looking to automate it using ExcelOnline. I found a helpful guide at this link provided by Microsoft Power Automate Cookbook.

Despite attempting to use OfficeScript to handle the data processing, I have encountered difficulties due to the size of the dataset. Unfortunately, I haven't been able to find a solution within the available resources.

Any suggestions or advice would be greatly appreciated!

Here is a snippet of the code I've been working on:

function main(workbook: ExcelScript.Workbook): RawData[] {
  let lastRow = workbook.getWorksheet('DETAIL').getUsedRange().getRowCount();
  let rows = workbook.getWorksheet('DETAIL').getRange('A3:CQ' + lastRow).getValues();
  let records: RawData[] = [];

  for (let row of rows) {
        let [ORNo,SubNo, ... , LineNo,ProductionLine] = row; //95 Columns 21619rows(including header)
        records.push({
            ORNo: ORNo as string,
            SubNo: SubNo as number,
            ...,
            LineNo: LineNo as string,
            ProductionLine: ProductionLine as string
        })
    }
    console.log(JSON.stringify(records))
    return records;
}

interface RawData {
    ORNo: string
    SubNo: number
    ...
    LineNo: string
    ProductionLine: string
}

The runtime error message states:

Line 3: Range getValues: The response payload size has exceeded the limit. Please refer to the documentation: "https://docs.microsoft.com/office/dev/add-ins/concepts/resource-limits-and-performance-optimization#excel-add-ins".

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Organizing requirejs and angularjs code structure into separate files

Looking to organize my Angular application with requirejs by separating controllers, services, and directives into different files. Hoping to achieve this structure: src/ components/ Navigation/ index.js module.js NavigationCon ...

What is the process to set up a WebSocket on Tornado for a production server rather than on a local machine?

Recently, I've delved into the world of WebSockets but have only experimented with them in a localhost environment while developing locally. Following this informative tutorial on real-time data visualization: https://medium.com/@benjaminmbrown/real-t ...

Display the same data point on a Google Map from two different rows of a database

I'm looking to display multiple values for profFName and profLName on a map with just one point showing this information. If I have (2) pID connected to the same marker, I want it to show both profFName and profLName on a single point. Currently, it ...

Persisted state in Vuex fails to retain data after the page is refreshed

I recently added persisted state to my Vue application using the command npm install --save vuex-persistedstate. After that, I configured my Vuex store.js file in the following way: import Vue from 'vue' import Vuex from 'vuex' import ...

Sending data to server with Backbone (using save() function and php script)

Wondering if you can assist me with an issue I'm facing. I am trying to utilize Backbone's save() method and send data to a php file. However, I encountered a problem right at the beginning. When I execute the following in the browser's co ...

Nextjs server encountered an issue where it was unable to connect to the localhost

npm run dev > <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="563239352239247b372626393f38223b3338227b3439393d3f38317b2133347b372626166678677866">[email protected]</a> dev > next dev ▲ Next.js 14.2. ...

Utilizing BEM Class Names in React

How can I utilize the Post component in a way that assigns unique classes to new and old posts following BEM recommendations? Assign a unique className to every element Avoid cascading dependencies like (.posts-new post or .posts-old post) Each component ...

Error message: Custom binding handler failed: 'Flatpickr' is not a valid constructor

Trying my hand at creating a custom binding handler in knockout for Flatpickr has hit a snag. Upon attempting to use it, an error is thrown: Uncaught TypeError: Unable to process binding "datetimepicker: function (){return startDate }" Message: Flatpickr ...

React is unable to assign a class field beyond the scope of axios

class App extends React.Component { app: Application; ... componentDidMound() { axios.get(…).then(res => { this.app.currentUser = res.data.data; // setting value inside lambda function. console.log(this.app.currentUser); // ...

Define the interface for a GraphQL resolver argument in code-first implementation

This specific GraphQL schema example from the Constructing Types page showcases how to define the Query type. // Creating the Query type var queryType = new graphql.GraphQLObjectType({ name: 'Query', fields: { user: { type: userType ...

Eliminate the standard blue border that appears when control-clicking on table elements

I've encountered this question before, but unfortunately none of the solutions provided have worked for me. Some things I've attempted are: Using event.preventDefault() - did not produce the desired result. Removing user-select from CS ...

Ways to identify the active anchor within an li element

Below is the code snippet I am currently working with: <li> <a href="<?php echo base_url()?>home/promos/call" class="promo-call active-call"></a> </li> <li> <a href="<?php echo base_url()?>home/promos/text ...

Display error messages upon submitting the form

I am currently working on an Angular 6 Form with validation. My main goal is to display error messages only after the form has been submitted. It is crucial that these messages remain constant while the user types in the input field. For instance, if a use ...

Creating a button in ReactJS with text displayed under an icon

Presently, I am working with a component that looks like this: https://i.stack.imgur.com/qGCwj.png This is the code for the component: import React from "react"; import {withStyles} from "material-ui/styles"; import Settings from "material-ui-icons/Setti ...

An error has occurred in Typescript stating that there is no overload that matches the call for AnyStyledComponent since the update to nextjs

Upgraded to the latest version of nextjs, 13.3.1, and encountered an error in the IDE related to a styled component: Received error TS2769 after the upgrade: No overload matches this call. Overload 1 of 2, '(component: AnyStyledComponent): ThemedSty ...

Which one should I prioritize learning first - AngularJS or Laravel?

As a novice web developer, I am embarking on my first journey into the world of frameworks. After much consideration, I have narrowed it down to two options: AngularJS and Laravel. Can you offer any advice on which one would be best for me to start with? ...

Compiling Typescript with module imports

In my project, I am working with two files named a.ts and b.ts. The interesting part is that file b exports something for file a to use. While the TypeScript compiler handles this setup perfectly, it fails to generate valid output for a browser environment ...

Stop auto-scrolling to the top when triggering a getJSON request

I'm feeling really puzzled at the moment. Here is the snippet of code I am struggling with: $("#combinations").on("change", "input", function (e) { e.preventDefault(); console.log(e) var $button, $row, $group, $form, $barcode ...

What is the best method to incorporate extra parameters into highcharts using data extracted from datatables?

I have a query regarding the integration of datatables with highcharts. The issue I'm facing is related to specific configurations for chart-A, which is of column type. Even after applying the configurations, it seems like they are not being incorpora ...

"The authentication scheme is unrecognized" - this is the message produced by Node-LinkedIn module

I am currently utilizing the node-linkedin npm package to authenticate and retrieve information about other users, such as their name, job title, company name, profile picture, and shared connections. While I am able to successfully receive and store the a ...