Using a memory cache in development with NextJS does not seem to be effective

When exporting my pages for my simple static blog site, everything runs smoothly and quickly. However, in development mode, the generation of posts is sluggish and I'm looking to implement caching to speed up the process. I have set up a file called post.ts where I define my cache as const posts: Post[] = [].

I anticipate that when running yarn dev, the post.ts file will be loaded once allowing me to populate and reuse my cached post array. Strangely, this TypeScript module seems to be loaded multiple times.

import fs from 'fs'
import path from 'path'
// ...
​
console.log('????? reloading page')
const posts: Post[] = []  // cached posts, hope to generate once
let postsGenerated = false
​
export async function getSortedPostsData(): Promise<Post[]> {
  // Get file names under /posts
  const pendings: Promise<any>[] = []
  if (postsGenerated) {
    console.log('+++++ Using cache ')
    return sortPostsByDate(posts)
  } else {
    console.log('==== Calculating all')
    postsGenerated = true
  }
  let i = 0
  traverseDir('content/blog', (path) => {
    if (path.includes('.md')) { ... })
  }
      
  await Promise.all(pendings)
  return sortPostsByDate(posts)
}

Despite my efforts, sometimes the cache is utilized while other times it isn't. Even on consecutive reloads of the same page, the cache usage is inconsistent. Why is this happening? And what steps can be taken to improve the reliability of the caching system?

Answer №1

In my opinion, utilizing dynamic routes along with getStaticPaths and getStaticProps could be a suitable solution for this scenario.

getStaticPaths will scan the existing files and generate corresponding routes, while getStaticProps will retrieve the necessary post content as required. Upon exporting, all data will be fetched and processed accordingly.

Check out this link for more information on static HTML export in Next.js

For details on using getStaticPaths for static generation, refer to this documentation

Answer №2

Fast Refresh functionality can be a bit unpredictable, especially when working with `.ts` files during development. One workaround could be using a file as a cache to store parsed data and persist it across reloads instead of relying solely on constants.

const devPostsCache = 'devPostsCache.json';
export async function getSortedPostsData(): Promise<Post[]> {
  // Retrieving file names under /posts
  const pendings: Promise<any>[] = []
  const postsGenerated = fs.existsSync(devPostsCache);
  if (postsGenerated) {
    const posts = JSON.parse(fs.readFileSync(devPostsCache));
    return sortPostsByDate(posts)
  } else {
    console.log('==== Calculating all')
    postsGenerated = true
  }
  let i = 0
  traverseDir('content/blog', (path) => {
    if (path.includes('.md')) { ... })
  }
  
  // Assuming pending promises fetch an array of post objects
  const parsedPosts = await Promise.all(pendings);
  fs.writeFileSync(devPostsCache, JSON.stringify(parsedPosts));
  return sortPostsByDate(posts)
}

Consider deleting the cache file each time you start development using a `predev` script.

  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "predev": "rm devPostsCache.json"
  },

UPDATE

Another approach to minimize code recompilation frequency is adjusting the onDemandEntries configuration settings.

module.exports = {
  onDemandEntries: {
    // Time interval (in ms) serverside pages are kept in buffer
    maxInactiveAge: 360 * 1000,
    // Number of simultaneous pages kept in memory without disposal
    pagesBufferLength: 10,
  },
}

The default values are currently set at `60 * 1000` for maxInactiveAge and 2 for pagesBufferLength.

Although tweaking webpack configurations was attempted, issues persisted as nextjs tends to recompile everything per request.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

core.js:15723 ERROR TypeError: Unable to access the 'OBJECT' property because it is undefined

Whenever I attempt to run this function, I encounter an issue. My goal is to retrieve the latitude and longitude of an address from Google's API. This error message pops up: core.js:15723 ERROR TypeError: Cannot read property 'geometry' of ...

What is the best way to incorporate React Odometerjs into a Next.js project?

It's quite disheartening that despite spending 3 days trying, I still can't seem to successfully integrate Odometer js into my Next js project. I'm at a loss as to where I might be going wrong in my code. Here is the code I've been work ...

Do page paths differ between localhost and deployment environments?

After successfully building my app using next.js and testing it in localhost, I encountered an issue when deploying it on Heroku. Surprisingly, only the front page showed up while all other page paths did not work even though they were correctly inputted i ...

What is causing ESLint errors not to be displayed in the browser when they occur?

Recently, I noticed that ESLint errors are no longer appearing on the browser screen while I develop my Next.js (v14) app. Despite having several ESLint rules in place, they do not display on the screen like they did when using CRA. I prefer it when the a ...

An issue has been identified where the Redux dispatch function is not functioning properly on

I am currently utilizing Next.js, ts, redux, redux-saga, redux-wrapper in their latest versions. I am focused on fetching recipes through an API connected to MongoDB. Initially, I use getServerSideProps to load recipes and upon clicking a button, more re ...

The issue of Headless UI Transition not functioning properly in conjunction with Dialog components

I have encountered an issue with the transition effect in my Next.js 13 project using Headless UI. The transition effect seems to work only when the element leaves, but not when it enters. I am unsure if this is a bug in the code or if I am missing somethi ...

Angular: the xhrRequest is failing to be sent

I am facing an issue with a service function that handles an HTTP post request. The request does not get sent for some reason. However, when I add a subscription to the post method, the request is successfully executed. In other services that have the sam ...

What is the reason for next.js to execute getStaticPaths each time a page is loaded?

During the pre-rendering of my product detail pages, I encountered an issue with Next.js where the getStaticPaths function was being called on every page refresh or URL change even if it was the same page. This caused a delay and made the page unresponsive ...

Error: Unable to modify a property that is marked as read-only on object '#<Object>' in Redux Toolkit slice for Firebase Storage in React Native

Hey there! I've been working on setting my downloadUrl after uploading to firebase storage using Redux Toolkit, but I'm facing some challenges. While I have a workaround, I'd prefer to do it the right way. Unfortunately, I can't seem to ...

Error: unable to locate module that was imported from

Every time I try to run the project using this command, an error pops up: > npm run build npm info using <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="c7a9b7aa87fee9f1e9f0">[email protected]</a> npm info using ...

How to Include Data in the <Head> Tag Using Query Parameters with NextJS

I have a dilemma with my website pages that are using the getStaticProps method. I am trying to figure out how to dynamically add 'noindex' meta tag to pages that have a 'q' parameter in the URL. Unfortunately, this approach does not se ...

Fetch data asynchronously prior to loading the DOM

Currently in the learning process and facing a hurdle concerning how to retrieve data from an async function and display it in the DOM. The issue I'm encountering is that I am receiving 'undefined' as the DOM gets rendered before the async ...

How can I set up authentication with NestJS, GraphQL, and Next.js? Is next-auth a suitable option for this setup?

Recently, I've developed an API with Nest.js and GraphQL which includes an endpoint for authenticating users and generating access and refresh tokens (JWT). Now, my challenge is integrating this API with a Next.js front-end application. Although I&ap ...

next-themes issue: (0, react__WEBPACK_IMPORTED_MODULE_0__.createContext) function is invalid

After installing next-themes, I encountered an error that I'm struggling to pinpoint. Error-1 Error-2 Attempted solutions so far include deleting the node_modules folder and package-lock.json, then running npm install. Unfortunately, this did not re ...

Combine two streams under certain conditions using RxJs

When working with streams, I am facing a scenario where I have two server calls to make in order to get the required response. However, if the first call returns data, I do not want to execute the second call. I have been struggling to find the proper comb ...

What is the purpose of specifying http://localhost:3000 when accessing API routes in Next.js?

I created an API route within the pages directory of my NextJS project. It is functioning properly as I am able to retrieve data by directly accessing the URL like http://localhost:3000/api/tv/popular. My goal is to fetch this data using getStaticProps and ...

What is the best way to group an array based on a dynamic key?

My goal is to group values in an array by a given ID. I've attempted a method for this, but it's not working for dynamic keys. Here is the current array: let employees = [{"employeeDetail": [{"empID": "XXYYZZ11"," ...

The specified type 'IterableIterator<number>' does not belong to either an array type or a string type

Encountering an error while attempting to generate a dynamic range of numbers. Error: Type 'IterableIterator<number>' is not recognized as an array or string type. Use the compiler option '--downlevelIteration' to enable iteratio ...

Is it possible to access the attributes of an interface in TypeScript without relying on external libraries?

Ensuring the properties of an interface align with an object that implements it is crucial for successful unit testing. If modifications are made to the interface, the unit test should fail if it is not updated with the new members. Although I attempted ...

Having trouble passing the theme in Next.js when using Styled Components?

I'm facing an issue with passing my theme object to styled-components in my Next.js project as it fails to receive it as props. Here is my _document.tsx: import { getInitialProps } from "@expo/next-adapter/document"; import Document, { DocumentConte ...