Tips for caching yarn packages in GitHub Actions

Question

Tips for caching yarn packages in GitHub Actions

I am currently utilizing GitHub Actions to compile my TypeScript project. Each time I execute the action, I have to wait for approximately 3 minutes for all dependencies to be installed.

Is there a method to cache yarn dependencies in order to reduce build time?

I attempted the following approach:

     - name: Obtain the directory path for yarn cache
       id: yarn-cache-dir-path
       run: echo "::set-output name=dir::$(yarn cache dir)"

     - uses: actions/cache@v1
       id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
       with:
         path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
         key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
         restore-keys: |
           ${{ runner.os }}-yarn-

    - name: Install yarn
      run: npm install -g yarn

    - name: Install project dependencies
      run: yarn

However, despite implementing this strategy, the build time remains unchanged.

typescript yarnpkg github-actions

Answer 1

Answer №1

When using actions/setup-node@v2 or a newer version:
```
- name: Set up Node.js
  uses: actions/setup-node@v3
  with:
    node-version: '16'
    cache: 'yarn'

- name: Install project dependencies
  run: yarn
```
With the newer versions of actions/setup-node@v2, caching is already included, eliminating the need to set up actions/cache.

If you are using actions/setup-node@v1 and want to cache Yarn global cache with actions/cache:

- name: Set up Node.js
  uses: actions/setup-node@v1
  with:
    node-version: '16'

- name: Get yarn cache directory path
  id: yarn-cache-dir-path
  run: echo "::set-output name=dir::$(yarn cache dir)"

- uses: actions/cache@v3
  id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
  with:
    path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
    key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
    restore-keys: |
    ${{ runner.os }}-yarn-

- name: Install project dependencies
  run: yarn --prefer-offline

The code above only caches and restores the Yarn global cache directory, not the node_modules directory itself.

To speed up installation, specify that Yarn should use cached downloads during installation whenever possible instead of downloading from the server:

- name: Install project dependencies
  run: yarn --prefer-offline

Caching node_modules with actions/cache (not recommended)

You can also cache the node_modules directory directly and skip running yarn if the cache is available.

However, this approach is not recommended for the following reasons:

yarn makes good use of the global cache. If the dependencies are already in the global cache, yarn can finish quickly. (see comment from @mvlabat).
node_modules could be corrupted. It's safer to let yarn decide whether to get files from the cache by re-running it each time (as yarn validates the cache before using it).

- name: Set up Node.js
  uses: actions/setup-node@v1
  with:
    node-version: '16'

- name: Get yarn cache directory path
    id: yarn-cache-dir-path
  run: echo "::set-output name=dir::$(yarn cache dir)"

- name: Cache yarn cache
    uses: actions/cache@v3
    id: cache-yarn-cache
    with:
      path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
      key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
      restore-keys: |
          ${{ runner.os }}-yarn-

- name: Cache node_modules
    id: cache-node-modules
    uses: actions/cache@v3
    with:
      path: node_modules
      key: ${{ runner.os }}-${{ matrix.node-version }}-nodemodules-${{ hashFiles('**/yarn.lock') }}
      restore-keys: |
          ${{ runner.os }}-${{ matrix.node-version }}-nodemodules-

- run: yarn
    if: |
        steps.cache-yarn-cache.outputs.cache-hit != 'true' ||
        steps.cache-node-modules.outputs.cache-hit != 'true'

Answer 2

When using actions/setup-node@v2 or a newer version:
```
- name: Set up Node.js
  uses: actions/setup-node@v3
  with:
    node-version: '16'
    cache: 'yarn'

- name: Install project dependencies
  run: yarn
```
With the newer versions of actions/setup-node@v2, caching is already included, eliminating the need to set up actions/cache.

If you are using actions/setup-node@v1 and want to cache Yarn global cache with actions/cache:

- name: Set up Node.js
  uses: actions/setup-node@v1
  with:
    node-version: '16'

- name: Get yarn cache directory path
  id: yarn-cache-dir-path
  run: echo "::set-output name=dir::$(yarn cache dir)"

- uses: actions/cache@v3
  id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
  with:
    path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
    key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
    restore-keys: |
    ${{ runner.os }}-yarn-

- name: Install project dependencies
  run: yarn --prefer-offline

The code above only caches and restores the Yarn global cache directory, not the node_modules directory itself.

To speed up installation, specify that Yarn should use cached downloads during installation whenever possible instead of downloading from the server:

- name: Install project dependencies
  run: yarn --prefer-offline

Caching node_modules with actions/cache (not recommended)

You can also cache the node_modules directory directly and skip running yarn if the cache is available.

However, this approach is not recommended for the following reasons:

yarn makes good use of the global cache. If the dependencies are already in the global cache, yarn can finish quickly. (see comment from @mvlabat).
node_modules could be corrupted. It's safer to let yarn decide whether to get files from the cache by re-running it each time (as yarn validates the cache before using it).

- name: Set up Node.js
  uses: actions/setup-node@v1
  with:
    node-version: '16'

- name: Get yarn cache directory path
    id: yarn-cache-dir-path
  run: echo "::set-output name=dir::$(yarn cache dir)"

- name: Cache yarn cache
    uses: actions/cache@v3
    id: cache-yarn-cache
    with:
      path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
      key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
      restore-keys: |
          ${{ runner.os }}-yarn-

- name: Cache node_modules
    id: cache-node-modules
    uses: actions/cache@v3
    with:
      path: node_modules
      key: ${{ runner.os }}-${{ matrix.node-version }}-nodemodules-${{ hashFiles('**/yarn.lock') }}
      restore-keys: |
          ${{ runner.os }}-${{ matrix.node-version }}-nodemodules-

- run: yarn
    if: |
        steps.cache-yarn-cache.outputs.cache-hit != 'true' ||
        steps.cache-node-modules.outputs.cache-hit != 'true'

Answer 3

Answer №2

According to the guidelines in the documentation of the GitHub package:

steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
  with:
    node-version: '14'
    cache: 'npm' # or yarn
- run: npm install
- run: npm test

Edit:

It turns out that the instructions were misleading and have now been updated to clarify that it does not cache the node_modules folder, but only the global cache directory, as explained in this issue.

In addition, as mentioned by Mrchief in the comments:

... you'll still experience the npm i time, but save on download time from the internet (if the module is in the npm cache)

Therefore, it is recommended to continue using this to save time on downloading packages from the internet, but if you want to cache the node_modules folder, refer to other solutions that utilize actions/cache.

Be sure to also read Quang Lam's response and its comments on why you should not cache the node_modules folder.

Answer 4

According to the guidelines in the documentation of the GitHub package:

steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
  with:
    node-version: '14'
    cache: 'npm' # or yarn
- run: npm install
- run: npm test

Edit:

It turns out that the instructions were misleading and have now been updated to clarify that it does not cache the node_modules folder, but only the global cache directory, as explained in this issue.

In addition, as mentioned by Mrchief in the comments:

... you'll still experience the npm i time, but save on download time from the internet (if the module is in the npm cache)

Therefore, it is recommended to continue using this to save time on downloading packages from the internet, but if you want to cache the node_modules folder, refer to other solutions that utilize actions/cache.

Be sure to also read Quang Lam's response and its comments on why you should not cache the node_modules folder.

Answer 5

Answer №3

Noted in the comment next to the id field for the caching step:

Verify cache-hit by checking if
steps.yarn-cache.outputs.cache-hit != 'true'

An essential conditional if property is missing to determine when the step should be executed:

- name: Install yarn
  run: npm install -g yarn

- name: Install project dependencies
  if: steps.yarn-cache.outputs.cache-hit != 'true' # Important detail!
  run: yarn

Additionally, consider using the Setup NodeJS GitHub Action that includes Yarn setup:

- uses: actions/setup-node@v1
  with:
    node-version: '10.x' # Specify the version of NodeJS.

Refer to the action.yml file for a complete list of valid inputs.

UPDATE: Yarn is actually pre-installed on the GitHub-hosted Ubuntu 18.04.4 LTS (ubuntu-latest/ubuntu-18.04) runner, eliminating the need for a global installation step.

Answer 6

Noted in the comment next to the id field for the caching step:

Verify cache-hit by checking if
steps.yarn-cache.outputs.cache-hit != 'true'

An essential conditional if property is missing to determine when the step should be executed:

- name: Install yarn
  run: npm install -g yarn

- name: Install project dependencies
  if: steps.yarn-cache.outputs.cache-hit != 'true' # Important detail!
  run: yarn

Additionally, consider using the Setup NodeJS GitHub Action that includes Yarn setup:

- uses: actions/setup-node@v1
  with:
    node-version: '10.x' # Specify the version of NodeJS.

Refer to the action.yml file for a complete list of valid inputs.

UPDATE: Yarn is actually pre-installed on the GitHub-hosted Ubuntu 18.04.4 LTS (ubuntu-latest/ubuntu-18.04) runner, eliminating the need for a global installation step.

Answer 7

Answer №4

actions/setup-node now offers caching options starting from version 2, providing customizable features.

- uses: actions/checkout@v3

- name: Setup Node.js
  uses: actions/setup-node@v3
  with:
    node-version: '16'
    cache: 'yarn'

- name: Install JS dependencies
  run: yarn install

The recommended approach for caching, as outlined in the documentation, involves caching only the yarn cache dir and not node_modules. Caching node_modules is discouraged due to potential issues when changing node versions.

_{This information has been updated:}

_{A one-liner cache solution designed specifically for Yarn can be found here: https://github.com/c-hive/gha-yarn-cache}

_{This tool aligns with GitHub's recommendations and supports both Yarn v1 and v2.}

_{Similarly, a cache solution tailored for NPM is available at: https://github.com/c-hive/gha-npm-cache}

Answer 8

actions/setup-node now offers caching options starting from version 2, providing customizable features.

- uses: actions/checkout@v3

- name: Setup Node.js
  uses: actions/setup-node@v3
  with:
    node-version: '16'
    cache: 'yarn'

- name: Install JS dependencies
  run: yarn install

The recommended approach for caching, as outlined in the documentation, involves caching only the yarn cache dir and not node_modules. Caching node_modules is discouraged due to potential issues when changing node versions.

_{This information has been updated:}

_{A one-liner cache solution designed specifically for Yarn can be found here: https://github.com/c-hive/gha-yarn-cache}

_{This tool aligns with GitHub's recommendations and supports both Yarn v1 and v2.}

_{Similarly, a cache solution tailored for NPM is available at: https://github.com/c-hive/gha-npm-cache}

Tips for caching yarn packages in GitHub Actions

Answer №1

Answer №2

Edit:

Answer №3

Answer №4

Similar questions

There is an error appearing in my .ts code: [ts] The property 'name' is not found in type 'any[]'

Inferencing repeated generics in Typescript

typescript throwing an unexpected import/export token error

Creating a global property access expression using the Typescript Compiler API

Exploring the world of Unicode in Angular

Creating Class Names Dynamically in Angular 2 Using ngFor Index

The error message "Can't resolve all parameters for CustomerService" is preventing Angular from injecting HttpClient

Purge React Query Data By ID

Ran into a situation where Nextjs13 had two children sharing the same key

An easy guide on utilizing ngSwitch for datatype selection in Angular

Can you explain the distinction between Reflect.getMetadata and Reflect.getOwnMetadata?

Guide for referencing brackets with Joi validation

Issue with Angular 11: Unable to bind to 'ngForOf' as it is not recognized as a valid property of 'tr' element

What is the best way to implement global error handling for NextJS API requests?

Creating and incorporating a generator function within an interface and class: A step-by-step guide

When using React Hook Form with TypeScript, the error message "Property 'inputRef' does not exist on type DetailedHTMLProps" may arise

When using TypeORM's save() method with an array of objects, the @PrimaryColumn() annotations are ignored, resulting

The Angular 5 keyup event is being triggered twice

Even when using module.exports, NodeJS and MongoDB are still experiencing issues with variable definitions slipping away

Exploring the various methods of setting up a Create React App project with TypeScript