Utilizing puppeteer-core with electron: A guide

Question

Utilizing puppeteer-core with electron: A guide

I found this snippet on a different Stack Overflow thread:

import electron from "electron";
import puppeteer from "puppeteer-core";

const delay = (ms: number) =>
  new Promise(resolve => {
    setTimeout(() => {
      resolve();
    }, ms);
  });

(async () => {
  try {
    const app = await puppeteer.launch({
      executablePath: electron,
      args: ["."],
      headless: false,
    });
    const pages = await app.pages();
    const [page] = pages;

    await page.setViewport({ width: 1200, height: 700 });
    await delay(5000);
    const image = await page.screenshot();
    console.log(image);
    await page.close();
    await delay(2000);
    await app.close();
  } catch (error) {
    console.error(error);
  }
})();

The Typescript compiler is throwing an error for the executablePath property in the launch method's options object because it should be of type string rather than Electron. How can we correctly pass the chromium executable path to puppeteer?

javascript typescript electron puppeteer

Answer 1

Answer №1

Utilizing the electron executable with Puppeteer directly requires some workarounds and flag adjustments due to their differences in API. Electron lacks certain key functionalities found in the chrome.* API, essential for proper operation of Chromium browser. Many flags do not yet have suitable replacements, such as the headless flag.

Outlined below are two methods to achieve this. However, it is crucial to adhere to the following points:

Ensure that Puppeteer is connected before launching the application.
Verify you have the correct version of Puppeteer or Puppeteer-core matching the Chrome version running within Electron!

Option 1: Leveraging `puppeteer-in-electron`

Among various alternatives, a recent addition includes the puppeteer-in-electron package, enabling the running of Puppeteer inside an Electron app utilizing Electron itself.

To begin, install the necessary dependencies:

npm install puppeteer-in-electron puppeteer-core electron

Subsequently, execute the code snippet provided below.

import {BrowserWindow, app} from "electron";
import pie from "puppeteer-in-electron";
import puppeteer from "puppeteer-core";

const main = async () => {
  const browser = await pie.connect(app, puppeteer);

  const window = new BrowserWindow();
  const url = "https://example.com/";
  await window.loadURL(url);

  const page = await pie.getPage(browser, window);
  console.log(page.url());
  window.destroy();
};

main();

Option 2: Obtain the debugging port and establish connection

Alternatively, one can acquire the remote-debugging-port of the Electron app and connect to it. This approach was shared by trusktr on the Electron forum.

import {app, BrowserWindow, ...} from "electron"
import fetch from 'node-fetch'

import * as puppeteer from 'puppeteer'

app.commandLine.appendSwitch('remote-debugging-port', '8315')

async function test() {
    const response = await fetch(`http://localhost:8315/json/versions/list?t=${Math.random()}`)
    const debugEndpoints = await response.json()

    let webSocketDebuggerUrl = debugEndpoints['webSocketDebuggerUrl ']

    const browser = await puppeteer.connect({
        browserWSEndpoint: webSocketDebuggerUrl
    })

    // utilize Puppeteer APIs now!
}

// Establish your window, etc., then proceed with:

  // after window load, connect Puppeteer:
  mainWindow.webContents.on("did-finish-load", () => { 
    test()
  })

Both solutions above rely on the usage of webSocketDebuggerUrl to address the issue.

Additional Information

Since many individuals use Electron for bundling applications, it's worth mentioning:

If building puppeteer-core and puppeteer-in-electron, ensure the utilization of hazardous and electron-builder to facilitate functioning of get-port-cli.

Place hazardous at the beginning of main.js:

// main.js
require ('hazardous');

Unpack the get-port-cli script and specify in package.json:

"build": {
  "asarUnpack": "node_modules/get-port-cli"
}

Upon completion of the build process:

Answer 2

Utilizing the electron executable with Puppeteer directly requires some workarounds and flag adjustments due to their differences in API. Electron lacks certain key functionalities found in the chrome.* API, essential for proper operation of Chromium browser. Many flags do not yet have suitable replacements, such as the headless flag.

Outlined below are two methods to achieve this. However, it is crucial to adhere to the following points:

Ensure that Puppeteer is connected before launching the application.
Verify you have the correct version of Puppeteer or Puppeteer-core matching the Chrome version running within Electron!

Option 1: Leveraging `puppeteer-in-electron`

Among various alternatives, a recent addition includes the puppeteer-in-electron package, enabling the running of Puppeteer inside an Electron app utilizing Electron itself.

To begin, install the necessary dependencies:

npm install puppeteer-in-electron puppeteer-core electron

Subsequently, execute the code snippet provided below.

import {BrowserWindow, app} from "electron";
import pie from "puppeteer-in-electron";
import puppeteer from "puppeteer-core";

const main = async () => {
  const browser = await pie.connect(app, puppeteer);

  const window = new BrowserWindow();
  const url = "https://example.com/";
  await window.loadURL(url);

  const page = await pie.getPage(browser, window);
  console.log(page.url());
  window.destroy();
};

main();

Option 2: Obtain the debugging port and establish connection

Alternatively, one can acquire the remote-debugging-port of the Electron app and connect to it. This approach was shared by trusktr on the Electron forum.

import {app, BrowserWindow, ...} from "electron"
import fetch from 'node-fetch'

import * as puppeteer from 'puppeteer'

app.commandLine.appendSwitch('remote-debugging-port', '8315')

async function test() {
    const response = await fetch(`http://localhost:8315/json/versions/list?t=${Math.random()}`)
    const debugEndpoints = await response.json()

    let webSocketDebuggerUrl = debugEndpoints['webSocketDebuggerUrl ']

    const browser = await puppeteer.connect({
        browserWSEndpoint: webSocketDebuggerUrl
    })

    // utilize Puppeteer APIs now!
}

// Establish your window, etc., then proceed with:

  // after window load, connect Puppeteer:
  mainWindow.webContents.on("did-finish-load", () => { 
    test()
  })

Both solutions above rely on the usage of webSocketDebuggerUrl to address the issue.

Additional Information

Since many individuals use Electron for bundling applications, it's worth mentioning:

If building puppeteer-core and puppeteer-in-electron, ensure the utilization of hazardous and electron-builder to facilitate functioning of get-port-cli.

Place hazardous at the beginning of main.js:

// main.js
require ('hazardous');

Unpack the get-port-cli script and specify in package.json:

"build": {
  "asarUnpack": "node_modules/get-port-cli"
}

Upon completion of the build process:

Answer 3

Answer №2

The solution provided did not work for me when using Electron 11 and Puppeteer-core 8. However, starting Puppeteer in the main process instead of the renderer process turned out to be effective. By utilizing ipcMain and ipcRenderer to communicate with each other, I was able to achieve the desired outcome. See the modified code below:

Main.ts (main process code)

import { app, BrowserWindow, ipcMain } from 'electron';
import puppeteer from 'puppeteer-core';
async function newGrabBrowser({ url }) {
  const browser = await puppeteer.launch({
    headless: false,
    executablePath:
      '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
  });
  const page = await browser.newPage();
  page.goto(url);
}
ipcMain.on('grab', (event, props) => {
  newGrabBrowser(JSON.parse(props));
});

Home.ts (renderer process code)

const { ipcRenderer } = require('electron');
ipcRenderer.send('grab',JSON.stringify({url: 'https://www.google.com'}));

Answer 4

The solution provided did not work for me when using Electron 11 and Puppeteer-core 8. However, starting Puppeteer in the main process instead of the renderer process turned out to be effective. By utilizing ipcMain and ipcRenderer to communicate with each other, I was able to achieve the desired outcome. See the modified code below:

Main.ts (main process code)

import { app, BrowserWindow, ipcMain } from 'electron';
import puppeteer from 'puppeteer-core';
async function newGrabBrowser({ url }) {
  const browser = await puppeteer.launch({
    headless: false,
    executablePath:
      '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
  });
  const page = await browser.newPage();
  page.goto(url);
}
ipcMain.on('grab', (event, props) => {
  newGrabBrowser(JSON.parse(props));
});

Home.ts (renderer process code)

const { ipcRenderer } = require('electron');
ipcRenderer.send('grab',JSON.stringify({url: 'https://www.google.com'}));

Answer 5

Answer №3

In addition to the previous method, there is another alternative that is compatible with electron versions 5.x.y and above (currently supported up to version 7.x.y, but not tested on beta version 8.x.y yet):

// const assert = require("assert");
const electron = require("electron");
const kill = require("tree-kill");
const puppeteer = require("puppeteer-core");
const { spawn } = require("child_process");

let pid;

const launchElectron = async () => {
  const port = 9200; // Debugging port
  const startTime = Date.now();
  const timeout = 20000; // Timeout in milliseconds
  let app;

  // Start Electron with custom debugging port
  pid = spawn(electron, [".", `--remote-debugging-port=${port}`], {
    shell: true
  }).pid;

  // Wait for Puppeteer to connect
  while (!app) {
    try {
      app = await puppeteer.connect({
        browserURL: `http://localhost:${port}`,
        defaultViewport: { width: 1000, height: 600 } // Optional I believe
      });
    } catch (error) {
      if (Date.now() > startTime + timeout) {
        throw error;
      }
    }
  }

  // Perform some actions, for example:
  // const [page] = await app.pages();
  // await page.waitForSelector("#someid")// 
  // const text = await page.$eval("#someid", element => element.innerText);
  // assert(text === "Your desired text");
  // await page.close();
};

launchElectron()
  .then(() => {
    // Carry out further actions
  })
  .catch(error => {
    // Handle errors
    kill(pid, () => {
      process.exit(1);
    });
  });

Obtaining the pid and utilizing kill is an optional step. It may not be necessary when running the script on a CI platform, but in a local environment you might need to manually close the electron application after each unsuccessful attempt.

For more details, please refer to this sample repository.

Answer 6

In addition to the previous method, there is another alternative that is compatible with electron versions 5.x.y and above (currently supported up to version 7.x.y, but not tested on beta version 8.x.y yet):

// const assert = require("assert");
const electron = require("electron");
const kill = require("tree-kill");
const puppeteer = require("puppeteer-core");
const { spawn } = require("child_process");

let pid;

const launchElectron = async () => {
  const port = 9200; // Debugging port
  const startTime = Date.now();
  const timeout = 20000; // Timeout in milliseconds
  let app;

  // Start Electron with custom debugging port
  pid = spawn(electron, [".", `--remote-debugging-port=${port}`], {
    shell: true
  }).pid;

  // Wait for Puppeteer to connect
  while (!app) {
    try {
      app = await puppeteer.connect({
        browserURL: `http://localhost:${port}`,
        defaultViewport: { width: 1000, height: 600 } // Optional I believe
      });
    } catch (error) {
      if (Date.now() > startTime + timeout) {
        throw error;
      }
    }
  }

  // Perform some actions, for example:
  // const [page] = await app.pages();
  // await page.waitForSelector("#someid")// 
  // const text = await page.$eval("#someid", element => element.innerText);
  // assert(text === "Your desired text");
  // await page.close();
};

launchElectron()
  .then(() => {
    // Carry out further actions
  })
  .catch(error => {
    // Handle errors
    kill(pid, () => {
      process.exit(1);
    });
  });

Obtaining the pid and utilizing kill is an optional step. It may not be necessary when running the script on a CI platform, but in a local environment you might need to manually close the electron application after each unsuccessful attempt.

For more details, please refer to this sample repository.

Answer 7

Answer №4

As stated in the documentation puppeteer-in-electron, everything should work properly. However, it is important to note that when working with ipcMain, you need to initialize the app before calling the main function.

main.js

// main.js
const {BrowserWindow, app} = require("electron");
const pie = require("puppeteer-in-electron")
const puppeteer = require("puppeteer-core");

app.whenReady().then(() => {
    ipcMain.handle('get-url', getUrl)
    createWindow()

    ///
})

async function initialize(){
    await pie.initialize(app);
}
initialize();

async function getUrl(){
  const browser = await pie.connect(app, puppeteer);
 
  const window = new BrowserWindow();
  const url = "https://example.com/";
  await window.loadURL(url);
 
  const page = await pie.getPage(browser, window);
  console.log(page.url());
  window.destroy();
  return page.url();
};

preload.js

const { contextBridge,ipcRenderer } = require('electron')

contextBridge.exposeInMainWorld('electronAPI', {
    getUrl: () => ipcRenderer.invoke('get-url')
})

renderer (vue3)

<script setup>
import { ref } from 'vue';

const res = ref();

async function get(){
    const result = await window.electronAPI.getUrl()
    res.value = result
    console.log(result);
}
</script>

my packages version

"puppeteer-core": "^19.7.1"
"puppeteer-in-electron": "^3.0.5"
"electron": "^23.1.0"

Answer 8

As stated in the documentation puppeteer-in-electron, everything should work properly. However, it is important to note that when working with ipcMain, you need to initialize the app before calling the main function.

main.js

// main.js
const {BrowserWindow, app} = require("electron");
const pie = require("puppeteer-in-electron")
const puppeteer = require("puppeteer-core");

app.whenReady().then(() => {
    ipcMain.handle('get-url', getUrl)
    createWindow()

    ///
})

async function initialize(){
    await pie.initialize(app);
}
initialize();

async function getUrl(){
  const browser = await pie.connect(app, puppeteer);
 
  const window = new BrowserWindow();
  const url = "https://example.com/";
  await window.loadURL(url);
 
  const page = await pie.getPage(browser, window);
  console.log(page.url());
  window.destroy();
  return page.url();
};

preload.js

const { contextBridge,ipcRenderer } = require('electron')

contextBridge.exposeInMainWorld('electronAPI', {
    getUrl: () => ipcRenderer.invoke('get-url')
})

renderer (vue3)

<script setup>
import { ref } from 'vue';

const res = ref();

async function get(){
    const result = await window.electronAPI.getUrl()
    res.value = result
    console.log(result);
}
</script>

my packages version

"puppeteer-core": "^19.7.1"
"puppeteer-in-electron": "^3.0.5"
"electron": "^23.1.0"

Utilizing puppeteer-core with electron: A guide

Answer №1

Option 1: Leveraging `puppeteer-in-electron`

Option 2: Obtain the debugging port and establish connection

Additional Information

Answer №2

Answer №3

Answer №4

my packages version

Similar questions

Searching dynamically using class names with JQuery

Angular JS effectively prevents redundant data from being displayed to users when scrolling infinitely while also efficiently removing DOM elements for previous data

Using Backbone.sync to customize dataType

The functionality of nested routing is not operating properly in react-router

Receiving an error in TypeScript stating that the property or data does not exist on the type for Vue props

Utilize React-router to display child components on the webpage

Forwarding parameter data type

What is the method to retrieve the base host in AngularJS?

The addition operation in JavaScript seems to be malfunctioning as it is not adding up my values

Tips on aligning a div to the right of a headline

GetServerSideProps function yielding varied prop values

What methods can be used to get npx to execute a JavaScript cli script on a Windows operating system

Converting JSONSchema into TypeScript: Creating a structure with key-value pairs of strings

Select the first item that is visible and chosen

Integrate NodeJs into Ionic framework for enhanced functionality and dynamic

Determine if every object in the array contains a specific key value pair

Can an AJAX upload progress bar be implemented in Internet Explorer without using Flash?

What is the best way to add a constant value to all objects within an array without having to iterate through each one

Providing structured Express app to deliver HTML and JavaScript content

Adding information to a single class that shares a common name

Utilizing puppeteer-core with electron: A guide

Answer №1

Option 1: Leveraging puppeteer-in-electron

Option 2: Obtain the debugging port and establish connection

Additional Information

Answer №2

Answer №3

Answer №4

my packages version

Similar questions

Searching dynamically using class names with JQuery

Angular JS effectively prevents redundant data from being displayed to users when scrolling infinitely while also efficiently removing DOM elements for previous data

Using Backbone.sync to customize dataType

The functionality of nested routing is not operating properly in react-router

Receiving an error in TypeScript stating that the property or data does not exist on the type for Vue props

Utilize React-router to display child components on the webpage

Forwarding parameter data type

What is the method to retrieve the base host in AngularJS?

The addition operation in JavaScript seems to be malfunctioning as it is not adding up my values

Tips on aligning a div to the right of a headline

GetServerSideProps function yielding varied prop values

What methods can be used to get npx to execute a JavaScript cli script on a Windows operating system

Converting JSONSchema into TypeScript: Creating a structure with key-value pairs of strings

Select the first item that is visible and chosen

Integrate NodeJs into Ionic framework for enhanced functionality and dynamic

Determine if every object in the array contains a specific key value pair

Can an AJAX upload progress bar be implemented in Internet Explorer without using Flash?

What is the best way to add a constant value to all objects within an array without having to iterate through each one

Providing structured Express app to deliver HTML and JavaScript content

Adding information to a single class that shares a common name

Option 1: Leveraging `puppeteer-in-electron`