Tips for obtaining the iframe #document with cheeriojs?

I've been struggling to scrape the anime videos page [jkanime], specifically with extracting the mp4 video formats embedded in an iframe #document.

Despite trying to use cheerio for querying, I've only managed to retrieve src links from Facebook plugins instead of the desired mp4 sources within the iframe.

After entering the following in chrome dev tools: $('#jkvideo_html5_api source')

The mp4 src is displayed, but cheerio doesn't yield any results with the same query.

I've been attempting to extract the mp4 links for weeks without success. Any assistance would be greatly appreciated.

Image

devtool source code section

  const getAnimeVideo = async (id: string, chapter: number) => {
    const res = await fetch(`${url}${id}/${chapter}/`);
    const body = await res.text();
    const $ = cheerio.load(body);
    const arr = [];
    $('iframe').each((index, element) => {
      const $element = $(element);
      const x = $element.attr('src');
      console.log(x);
      arr.push(x);
    });
    return arr;
}

Current Output

{
  "videos": [
    "https://www.facebook.com/plugins/like.php?href=https%3A%2F%2Fwww.facebook.com%2Fjkanimetv%2F&width=132&layout=box_count&action=like&size=large&show_faces=false&share=false&height=21&appId=149291901844100",
    "https://www.facebook.com/plugins/like.php?href=https://jkanime.net/tokyo-ghoul/1/&width=76&layout=box_count&action=like&size=small&show_faces=false&share=false&height=65&appId=149291901844100"
  ]
}

Desired Output

{
  "videos": [
    "https://storage.googleapis.com/markesito.appspot.com/blakkkk-88.mp4"
   ]
}

Update: 10:52 pm

Through the use of puppeteer, I was able to access the iframe with the class "player_conte," resulting in the output shown in the terminal:

_navigationURL

Now, my challenge lies in retrieving the link from _navigationURL in order to make reference to the video source using cheerio.

Updated Code

const getAnimeVideo = async (id: string, chapter: number) => {
  const BASE_URL = `${url}${id}/${chapter}/`  // => https://jkanime.net/tokyo-ghoul/1/
  const browser = await puppeteer.launch() 
  const page = await browser.newPage()
  await page.goto(BASE_URL);

  const elementHandle = await page.$('.player_conte')
  const frame = await elementHandle.contentFrame();
  const $ = cheerio.load(`${frame}`);
  console.log(frame)
}

Answer №1

Resolved the issue by utilizing puppeteer

const fetchAnimeVideo = async (id: string, episode: number) => {
  const BASE_URL = `${url}${id}/${episode}/`  // => https://jkanime.net/tokyo-ghoul/1/
  const browser = await puppeteer.launch() 
  const page = await browser.newPage()
  await page.goto(BASE_URL);
  const elementHandle = await page.$('.player_conte')
  const frame = await elementHandle.contentFrame();
  const video = await frame.$eval('#jkvideo_html5_api', el =>
  Array.from(el.getElementsByTagName('source')).map(e => e.getAttribute("src")));
  return video;
 }

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Utilizing Node.js to Retrieve a POST Request JSON and Modify its Format

Received an incoming Post request in JSON format like this: [{"username":"ali","hair_color":"brown","height":1.2},{"username":"marc","hair_color":"blue","height":1.4},{"username":"zehua","hair_color":"black","height":1.8}] Need to transform it into the f ...

To trigger OnClick events in Next.js SSG, you will need to double click the element for it to run properly

There seems to be an issue where elements with an onClick event listener require a second click to run. It appears that the state is not updating on the initial click, causing this behavior. This problem may be related to using getStaticProps() for SSG on ...

Verify WTForm after dynamic updates to select field options with Jquery

As I work on developing a flask application, I have successfully created a form using WTForms. This form consists of two SelectFields (dropdowns) and a submit button. My goal is to make the dropdowns dynamic - meaning that when the user selects an option f ...

Incorporate Y-axis titles onto D3 bar chart using the attribute 'name' from JSON data

While following the tutorial on creating a bar chart, I encountered an issue in step three. The bars are rotated to columns, but I am struggling to iterate over a JSON dataset and add Y-axis labels for each bar using the name attribute from the returned JS ...

Creating a dynamic dropdown list with PHP and AJAX using JQuery

I was attempting to create a dynamic dependent select list using AJAX, but I am facing issues with getting the second list to populate. Below is the code I have been working with. The gethint.php file seems to be functioning properly. I'm not sure whe ...

Animation for maximum height with transition from a set value to no maximum height

While experimenting with CSS-transitions, I encountered an unusual issue when adding a transition for max-height from a specific value (e.g. 14px) to none. Surprisingly, there is no animation at all; the hidden elements simply appear and disappear instant ...

Issue: Actions should be in the form of plain objects. However, the given type is 'Promise'. To resolve this, consider incorporating middleware into your React Native store

Currently, I am facing an issue while trying to retrieve products from Firebase. Despite having redux-thunk installed to manage promises and using middleware in my store, I encountered the following error: Actions must be plain objects. The actual type d ...

Sending Ajax data to a controller function

I am looking for guidance on how to pass values from my view to my controller using ajax. As someone who is new to working with ajax, I would appreciate some assistance in understanding the process. Full Page @model ALSummary.Models.MonthReport @{ ...

error detection in AJAX response handler

My web-application was created using PHP, AJAX, and jQuery, and the development process went smoothly. The majority of the requests to the application are made via AJAX for operations such as insert, update, delete, and select. I have already implemented ...

Send a single piece of data using AJAX in Flask

I have a very basic HTML form containing only one <input type='text'> field for entering an email address. I am trying to send this value back to a Python script using AJAX, but I am having trouble receiving it on the other end. Is there a ...

Using Jquery to increase input values in multiples on keyup event

Is there a way to ensure that an input element only accepts numbers in increments of 50? While we know we can use the step="50" attribute, is it possible to achieve this using the keyup event instead? I came across this code that restricts users from inp ...

Updating a field in Mongoose by referencing an item from another field that is an array

I have developed an innovative Expense Tracker Application, where users can conveniently manage their expenses through a User Collection containing fields such as Name, Amount, Expenses Array, Incomes Array, and more. The application's database is p ...

Ways to initiate a page redirection within the componentWillReceiveProps lifecycle method

When my webpage or component generates a form and sends it to the backend API upon submission, I receive an object in return if the process is successful. This object is then added to my redux store. In order to determine whether the reducer successfully ...

Discovering the import path of Node modules in ReactAlgorithm for determining the import path of

Software Development In my current project, I am utilizing Typescript along with React. To enhance the application, I integrated react-bootstrap-date-picker by executing yarn install react-bootstrap-date-picker. Unfortunately, there is no clear instruct ...

Experience dynamic data transformations with Vue's server-side rendering feature

Incorporating Vue into server-side rendering presents a challenge when the content data within the template needs to be fetched from another CMS server. <template> <h1>{{ content.heading }}</h1> </template> <script> expo ...

Navigate to a different page using Angular with a simple click

How can I implement a redirect from clicking on the "Firms" word to another component page in Angular? I have tried using routerLink="/", [routerLink]="['/']". I have imported Routes in the app.module. I have also attempted this approach: import ...

How do I insert a new column into the result set of a query in Prisma?

Imagine a scenario where there are two tables: User, which has fields for name and Id, and Post, which has fields for name and content. These tables are connected through a many-to-many relationship (meaning one post can have multiple users/authors and eac ...

Oops! Remember to always `await server.start()` first before using `server.createHandler()` in next.js

An error is popping up when I attempt to check the functionality of Apollo GraphQL. Error: You must await server.start() before calling server.createHandler() Note: Although there is a similar question regarding this issue, it is specific to Express. Error ...

When `strictNullChecks` is turned on, how does the `void` type differ from the `undefined` literal type?

When strictNullChecks is turned on: (u: undefined, v: void, n: null) => { v = u; u = v; // type error: Type 'void' is not assignable to type 'undefined' v = n; // type error: Type 'null' is not assignable to type &ap ...

finding the adjacent li element using the document.(property)

Utilizing a pub/sub solution named ably.io for real-time data updates, I have implemented a method that assigns dynamic ids to each ngFor listing. This allows me to easily identify and update the values received from ably.io subscribe. document.getElement ...