Cheerio fails to retrieve items from the specified directory

Question

Cheerio fails to retrieve items from the specified directory

My main goal with cheerio is to scrape the titles from this IMDb ranking:

Despite following the documentation and specifying the exact HTML path for the titles, I am getting back random and confusing objects like:

  'x-attribsNamespace': [Object: null prototype] {},
    'x-attribsPrefix': [Object: null prototype] {}
  },
  '80': <ref *81> Element {
    parent: Element {
      ...

Code:

import * as cheerio from 'cheerio';
import axios from 'axios';
import fs from 'fs';


axios("https://www.imdb.com/chart/tvmeter/?ref_=nv_tvv_mptv").then(res => {
    const data = res.data;
    const $ = cheerio.load(data);

    var cheerioData = $('.lister-list>tr').each((i, e) => {
        const title = $(e).find('.titleColumn a').text();
        console.log(title);
    })
    console.log(cheerioData);
})

I'm struggling to figure out what's going wrong since the path seems to be correct. Can anyone offer some help?

typescript web-scraping axios cheerio

Answer 1

Answer №1

To transform the Cheerio objects array into a text array, you can utilize map in combination with spread syntax, .get(), or .toArray().

For instance, employing spread and vanilla JS Array#map:

import axios from "axios";
import cheerio from "cheerio";

const url = "<Your URL>";

axios(url).then(res => {
  const $ = cheerio.load(res.data);
  const text = [...$(".lister-list > tr")].map(e =>
    $(e).find(".titleColumn a").text().trim()
  );
  console.log(text);
})

Alternatively, you can use .get() or .toArray() after executing a Cheerio .map (where the index serves as the first argument):

const text = $(".lister-list > tr").map((i, e) =>
  $(e).find(".titleColumn a").text().trim()
).get();

If you opt for .each, you have the option to .push() each text string onto a vanilla array, although this approach isn't as elegant as utilizing .map which simplifies this process:

const text = [];
$(".lister-list > tr").each((i, e) => {
  text.push($(e).find(".titleColumn a").text().trim());
});

Answer 2

To transform the Cheerio objects array into a text array, you can utilize map in combination with spread syntax, .get(), or .toArray().

For instance, employing spread and vanilla JS Array#map:

import axios from "axios";
import cheerio from "cheerio";

const url = "<Your URL>";

axios(url).then(res => {
  const $ = cheerio.load(res.data);
  const text = [...$(".lister-list > tr")].map(e =>
    $(e).find(".titleColumn a").text().trim()
  );
  console.log(text);
})

Alternatively, you can use .get() or .toArray() after executing a Cheerio .map (where the index serves as the first argument):

const text = $(".lister-list > tr").map((i, e) =>
  $(e).find(".titleColumn a").text().trim()
).get();

If you opt for .each, you have the option to .push() each text string onto a vanilla array, although this approach isn't as elegant as utilizing .map which simplifies this process:

const text = [];
$(".lister-list > tr").each((i, e) => {
  text.push($(e).find(".titleColumn a").text().trim());
});

Cheerio fails to retrieve items from the specified directory

Answer №1

Similar questions

Developing applications using ReactJS with Typescript can sometimes lead to errors, such as the "onclick does not exist on type x

Integrating Constant Contact API into a Next.js application

Sending data to a React component from regular HTML

Instructions for sending a PNG or JPEG image in binary format from a React app to a Node.js server

Unable to perform type casting in Typescript

Why do referees attempt to access fields directly instead of using getters and setters?

What is the reason for encountering the error message "Property 'floatThead' does not exist on type 'JQuery<any>' when trying to use floatThead in Angular/TypeScript?

Having trouble with gsap.reverse() not functioning properly when using onMouseLeave event in React

There are no properties shared between type 'dateStyle: string' and type 'DateTimeFormatOptions'

Unable to mock ESM with unstable_mockModule

Several arrays within the filteredData (MatTableDataSource) are being utilized

Error in Typescript: A computed property name must be one of the types 'string', 'number', 'symbol', or 'any'

How can I display an agm-polyline within a map in Angular 7?

How to use TypeScript variables in React applications

Functions outside of the render method cannot access the props or state using this.props or this.state

Experiencing trouble accessing a property in TypeScript

Expanding User Type to Include Extra User Data Using NextAuth.js

Guide on filtering FlatList Data in react native by selecting multiple categories from an array

Angular time-based polling with conditions

Angular - Highlight a section of a string variable