Navigating through multiple pages using an Observable in Angular

After countless attempts, I still haven't been able to figure it out. Any assistance would be greatly appreciated; I recently came across Angular and RxJs.

The issue I'm facing involves a service that fetches resources from various URLs of the swapi API. The problem is that I don't know in advance how many pages need to be fetched. To tackle this, I have been using concat for each http.get(url) call to create observables.
Currently, only the first page of data is being displayed in the component (i.e. firstPage); all requests are being sent though.

export class PeoplesService {
  urlSource = "https://swapi.co/api/people/";
  pageResponse: GeneralResponse<People>;
  fullResponse: Observable<GeneralResponse<People>>;

  constructor(private _http:HttpClient) {
  }

  getPaged(n: number): string {
    return this.urlSource + "?page=" + n;
  }

  fetch(): Observable<GeneralResponse<People>> {
    let firstPage = this._http
                      .get<GeneralResponse<People>>(this.urlSource);
    firstPage.subscribe(page => {
      this.fullResponse = firstPage; // first page fetched
      let pageToDownload = Math.ceil(page.count / page.results.length);
      for(let i=2; i<=pageToDownload; i++) {
        // Merge all observable (so all request) into one
        concat(this.fullResponse,
               this._http.get<GeneralResponse<People>>(this.getPaged(i)));
      }
    });
    return this.fullResponse;
  }
}

This is the basic code structure of my component:

  ngOnInit() {
    this.peoplesService.fetch().subscribe(r => this.movies = r.results);
    // perhaps something like fetch().onNextFetch(this.movies.push(...r.results)) would work better here
    // as every piece of data on all pages needs to be merged into this.movies
    // or maybe we require a fetch().subscribeUntilCompleted(r => this.peoples = r.results)
  }

I've been struggling to find an alternative to using subscribe - something that waits for the Observable to complete and collects all the returned values at once.

It seems like subscribe doesn't wait for the Observable's "onCompleted" status and doesn't get triggered each time to capture all the existing values. How can I fetch all the data?

Is there a way to make the Observable behave like a stream and direct it to this.peoples.push(...r.results)? I'm not sure if I'm heading in the right direction with this approach.

Answer №1

After several hours of work, I finally found the solution. By utilizing mergeMap() to retrieve data from the initial Observable and merge() to combine all Observables into one.

import { HttpClient } from '@angular/common/http';
import { Observable, merge } from 'rxjs';
import { take, map} from 'rxjs/operators';

@Injectable({
  providedIn: 'root'
})
export class PeoplesService {

  private getPaged(n: number): string {
    return this.urlSource + "?page=" + n;
  }

  /**
   * Return an array containing natural numbers.
   * @param size : size
   * @param start : first number
   * Examples: 
   *    naturalNumberArrayFactory(3) returns [1, 2, 3]
   *    naturalNumberArrayFactory(3, 2) returns [3, 4, 5]
   */
  protected static naturalNumberArrayFactory(n: number, start = 1): number[]{
    return [ ...Array(n).keys() ].map(x => x + start);
  }

  fetch(): Observable<GeneralResponse<People>> {
    return this._http
      .get<GeneralResponse<T>>(this.urlSource)
      .pipe(take(1), map(firstPage => {
        // Determine number of pages to download (subtracting firstPage)
        let pageToDownload = Math.ceil(firstPage.count / firstPage.results.length) - 1;
        // Generate array of ids and replace them with GET requests
        let observables = GenericService.naturalNumberArrayFactory(pageToDownload, 2)
          .map(id => this._http.get<GeneralResponse<T>>(this.getPaged(id)));
        // Add the first page to avoid duplicate request
        observables.unshift(of(firstPage));
        return merge(...observables);
      }));
  }
}

Answer №2

Perhaps you might be misusing the concat function:

Concat doesn't alter the original arrays, but instead creates a new array that combines the values from both arrays.

For instance

const mergedArray = concat(array1, array2);
mergedArray.subscribe(item => console.log(item));

rxjs.concat

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Error encountered in Typescript while attempting to $set a subdocument key value in MongoDB

This is a sample data entry. { _id: ObjectId('63e501cc2054071132171098'), name: 'Ricky', discriminator: 7706, registerTime: ISODate('2023-02-09T14:23:08.159Z'), friends: { '63e502f4e196ec7c04c4 ...

Next.js Troubleshooting: Unexpected Issue with 'use client' Triggering Error in React Client Component

Keeping it simple, here is a useState function that I am using: 'use client' import {useState} from 'react'; const Home = () => { const [fruit, setFruit] = useState('apple'); return ( & ...

Is there a way to display my modal separately from my sidenav while utilizing :host in Angular?

I implemented a :host with hostlistener() in my navmenu-component.ts to enable a sidemenu that slides out from my sidenavbar when a button is pressed. My goal is to display a modal for editing purposes. I have included the modal in the navmenu-component.h ...

Choose between using Angular with either PHP and Python or with Django and Python in PHP

For my graduation project, I have developed the frontend using Angular and created a machine learning system with Python. Now, I need to integrate these two components by writing a Web API for Angular using Django, even though I have no prior experience wi ...

Difficulty establishing a connection between Typescript and Postgres results in a prolonged

I am attempting to establish a connection to a Postgres database using typescript. For the ORM, I have opted for sequelize-typescript. The issue lies in the fact that the script seems to hang at await sequelize.sync();. Below is the content of the sequeliz ...

Ways to retrieve the component name from a service in Angular without relying on private APIs such as view container refs

How can I access the component name inside a service that is calling a method within the service? I have tried using console.log(this.vcr['_view'].component) and console.log(this.vcr['_view'].component.constructor.name), but they do not ...

How can I access a service without the need to import its provider module?

My goal is to grasp the concept of multiple NgModules in an angular application and how they interact, specifically focusing on importing a SharedModule for commonly used services into lazily loaded feature modules. Here's the sequence of steps I fol ...

Angular 2: The *ngFor directive is unable to locate a suitable differing framework

Below is the code for client.service.ts clients: Client[]; getClientList() { let headers = new Headers(); headers.append('Content-Type', 'application/json'); let authToken = localStorage.getItem('auth_token&apo ...

Access to Angular CORS request has been blocked

I'm currently working on establishing a connection between my Angular application and a basic REST server using express. The server responds to requests with JSON data exclusively. To enable CORS support, I've integrated the cors module from npm ...

What is the purpose of declaring data types such as string, number, boolean, void, and others in Angular 2

I'm curious as to why programmers declare classes in individual scope or something. For example, take a look at this code snippet: result: boolean = false public generateRand(x): void { } This is also similar to the following: result = false publi ...

The latest version of Angular, Angular 16, brings along its own set of challenges

Just completed the update to version 16 of Angular and encountered the following error message: The injectable CustomMsalInterceptor inherits its constructor from MsalInterceptor, but MsalInterceptor does not have its own Angular decorator. This will resu ...

Having trouble clicking on a button with Protractor because the button text is located within a child span element

Having trouble clicking a button with protractor. The DOM structure is displayed in the image above. Here are some of the locators I've attempted to use: element(by.xpath("(//div[@class='mat-drawer-backdrop ng-star-inserted'])//a followin ...

"Looking to swap out the Angular theme's CSS stylesheet for your entire application? Here's

I was initially facing an issue while trying to import CSS through index.html as I kept getting a MIME type error: enter image description here The browser refused to apply the stylesheet from 'http://localhost:4200/css/pink-bluegrey.css' because ...

Utilizing Angular HTTP Interceptor to Show Loading Spinner Across Multiple Modules

My goal is to utilize the ng4-loading-spinner spinner when making HTTP calls to my API. I referred to the examples provided in the following resources: Angular Guide on Intercepting HTTP Requests/Responses Stack Overflow Post on Using HttpClient Interce ...

Struggling to dynamically update array values by comparing two arrays

I am faced with a scenario where I have two arrays within an Angular framework. One of the arrays is a regular array named A, containing values such as ['Stock_Number', 'Model', 'Type', 'Bill_Number'] The other arr ...

Encountering an error message stating "Unsupported file format" when attempting to upload an Excel file to Azure Data

I am currently working on a project that involves user file input in an application using angular7 and .net core. The payload is sent to the backend through a websocket. Although I have successfully uploaded files to azure datalake, there seems to be an is ...

What is the best way to include rxjs in an npm library - as a dependency, peer dependency, or both?

After researching numerous posts and articles on dependencies versus peerDependencies, I am still not entirely certain what to do in my particular situation.... I have a library (which is published to a private npm repository) that utilizes rxjs; for exam ...

Issue with VueJS 2 and TypeScript: computed value unable to recognize property specified in data object

When creating the following component: <template lang="html"> <div> <p>{{ bar }}</p> </div> </template> <script lang="ts"> import Vue from 'vue'; export const FooBar = Vue.ex ...

Having trouble typing computed values in Vue Composition API with TypeScript?

I'm attempting to obtain a computed value using the composition API in vue, and here is the code I am working with: setup() { const store = useStore(); const spaUrls = inject<SpaUrls>('spaUrls'); const azureAd = inject<AzureAd ...

Retrieve a specific element from an array list

Hey everyone, I'm facing a problem in my application where I need to extract a specific value from my array and display it in a table for users to see. Check out the code snippet below: Here's my mock data: { "Data": "2020-0 ...