Navigating through multiple pages using an Observable in Angular

After countless attempts, I still haven't been able to figure it out. Any assistance would be greatly appreciated; I recently came across Angular and RxJs.

The issue I'm facing involves a service that fetches resources from various URLs of the swapi API. The problem is that I don't know in advance how many pages need to be fetched. To tackle this, I have been using concat for each http.get(url) call to create observables.
Currently, only the first page of data is being displayed in the component (i.e. firstPage); all requests are being sent though.

export class PeoplesService {
  urlSource = "https://swapi.co/api/people/";
  pageResponse: GeneralResponse<People>;
  fullResponse: Observable<GeneralResponse<People>>;

  constructor(private _http:HttpClient) {
  }

  getPaged(n: number): string {
    return this.urlSource + "?page=" + n;
  }

  fetch(): Observable<GeneralResponse<People>> {
    let firstPage = this._http
                      .get<GeneralResponse<People>>(this.urlSource);
    firstPage.subscribe(page => {
      this.fullResponse = firstPage; // first page fetched
      let pageToDownload = Math.ceil(page.count / page.results.length);
      for(let i=2; i<=pageToDownload; i++) {
        // Merge all observable (so all request) into one
        concat(this.fullResponse,
               this._http.get<GeneralResponse<People>>(this.getPaged(i)));
      }
    });
    return this.fullResponse;
  }
}

This is the basic code structure of my component:

  ngOnInit() {
    this.peoplesService.fetch().subscribe(r => this.movies = r.results);
    // perhaps something like fetch().onNextFetch(this.movies.push(...r.results)) would work better here
    // as every piece of data on all pages needs to be merged into this.movies
    // or maybe we require a fetch().subscribeUntilCompleted(r => this.peoples = r.results)
  }

I've been struggling to find an alternative to using subscribe - something that waits for the Observable to complete and collects all the returned values at once.

It seems like subscribe doesn't wait for the Observable's "onCompleted" status and doesn't get triggered each time to capture all the existing values. How can I fetch all the data?

Is there a way to make the Observable behave like a stream and direct it to this.peoples.push(...r.results)? I'm not sure if I'm heading in the right direction with this approach.

Answer №1

After several hours of work, I finally found the solution. By utilizing mergeMap() to retrieve data from the initial Observable and merge() to combine all Observables into one.

import { HttpClient } from '@angular/common/http';
import { Observable, merge } from 'rxjs';
import { take, map} from 'rxjs/operators';

@Injectable({
  providedIn: 'root'
})
export class PeoplesService {

  private getPaged(n: number): string {
    return this.urlSource + "?page=" + n;
  }

  /**
   * Return an array containing natural numbers.
   * @param size : size
   * @param start : first number
   * Examples: 
   *    naturalNumberArrayFactory(3) returns [1, 2, 3]
   *    naturalNumberArrayFactory(3, 2) returns [3, 4, 5]
   */
  protected static naturalNumberArrayFactory(n: number, start = 1): number[]{
    return [ ...Array(n).keys() ].map(x => x + start);
  }

  fetch(): Observable<GeneralResponse<People>> {
    return this._http
      .get<GeneralResponse<T>>(this.urlSource)
      .pipe(take(1), map(firstPage => {
        // Determine number of pages to download (subtracting firstPage)
        let pageToDownload = Math.ceil(firstPage.count / firstPage.results.length) - 1;
        // Generate array of ids and replace them with GET requests
        let observables = GenericService.naturalNumberArrayFactory(pageToDownload, 2)
          .map(id => this._http.get<GeneralResponse<T>>(this.getPaged(id)));
        // Add the first page to avoid duplicate request
        observables.unshift(of(firstPage));
        return merge(...observables);
      }));
  }
}

Answer №2

Perhaps you might be misusing the concat function:

Concat doesn't alter the original arrays, but instead creates a new array that combines the values from both arrays.

For instance

const mergedArray = concat(array1, array2);
mergedArray.subscribe(item => console.log(item));

rxjs.concat

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Develop a custom data structure by combining different key elements with diverse property values

Within my coding dilemma lies a Union of strings: type Keys = 'a' | 'b' | 'c' My desire is to craft an object type using these union keys, with the flexibility for assigned values in that type. The typical approach involves ...

Unable to incorporate .tsx files into a Node.js Web Application project

After creating a new Node.js Web Application in Visual Studio 2015, I encountered an issue with adding and compiling .tsx files to the project. Instead of being added to the actual project, the .tsx file was placed into a Virtual Project. The project is co ...

Heroku build is reporting that it cannot locate the `@types` in the package.json file

Encountered Heroku Build Error - TSError: ⨯ Struggling to compile TypeScript: - src/server.ts(1,38): error TS7016: File declaration for module 'express' not found. '/app/node_modules/express/index.js' is implicitly of type 'any&a ...

Issue: ALERT found in ./src/styles.scss (./node_modules/raw-loader!./node_modules/postcss-loader/lib?

Hello to all my fellow developers! I've been diving into the Angular 6 project recently and things are going smoothly for the most part. However, I keep encountering a warning in the console after running the application with (ng serve). WARNING i ...

How can I detect if a control value has been changed in a FormGroup within Angular 2? Are there any specific properties to look

I am working with a FormGroup that contains 15 editable items, including textboxes and dropdowns. I am looking to identify whether the user has made any edits to these items. Is there a specific property or method I can use to check if the value of any i ...

Invoke a static method from within a class in Typescript (Angular HttpInterceptor)

Recently, I've been working on an http interceptor that was functioning smoothly until just yesterday. It consists of static methods, and for some reason, one of them is now causing issues. Here is the error message displayed in the console: my.c ...

No interface 'JSX.IntrinsicElements' could be found, causing the JSX element to be implicitly of type 'any'

<Header> <title>Real Estate API Application</title> <meta name="description" content="Generated by create next app" /> <meta name="viewport" content="width=device-width, ...

Error message indicating unfulfilled peer dependency in Ionic Angular when using npm

Having trouble integrating the angular google maps package npm install @agm/core Encountering errors with unmet peer dependencies, unsure of the reason. Could it be that the version of Angular in my project is incompatible with the agm/core package? This ...

Introducing a delay in an observable causes incomplete data to be received in Angular using rxjs

Currently, I am facing an issue in my code where I am trying to introduce a delay using timer(500). However, the problem is that it is only returning partial data. Instead of the expected 17 fields, it is only returning 2 fields. Below is my code snippet f ...

When I attempt to conceal the filter within mat-table using *ngIf, I encounter an issue where I am unable to read the property 'value' due to it being

After creating a mat-table, I encountered an issue when trying to show and hide my input filter area using a button. If I use *ngIf="showInputFilter" to hide the filter area, I receive the error message Cannot read property 'value' of u ...

Troubleshooting HttpErrorResponse in an Angular App with PHP API

In my Angular application, I am utilizing Angular HttpClient along with PHP on the backend to handle data. However, I encountered an error while attempting to save data. Error Encountered: To connect to the database and pass data, I am using the database ...

Positioning CSS for a Responsive Button

Currently, I am working on making my modal responsive. However, I am encountering an issue with the positioning of the "SAVE" button. The button is not staying in the desired position but instead disappears. Below is the snippet of my HTML and CSS: .dele ...

Retrieving data from a different component in Angular 7

I need to separate the navbar and form-dialog components. I want to be able to access a value from form-dialog in the navbar. Here is my code for navbar.ts: import { Component, OnInit } from "@angular/core"; import { MenuItemModels } from "./models/MenuI ...

After unsubscribing from RxJS timer, it starts again

Trying out a simple reflex-testing game here. The player has to click on the red rectangle when it turns green, and their score is displayed. However, the issue is that the timer restarts after clicking on the rectangle even though I tried using 'unsu ...

What is the best way to add a table header with a column of interactive buttons in Angular?

I am currently utilizing Angular and have created a table displaying important data. The first column features checkboxes for individual selection or selecting all. Following the checkbox column are additional columns of data for updating/deleting informat ...

What is the easiest way to choose a child vertex with just one click on mxgraph?

I have nested vertices and I'm looking to directly select the child vertex with just one click. Currently, when I click on a child vertex, it first selects the parent vertex instead. It's selecting the parent vertex initially: To select the ch ...

One cannot use a type alias as the parameter type for an index signature. It is recommended to use `[key: string]:` instead

I encountered this issue in my Angular application with the following code snippet: getLocalStreams: () => { [key: Stream['key']]: Stream; }; During compilation, I received the error message: An index signature parameter typ ...

What is the best way to retrieve data in my client component without risking exposing my API key to unauthorized users?

To retrieve information, I plan to use pagination in order to specify a particular page number within the API URL and fetch additional data by updating the value of page. The process of fetching data in my server component is as follows: // fetchData.tsx ...

Testing Angular application with a currency pipe results in an error stating "currency

Utilizing the built-in angular currency pipe in my components works perfectly fine. However, when attempting to unit test my component, an error occurs: https://i.stack.imgur.com/J18JL.png I am using Angular 10 with Ivy and have imported the CommonModule, ...

The Node Server running on localhost's Port 4200 cannot be accessed through the Web Browser

Running an Angular server on my Pixelbook in Dev Mode has proven to be quite challenging. While I have successfully done this numerous times on a traditional Ubuntu development box, there seems to be something about this Chrome-based environment that is ca ...