Tips for efficiently loading data in a sequence using rxjs: within each sequence, there could be multiple calls made

Question

Tips for efficiently loading data in a sequence using rxjs: within each sequence, there could be multiple calls made

const dates = [{start:'03/06/2020', end: '03/09/2020'}, {start:'03/12/2020', end: '03/03/2021'}, ...]

const fetchData =  service.get(page = 1, dates[0]) and service.get(page = 2, dates[0])

retrieves the necessary data, but considering multiple pages might be involved. Each page's data must be collected before moving on to the next date.

data.subscribe(resp=> somesubject.next(this.values = this.values.concat(resp)))

'resp' represents the result of each API call. The values for each date are merged together on the client side.

What would be the most effective approach to manage this situation? Feel free to ask if you need further clarification.

angular typescript rxjs angular5 angular8

Answer 1

Answer №1

Assuming the following initial conditions / assumptions:

dates represents an array of date intervals to facilitate data querying.
PageResponse is a response type from the api service, containing the current page number and returned items.
getPage is a method that returns an Observable<Response> for a specific date range and page number (starting from 0). When there are no more results, the response includes an empty array of items.

  export type PageResponse = {
    page: number;
    items: any[];
  };

  getPage(dateInterval: any, page: number): Observable<Response> {
    // use dateInterval to query expected data...
    // or call service.get...
    return this.http.get<PageResponse>(`http://.../pages/${page}`);
  }

You can implement

from(dates)
      .pipe(
        concatMap(dateInterval =>
          this.getPage(dateInterval, 0).pipe(
            expand(response => {
              if (response.items?.length) {
                return this.getPage(dateInterval, response.page + 1);
              } else {
                return EMPTY;
              }
            }),
            map((page) => page.results)
          )
        ),
        mergeAll(),
        toArray()
      )
      .subscribe(results => {
         // results is a flat array of all items returned
         ...
      });

This code serves as an illustration and should be customized to fit your specific scenario. Some types may be omitted (such as items), but this is to simplify the example.

Answer 2

Assuming the following initial conditions / assumptions:

dates represents an array of date intervals to facilitate data querying.
PageResponse is a response type from the api service, containing the current page number and returned items.
getPage is a method that returns an Observable<Response> for a specific date range and page number (starting from 0). When there are no more results, the response includes an empty array of items.

  export type PageResponse = {
    page: number;
    items: any[];
  };

  getPage(dateInterval: any, page: number): Observable<Response> {
    // use dateInterval to query expected data...
    // or call service.get...
    return this.http.get<PageResponse>(`http://.../pages/${page}`);
  }

You can implement

from(dates)
      .pipe(
        concatMap(dateInterval =>
          this.getPage(dateInterval, 0).pipe(
            expand(response => {
              if (response.items?.length) {
                return this.getPage(dateInterval, response.page + 1);
              } else {
                return EMPTY;
              }
            }),
            map((page) => page.results)
          )
        ),
        mergeAll(),
        toArray()
      )
      .subscribe(results => {
         // results is a flat array of all items returned
         ...
      });

This code serves as an illustration and should be customized to fit your specific scenario. Some types may be omitted (such as items), but this is to simplify the example.

Tips for efficiently loading data in a sequence using rxjs: within each sequence, there could be multiple calls made

Answer №1

Similar questions

There are no properties shared between type 'dateStyle: string' and type 'DateTimeFormatOptions'

Verify user using Cognito user pool log-in information

Tips for transferring state information from a client to a server component within Nextjs?

Create an observable array that contains both outer and inner response properties within an Angular application

"Passing Data from Angular Parent Component to Child Component

Unable to locate the module from my personal library in Typescript

The error message "localStorage is undefined in Angular Universal" indicates that the local

Make sure to include a warning in the renderItem prop of your Flashlist component

Struggling to refine the result set using jsonpath-plus

Unable to open modal externally without encountering an error in MaterializeCSS

Finding the solution to the perplexing issue with Generic in TypeScript

What is the best method for compressing and decompressing JSON data using PHP?

Validate that the input is an array

Customize YouTube iframe styles in Angular 4+ with TypeScript

I'm facing issues with Angular commands not functioning properly even after installing the Angular CLI and configuring the

What purpose does the array.pop()!(object) syntax serve within Codemirror?

Error in Template Syntax for External Pug Templates: Component template must have a root element, not just plain text

"Exploring the Power of Angular 2 with Route Resolving

A guide on transforming JSON data to an array format where nested arrays are encapsulated within objects using curly braces instead of square brackets in TypeScript

Angular: encountering template parse errors with unknown <component> element