Tips for finalizing a subscriber after a for loop finishes?

When you send a GET request to , you will receive the repositories owned by the user benawad. However, GitHub limits the number of repositories returned to 30.

The user benawad currently has 246 repositories as of today (14/08/2021).

In order to workaround this limitation, you can make a GET request with additional parameters. GitHub uses pagination in its API, so you need to specify the desired page and the number of repositories per page in the URL.

Your modified GET request should resemble the following:

However, GitHub restricts the maximum number of repositories per page to 100. Therefore, your modified GET request will only return 100 repos instead of the total 246 repositories owned by the user benawad.

I have implemented the code below in my Angular service to fetch all repositories from multiple pages:

public getUserRepos(user: string): Observable<RepositoryI[]> {
    return new Observable((subscriber: Subscriber<RepositoryI[]>) => {
        this.getUserData(user).subscribe((data: UserI) => {
            const pages: number = Math.ceil(data.public_repos / 100);

            for (let i = 1; i <= pages; i++) {
                this.http
                    .get(`https://api.github.com/users/${user}/repos?page=${i}&per_page=100`)
                    .subscribe((data: RepositoryI[]) => {
                        subscriber.next(data);
                    });
            }
        });
    });
}

To handle when the Observable has completed emitting values, I subscribe to it in my component with the following code snippet:

this.userService.getUserRepos(id).subscribe((repos)=>{
    this.repositories.push(...repos);
})

An issue arises where I lack control over determining when the Observable has finished emitting values. In my component, I want to execute a function when the Observable is complete.

I attempted the approach below:

public getUserRepos(user: string): Observable<RepositoryI[]> {
    return new Observable((subscriber: Subscriber<RepositoryI[]>) => {
        this.getUserData(user).subscribe((data: UserI) => {
            const pages: number = Math.ceil(data.public_repos / 100);

            for (let i = 1; i <= pages; i++) {
                this.http
                    .get(`https://api.github.com/users/${user}/repos?page=${i}&per_page=100`)
                    .subscribe((data: RepositoryI[]) => {
                        subscriber.next(data);
                        if(pages == i) subscriber.complete();
                    });
            }
        });
    });
}

In my component, I do the following:

this.userService.getUserRepos(id).subscribe(
    (repos) => {
        this.repositories.push(...repos);
    },
    (err) => {
        console.log(err);
    },
    () => {
        console.log(this.repositories); 
        // The output logs only 46 repositories instead of the expected 246
        // I wish to trigger a function here
    }
);

The console.log() statement displays 46 repositories instead of 246. This discrepancy may occur due to prematurely completing the subscriber before fetching all three pages. Even though I call .complete() within the subscription, there seems to be an error. Can someone guide me on what I might be doing incorrectly? Thank you in advance.

Answer №1

To accomplish this task with just one subscription, utilize RxJS operators and functions as shown below:

public fetchUserRepositories(user: string): Observable<RepositoryI[]> {
  // Create an observable that combines all repositories into one:
  return this.getUserData(user).pipe(
    switchMap((data: UserI) => {
      const pages: number = Math.ceil(data.public_repos / 100);

      // Combine results from multiple observables using forkJoin:
      return forkJoin(
        Array.from(new Array(pages)).map((_, page) =>
          this.http.get<RepositoryI[]>(
            `https://api.github.com/users/${user}/repos?page=${page + 1}&per_page=100`
          )
        )
      ).pipe(
        // Flatten the result array to a single RepositoryI[]:
        map((res) =>
          res.reduce((acc, value) => acc.concat(value), [])
        )
      );
    })
  );
}

In your component, subscribe to the observable to retrieve all repositories upon completion:

this.userService.fetchUserRepositories(id).subscribe((repos) => {
    this.repositories = repos;
    // Execute necessary actions here...
});

Answer №2

          const subscription = interval(1000).subscribe(() => {
          this.http
            .get(
              `https://api.github.com/users/${user}/repos?page=${i}&per_page=100`
            )
            .subscribe((data: RepositoryI[]) => {
              subscriber.next(data);
              if(!data) { // check for null data
                subscription.unsubscribe();  // cancel the interval subscription
                return;  // stop the subscription
              }
            });
          })
        }

This approach ensures that you handle the subscription properly by unsubscribing when there is no data.

Please confirm if this solution meets your requirements.

Answer №3

One interesting feature of Javascript is its ability to run things asynchronously.

For example, if you use a 'For loop' to make API calls in parallel, the count will increment simultaneously.

When making multiple parallel API calls, each call may return at different times.

However, there is a possibility that before all API calls complete, the value of 'i' reaches 3.

In such cases, the third API call with fewer records might reach the subscribe logic first, resulting in only 46 records being displayed.

subscriber.next(data); // 3rd api comes here 
              
if(pages == i) subscriber.complete();  // now your pages and i values are same and it gets completed.
            });

SOLUTION


import {defer } from 'rxjs';

defer(async () => {
     for (let i = 1; i <= pages; i++) {
          await this.http
            .get(
              `https://api.github.com/users/${user}/repos?page=${i}&per_page=100`
            )
            .toPromise().then((data: RepositoryI[]) => {
              subscriber.next(data);
            });
        }
}.subscribe(x => { subscriber.complete(); });

Answer №4

Outline

In this explanation, I will outline two different methods to accomplish the task at hand. The first approach involves an imperative way (utilizing a loop), while the second method is reactive in nature (using rxjs).

Imperative Approach

To achieve this imperatively, you can implement a simple loop to retrieve all the pages from the API endpoint. This function returns a promise, but it can be converted into an observable if required by using from:

public getUserRepos(user: string): Promise<Repository[]> {
  let pageNumber = 0;
  let results = [];

  const getPage = pageNumber =>
    this.http
      .get<Repository[]>(`https://api.github.com/users/${user}/repos?page=${pageNumber}&per_page=100`)
      .toPromise();

  do {
    const pageResults = await getPage(pageNumber++);
    results = results.concat(pageResults);
  } while(pageResults.length !== 0);

  return results;
}

Reactive Approach

For a more reactive solution, you can utilize the expand operator provided by rxjs to fetch each page sequentially:

When implementing this method, ensure that the http response is mapped to an object containing the corresponding page number. This enables the expand operator to determine the subsequent page to request.

Here's an example in your context:

public getUserRepos(user: string): Observable<Repository[]> {
  const getPage = pageNumber =>
    this.http
      .get<Repository[]>(`https://api.github.com/users/${user}/repos?page=${pageNumber}&per_page=100`)
      .pipe(map(z => ({ results: z, page: pageNumber }))); // Include page number in results

  return getPage(0).pipe(
    expand(pr => getPage(pr.page + 1)),
    takeWhile(pr => pr.results.length !== 0),
    map(pr => pr.results),
    scan((acc, v) => [...acc, ...v])
  );
}

This implementation initiates a query for page 0, followed by consecutive requests for subsequent pages until a page returns no entries, resulting in the completion of the observable stream.

The use of scan allows for displaying partial results as pages are fetched, providing real-time feedback rather than waiting for all pages to load before showing any data.

Refer to this StackBlitz for a demonstration.

Note: There may be an additional query for a final page which will be cancelled. If you prefer to avoid this, adjust the expand call like so:

expand(pr => (pr.results.length !== 0 ? getPage(pr.page + 1) : EMPTY))

Answer №5

Here is a more concise approach to providing an answer. If the nested structure bothers you, consider defining an inner observable as a separate function.

public getUserRepos = (user:string) =>
  this.getUserData(user)
  .pipe(
    map(({public_repos})=>Math.ceil(public_repos / 100)),
    switchMap(max=>interval(1)
    .pipe(
      takeWhile(i=>++i<=max)
      mergeMap(i=>this.http.get<RepositoryI[]>(`https://api.github.com/users/${user}/repos?page=${++i}&per_page=100`))
    )),
    scan((acc, repo)=>[...acc, ...repo], [] as RepositoryI[])
  );

This was my strategy:

  1. Retrieve the page limit using object destructuring from getUserData().
  2. Utilize interval() to initiate a counting observable (starting with 0 and utilizing
    ++</code later on).</li>
    <li>Employ <code>takeWhile()
    to verify if the count surpasses the page limit; otherwise, the observable completes at this point.
  3. Use mergeMap() to line up a sequence of API calls for each number emitted by interval().
  4. Aggregate all responses into a singular array response with scan().

Update: When testing this method, your component logic will require an update.

Since scan() consolidates all API requests into a single observable, you can assign these values to your state array instead of appending them. The observable will emit right after the initial API request. With every subsequent API request completion, a new array containing the added items will be emitted. This ensures that your component can display data without having to wait for all API requests to finish.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is the best approach for determining the most effective method for invoking a handler function in React?

As a newcomer to React, I am navigating through the various ways to define callback functions. // Approach 1 private handleInputChange = (event) => { this.setState({name: event.target.value}); } // Approach 2 private handleInputChange(event){ t ...

Guide to configuring an Appium-Webdriver.io project for compiling TypeScript files: [ ISSUE @wdio/cli:launcher: No test files detected, program exiting with error ]

I have decided to transition my Appium-Javascript boilerplate project into a typescript project. I am opting for the typed-configuration as it is officially recommended and have followed the steps outlined in the documentation. Here is an overview of the ...

Limitations require a member to only accept a type (and not an instance) that extends or implements another type [TypeScript]

I'm seeking assistance with a set of abstract concepts in TypeScript. I am looking to restrict a member to only accept types as values, but those types must also implement or extend other types or interfaces. For example: The code snippet below is ...

Having trouble importing AnimeJS into an Ionic-Angular project

I successfully added AnimeJS to my Ionic 4 project using the commands below: npm i animejs --save npm i @types/animejs --save To reference AnimeJS, I used the following import statement: import * as anime from 'animejs' However, whenever I tr ...

How can I transfer the document id from Angular Firestore to a different component?

I'm seeking assistance on how to achieve a specific task related to pulling data from Firestore in my Angular application and displaying it in a list. Everything is working smoothly, including retrieving the document ID. My goal is to have the retrie ...

Multiple values are found in the 'Access-Control-Allow-Origin' header in Angular 7

After serving my Angular app on port 4202, I attempted to connect to a remote Spring MVC app using the code snippet below. this.http.post<Hero[]>('http://localhost:8080/api/hero/list', {"id":1}, httpOptions) However, I encountered the fol ...

Despite declaring a default export, the code does not include one

Software decays over time. After making a small modification to a GitHub project that was three years old, the rebuild failed due to automatic security patches. I managed to fix everything except for an issue with a default import. The specific error mess ...

`Angular 9 template directives`

I am facing an issue with my Angular template that includes a ng-template. I have attempted to insert an embedded view using ngTemplateOutlet, but I keep encountering the following error: core.js:4061 ERROR Error: ExpressionChangedAfterItHasBeenCheckedEr ...

Steps for fixing the error message "Type 'Observable<string>' is not assignable to type 'Observable<{ userId: string; partyId: string; }>'"

Having some trouble with my Angular project. I have this code snippet in my getUser() method, but the compiler isn't happy about it and I can't seem to figure out what's wrong: public getUser(): Observable<{ userId: string; partyId: strin ...

Avoiding the restriction of narrowing generic types when employing literals with currying in TypeScript

Trying to design types for Sanctuary (js library focused on functional programming) has posed a challenge. The goal is to define an Ord type that represents any value with a natural order. In essence, an Ord can be: A built-in primitive type: number, str ...

Converting an array of objects to an array based on an interface

I'm currently facing an issue with assigning an array of objects to an interface-based array. Here is the current implementation in my item.ts interface: export interface IItem { id: number, text: string, members: any } In the item.component.ts ...

Encountering issues while attempting to upload items to AWS S3 bucket through NodeJS, receiving an Access Denied error 403

I encountered an issue while attempting to upload objects into AWS S3 using a NodeJS program. 2020-07-24T15:04:45.744Z 91aaad14-c00a-12c4-89f6-4c59fee047a1 INFO uploading to S3 2020-07-24T15:04:47.383Z 91aaad14-c00a-12c4-89f6-4c59fee047a1 IN ...

Each page in NextJS has a nearly identical JavaScript bundle size

After using NextJS for a considerable amount of time, I finally decided to take a closer look at the build folder and the console output when the build process is successful. To my surprise, I noticed something peculiar during one of these inspections. In ...

Encountering an issue while constructing an Angular library project. The 'ng serve' command runs smoothly on local environment, but an error message stating

I recently developed an npm package called Cloudee. While it functions perfectly locally, I encounter an issue when attempting to deploy it. The error message states: 'Unexpected value 'CloudyModule in /home/hadi/dev/rickithadi/node_modules/cloud ...

"Encountering an unexpected error when Dockerizing an Angular App on Win10 Professional: EXPAND-ARCHIVE instruction not

I need to create a docker image for an Angular web application and deploy it on a Windows machine. However, I'm running into an issue when executing the command: docker build -t node . The following exception is thrown: Error response from daemon: ...

Using Angular: filtering data streams from a date range observable object

I have a piece of code that seems to be functioning correctly, but I can't shake the feeling that it might just be working by chance due to an undocumented feature. I'm torn between questioning its validity or accepting that it is indeed designed ...

Angular alert: The configuration object used to initialize Webpack does not conform to the API schema

Recently encountered an error with angular 7 that started popping up today. Unsure of what's causing it. Attempted to update, remove, and reinstall all packages but still unable to resolve the issue. Error: Invalid configuration object. Webpack ini ...

Angular: how to manually trigger change detection without using Angular's dependency injection system

Is there a way to globally initiate angular change detection without having to inject something like ApplicationRef? I am looking to utilize the functionality as a standard JavaScript function rather than a service method, in order to avoid the need for ...

Deploying an Azure Blob Trigger in TypeScript does not initiate the trigger

After successfully testing my Azure function locally, I deployed it only to find that it fails to trigger when a file is uploaded to the video-temp container. { "bindings": [ { "name": "myBlob", "type&qu ...

Explain the functionality of the Angular EventListener that is triggered on the scroll event of

Currently, I am exploring ways to track the position of the navbar beneath the landing page. The goal is for the navbar to become sticky at the top once it reaches that position until you scroll back up. Despite trying various solutions on Stack Overflow a ...