Tips for finalizing a subscriber after a for loop finishes?

When you send a GET request to , you will receive the repositories owned by the user benawad. However, GitHub limits the number of repositories returned to 30.

The user benawad currently has 246 repositories as of today (14/08/2021).

In order to workaround this limitation, you can make a GET request with additional parameters. GitHub uses pagination in its API, so you need to specify the desired page and the number of repositories per page in the URL.

Your modified GET request should resemble the following:

However, GitHub restricts the maximum number of repositories per page to 100. Therefore, your modified GET request will only return 100 repos instead of the total 246 repositories owned by the user benawad.

I have implemented the code below in my Angular service to fetch all repositories from multiple pages:

public getUserRepos(user: string): Observable<RepositoryI[]> {
    return new Observable((subscriber: Subscriber<RepositoryI[]>) => {
        this.getUserData(user).subscribe((data: UserI) => {
            const pages: number = Math.ceil(data.public_repos / 100);

            for (let i = 1; i <= pages; i++) {
                this.http
                    .get(`https://api.github.com/users/${user}/repos?page=${i}&per_page=100`)
                    .subscribe((data: RepositoryI[]) => {
                        subscriber.next(data);
                    });
            }
        });
    });
}

To handle when the Observable has completed emitting values, I subscribe to it in my component with the following code snippet:

this.userService.getUserRepos(id).subscribe((repos)=>{
    this.repositories.push(...repos);
})

An issue arises where I lack control over determining when the Observable has finished emitting values. In my component, I want to execute a function when the Observable is complete.

I attempted the approach below:

public getUserRepos(user: string): Observable<RepositoryI[]> {
    return new Observable((subscriber: Subscriber<RepositoryI[]>) => {
        this.getUserData(user).subscribe((data: UserI) => {
            const pages: number = Math.ceil(data.public_repos / 100);

            for (let i = 1; i <= pages; i++) {
                this.http
                    .get(`https://api.github.com/users/${user}/repos?page=${i}&per_page=100`)
                    .subscribe((data: RepositoryI[]) => {
                        subscriber.next(data);
                        if(pages == i) subscriber.complete();
                    });
            }
        });
    });
}

In my component, I do the following:

this.userService.getUserRepos(id).subscribe(
    (repos) => {
        this.repositories.push(...repos);
    },
    (err) => {
        console.log(err);
    },
    () => {
        console.log(this.repositories); 
        // The output logs only 46 repositories instead of the expected 246
        // I wish to trigger a function here
    }
);

The console.log() statement displays 46 repositories instead of 246. This discrepancy may occur due to prematurely completing the subscriber before fetching all three pages. Even though I call .complete() within the subscription, there seems to be an error. Can someone guide me on what I might be doing incorrectly? Thank you in advance.

Answer №1

To accomplish this task with just one subscription, utilize RxJS operators and functions as shown below:

public fetchUserRepositories(user: string): Observable<RepositoryI[]> {
  // Create an observable that combines all repositories into one:
  return this.getUserData(user).pipe(
    switchMap((data: UserI) => {
      const pages: number = Math.ceil(data.public_repos / 100);

      // Combine results from multiple observables using forkJoin:
      return forkJoin(
        Array.from(new Array(pages)).map((_, page) =>
          this.http.get<RepositoryI[]>(
            `https://api.github.com/users/${user}/repos?page=${page + 1}&per_page=100`
          )
        )
      ).pipe(
        // Flatten the result array to a single RepositoryI[]:
        map((res) =>
          res.reduce((acc, value) => acc.concat(value), [])
        )
      );
    })
  );
}

In your component, subscribe to the observable to retrieve all repositories upon completion:

this.userService.fetchUserRepositories(id).subscribe((repos) => {
    this.repositories = repos;
    // Execute necessary actions here...
});

Answer №2

          const subscription = interval(1000).subscribe(() => {
          this.http
            .get(
              `https://api.github.com/users/${user}/repos?page=${i}&per_page=100`
            )
            .subscribe((data: RepositoryI[]) => {
              subscriber.next(data);
              if(!data) { // check for null data
                subscription.unsubscribe();  // cancel the interval subscription
                return;  // stop the subscription
              }
            });
          })
        }

This approach ensures that you handle the subscription properly by unsubscribing when there is no data.

Please confirm if this solution meets your requirements.

Answer №3

One interesting feature of Javascript is its ability to run things asynchronously.

For example, if you use a 'For loop' to make API calls in parallel, the count will increment simultaneously.

When making multiple parallel API calls, each call may return at different times.

However, there is a possibility that before all API calls complete, the value of 'i' reaches 3.

In such cases, the third API call with fewer records might reach the subscribe logic first, resulting in only 46 records being displayed.

subscriber.next(data); // 3rd api comes here 
              
if(pages == i) subscriber.complete();  // now your pages and i values are same and it gets completed.
            });

SOLUTION


import {defer } from 'rxjs';

defer(async () => {
     for (let i = 1; i <= pages; i++) {
          await this.http
            .get(
              `https://api.github.com/users/${user}/repos?page=${i}&per_page=100`
            )
            .toPromise().then((data: RepositoryI[]) => {
              subscriber.next(data);
            });
        }
}.subscribe(x => { subscriber.complete(); });

Answer №4

Outline

In this explanation, I will outline two different methods to accomplish the task at hand. The first approach involves an imperative way (utilizing a loop), while the second method is reactive in nature (using rxjs).

Imperative Approach

To achieve this imperatively, you can implement a simple loop to retrieve all the pages from the API endpoint. This function returns a promise, but it can be converted into an observable if required by using from:

public getUserRepos(user: string): Promise<Repository[]> {
  let pageNumber = 0;
  let results = [];

  const getPage = pageNumber =>
    this.http
      .get<Repository[]>(`https://api.github.com/users/${user}/repos?page=${pageNumber}&per_page=100`)
      .toPromise();

  do {
    const pageResults = await getPage(pageNumber++);
    results = results.concat(pageResults);
  } while(pageResults.length !== 0);

  return results;
}

Reactive Approach

For a more reactive solution, you can utilize the expand operator provided by rxjs to fetch each page sequentially:

When implementing this method, ensure that the http response is mapped to an object containing the corresponding page number. This enables the expand operator to determine the subsequent page to request.

Here's an example in your context:

public getUserRepos(user: string): Observable<Repository[]> {
  const getPage = pageNumber =>
    this.http
      .get<Repository[]>(`https://api.github.com/users/${user}/repos?page=${pageNumber}&per_page=100`)
      .pipe(map(z => ({ results: z, page: pageNumber }))); // Include page number in results

  return getPage(0).pipe(
    expand(pr => getPage(pr.page + 1)),
    takeWhile(pr => pr.results.length !== 0),
    map(pr => pr.results),
    scan((acc, v) => [...acc, ...v])
  );
}

This implementation initiates a query for page 0, followed by consecutive requests for subsequent pages until a page returns no entries, resulting in the completion of the observable stream.

The use of scan allows for displaying partial results as pages are fetched, providing real-time feedback rather than waiting for all pages to load before showing any data.

Refer to this StackBlitz for a demonstration.

Note: There may be an additional query for a final page which will be cancelled. If you prefer to avoid this, adjust the expand call like so:

expand(pr => (pr.results.length !== 0 ? getPage(pr.page + 1) : EMPTY))

Answer №5

Here is a more concise approach to providing an answer. If the nested structure bothers you, consider defining an inner observable as a separate function.

public getUserRepos = (user:string) =>
  this.getUserData(user)
  .pipe(
    map(({public_repos})=>Math.ceil(public_repos / 100)),
    switchMap(max=>interval(1)
    .pipe(
      takeWhile(i=>++i<=max)
      mergeMap(i=>this.http.get<RepositoryI[]>(`https://api.github.com/users/${user}/repos?page=${++i}&per_page=100`))
    )),
    scan((acc, repo)=>[...acc, ...repo], [] as RepositoryI[])
  );

This was my strategy:

  1. Retrieve the page limit using object destructuring from getUserData().
  2. Utilize interval() to initiate a counting observable (starting with 0 and utilizing
    ++</code later on).</li>
    <li>Employ <code>takeWhile()
    to verify if the count surpasses the page limit; otherwise, the observable completes at this point.
  3. Use mergeMap() to line up a sequence of API calls for each number emitted by interval().
  4. Aggregate all responses into a singular array response with scan().

Update: When testing this method, your component logic will require an update.

Since scan() consolidates all API requests into a single observable, you can assign these values to your state array instead of appending them. The observable will emit right after the initial API request. With every subsequent API request completion, a new array containing the added items will be emitted. This ensures that your component can display data without having to wait for all API requests to finish.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is the best way to combine a Signal containing an array of Signals in Angular using the merge(/mergeAll) operator?

When working in the world of rxjs, you have the ability to combine multiple Observables using the merge operator. If you have an array of Observables, all you need to do is spread that array into the merge operator like this: merge(...arrayOfObservables). ...

Can a substring within a string be customized by changing its color or converting it into a different HTML tag when it is defined as a string property?

Let's discuss a scenario where we have a React component that takes a string as a prop: interface MyProps { myInput: string; } export function MyComponent({ myInput }: MyProps) { ... return ( <div> {myInput} </div> ...

Enhancing a component with injected props by including type definitions in a higher-order component

Implementing a "higher order component" without types can be achieved as shown below: const Themeable = (mapThemeToProps) => { return (WrappedComponent) => { const themedComponent = (props, { theme: appTheme }) => { return <Wrapped ...

A helpful guide on showcasing error messages at the top of a form in Angular using reactive forms

I have a Validation summary component that is designed to fetch an ngForm, but it is currently unable to subscribe to status changes or value changes and display the summary of error messages. export class CustomValidationSummaryComponent implements OnIni ...

How to modify a specific property of an array object in JavaScript

I have an array of objects that looks like this: [ { number: 1, name: "A" }, { number: 2, name: "e", }, { number: 3, name: "EE", } ] I am looking for a way to insert an object into the array at a specific position and ...

The pipe seems to have malfunctioned

Here is how my pipe file appears: pipe.ts import { Pipe, PipeTransform } from '@angular/core'; @Pipe({ name: 'unique', pure: false }) export class UniquePipe implements PipeTransform { transform(value: any, args?: any): any { ...

Guide to integrating Inversify with Mocha

In my TypeScript Node.js application, I am implementing Dependency Injection using Inversify. The functionality works perfectly during the app's execution. However, I encountered an issue with the @injectable() annotation when running tests. An error ...

Mastering Inter-Composable Communication in Vue 3: A Guide

Composables in Vue documentation demonstrate how small composition functions can be used for organizing code by composing the app. Discover More About Extracting Composables for Code Organization "Extracted composables act as component-scoped servi ...

TypeScript is unaware that a component receives a necessary prop from a Higher Order Component (HOC)

My component export is wrapped with a higher-order component (HOC) that adds a required prop to it, but TypeScript seems unaware that this prop has already been added by the HOC. Here's the Component: import * as React from "react"; import { withTex ...

Personalizing the predefined title bar outline of the input text field

The outline color of the title in the input textbox appears differently in Google Chrome, and the bottom border line looks different as well. <input type="text" title="Please fill out this field."> https://i.stack.imgur.com/iJwPp.png To address th ...

Revise Swagger UI within toggle button switch

My project aims to showcase three distinct OpenApi definitions within a web application, enabling users to explore different API documentation. The concept involves implementing a toggle button group with three buttons at the top and the Swagger UI display ...

Encountering an issue when using npm to add a forked repository as a required package

While attempting to install my fork of a repository, I encountered the error message "Can't install github:<repo>: Missing package name." The original repository can be accessed here, but the specific section I am modifying in my fork is located ...

What is the best way to define a margin according to the size of the device being used

I am working on an angular/bootstrap web app and need to set a left margin of 40px on md, xl, lg devices and 0px on sm device. I attempted to create a spacer in styles.scss like this: $spacer: 1rem; .ml-6{ margin-left:($spacer*2.5); } Then in my HTML, ...

Angular 4 Operator for adding elements to the front of an array and returning the updated array

I am searching for a solution in TypeScript that adds an element to the beginning of an array and returns the updated array. I am working with Angular and Redux, trying to write a reducer function that requires this specific functionality. Using unshift ...

Before running any unit tests, I have to address all linting issues as required by ng test

Upon running ng test, the output I receive is as follows: > ng test 24 12 2019 14:20:07.854:WARN [karma]: No captured browser, open http://localhost:9876/ 24 12 2019 14:20:07.860:INFO [karma-server]: Karma v4.4.1 server started at http://0.0.0.0:9876/ ...

What could be causing the presence of a "strike" in my typescript code?

While transitioning my code from JavaScript to TypeScript for the first time, I noticed that some code has been struck out. Can someone explain why this is happening and what it signifies? How should I address this issue? Here's a screenshot as an exa ...

Storing Json data in a variable within Angular 2: a step-by-step guide

Within the params.value object, there are 3 arrays containing names that I need to extract and store in a variable. I attempted to use a ForEach loop for this purpose, but encountered an issue. Can you spot what's wrong? var roles = params.forEach(x ...

Deploying a nodejs application with angular as the user interface framework using Firebase

Is there a way to set up an express.js application with angular as the front-end framework, multiple route files, and communication between the server and angular via service level API calls, in order to deploy it on firebase using Firebase Hosting? Curre ...

Is it possible to modify the CSS styling in React using the following demonstration?

I am attempting to create an interactive feature where a ball moves to the location where the mouse is clicked. Although the X and Y coordinates are being logged successfully, the ball itself is not moving. Can anyone help me identify what I might be overl ...

Deleting items with a swipe gesture in Angular 10

Hey there, fellow developer! I could really use some assistance in implementing the swipe delete feature for our Angular project. Could you take a look at the screenshot provided below? The code snippet given to me for this feature is as follows: &l ...