I am faced with the challenge of extracting data from an API that is paginated, and unfortunately, I cannot determine the total number of pages in advance. However, I can identify when I have reached the last page. My goal is to develop a function that retrieves each page sequentially.
The structure of the page is as follows:
interface PaginatedResult<T> {
count: number;
next: string;
prev: string;
results: T[];
}
Once I reach the final page, the next attribute becomes null. My desired approach is demonstrated below:
let data: Data[] = []
let url: string = API_ENDPOINT
while (url !== null) {
this.http.get(url).subscribe((page: PaginatedResult<Data>) => {
data = data.concat(page.results)
url = page.next
})
}
However, executing this code leads to multiple simultaneous requests until one sets the url variable to null, causing the browser to freeze. I have explored methods for chaining subscriptions but have not discovered a solution to combine an unknown number of subscriptions. With no prior knowledge of the total number of pages, my only option is to retrieve one page at a time.
The main issue lies in needing to wait for each page's data before determining whether the next page should be requested.
Do you have any suggestions or ideas on how to tackle this problem effectively?
It is worth noting that while a similar question on Stack Overflow presents strategies for handling an indeterminate number of observables (link provided), it still differs from my current predicament where both the total number of requests and their sequential execution remain uncertain.