Recursive Fetching with RxJS expand operator
Thursday, December 10, 2020
Recursive Fetching with RxJS expand operator
A couple of weeks back, I stumbled upon a thread on Twitter where they implement a use-case in different frameworks/libraries (and Angular was missing). The use-case involves a recursive fetching scenario. You can check out the React and Svelte implementations here:
I wanted to implement the same thing in Angular then it dawns on me that recursive fetching with RxJS is, well, not documented. My naive approach was retryWhen. When a condition is passed for the recursion to happen, I deliberately throw something; so retryWhen can be invoked so the my pipeline can be re-executed.
Then I learned about expand() operator and everything just makes sense. Check code snippet below.
expand(): Recursively projects each source value to an Observable which is merged in the output Observable.
Here's the Stackblitz to the Angular implementation: https://stackblitz.com/edit/angular-ivy-rxjs-expand-recursion?file=src/app/app.component.ts
Code Examples
get-expand.ts
getHN(postId: string) {
const fetch$ = this.http.get(`https://hacker-news.firebaseio.com/v0/item/${postId}.json`)
.pipe(
delay(1000), // to stimulate loading
);
// expand() approach
return fetch$.pipe(
expand((post: any) => { // the original "post" from "postId" and every "parentPost" afterwards. This is recursive
return post.parent // if there's "parent", we refetch, if not, we return EMPTY to complete the "expand()"
? this.http.get(`https://hacker-news.firebaseio.com/v0/item/${post.parent}.json`)
: EMPTY
}),
reduce((finalPost, latestPost: any) => { // rxjs "reduce()" acts exactly like Array.reduce. Here, "latestPost" is each "post" from "expand()" including the original "post". reduce() will keep reducing until "expand()" completes with EMPTY
return latestPost.title
? { ...finalPost, title: latestPost.title, date: new Date(d.time * 1000).toLocaleDateString("en-US") }
: finalPost;
})
);
}get-retry-when.ts
getFN(postId: string) {
const fetch$ = this.http.get(`https://hacker-news.firebaseio.com/v0/item/${postId}.json`)
.pipe(
delay(1000), // to stimulate loading
);
// naive approach
return fetch$.pipe(
concatMap((post: any) => {
// concatMap to fetch parent post or return the post
let temp = post; // create "temp" to hold new post each time
return temp.parent // if current "post" has parent (we need to fetch this parent recursively)
? defer(
// use defer() so "temp.parent" has the newest parent (recursive)
() => this.http.get(`https://hacker-news.firebaseio.com/v0/item/${temp.parent}.json`)
).pipe( // we "pipe" on the "defer()" so "retryWhen" will retry the whole "defer()"
map((parent: any) => {
if (parent.parent) { // again, check if there's "parent" still
temp = parent; // reassign "temp" so when "defer()" is retried, it has new value of "temp"
throw temp; // you can throw ANYTHING here
}
return { ...data, title: `re: ${parent.title}` }; // no parent, return with the title
}),
retryWhen(_ => _) // this is ugly. In the "map", we throw so retryWhen is invoked but we don't need to do anything, just invoke so the whole "defer()" gets re-run
)
: of(post)
}),
map((finalPost: any) => ({...finalPost, date: new Date(d.time * 1000).toLocaleDateString("en-US")}))
);
}
Have a question or comment?