I am currently working with an API written in typescript and attempting to execute parallel queries for the same document by using Promise.allSettled. However, I have noticed that it is performing poorly and seems to be running sequentially instead of in parallel. Is there a way to improve performance and actually run parallel queries on the same document within the same connection for MongoDB? Below is the code snippet:
console.time("normal");
let normal = await ContentRepo.geBySkillIdWithSourceFiltered(
[chosenSkillsArr[0].sid!],
readContentIds,
body.isVideoIncluded,
true,
true
);
console.timeEnd("normal");
console.time("parallel");
const parallel = await Promise.allSettled(
chosenSkillsArr.map(async (skill: IScrapeSkillDocument) => {
const result = await ContentRepo.geBySkillIdWithSourceFiltered(
[skill.sid!],
readContentIds,
body.isVideoIncluded,
true,
true
);
})
);
console.timeEnd("parallel");
You can find the relevant function being called below:
async geBySkillIdWithSourceFiltered(
skillIds: string[],
contentIds: string[],
isVideoIncluded?: boolean,
isCuratorIdFilter?: boolean,
activeSourceFilter?: boolean
): Promise<IContentWithSource[]> {
try {
console.time(`single-${skillIds}`);
var contents = await ContentM.find({
$and: [
{ "skills.skillId": { $in: skillIds } },
{ recordStatus: true },
isCuratorIdFilter ? { curatorId: 0 } : {},
isVideoIncluded ? {} : { type: contentTypeNumber.read },
{ _id: { $nin: contentIds } },
],
}).exec();
var items: IContentWithSource[] = [];
var sourceIds = new Set<string>();
contents.forEach((content) => {
if (!this.isEmpty(content.sourceId)) {
sourceIds.add(content.sourceId!);
}
});
var sources: any = {};
var sourcesArr = await new SourceRepo().getByIds(
Array.from(sourceIds)
);
sourcesArr.forEach((source) => {
sources[source._id] = source;
});
if (activeSourceFilter) {
contents
.map((i) => i.toJSON() as IContentWithSource)
.map((k) => {
if (sources[k.sourceId!].isActive) {
k.source = sources[k.sourceId!];
items.push(k);
}
});
} else {
contents
.map((i) => i.toJSON() as IContentWithSource)
.map((k) => {
k.source = sources[k.sourceId!];
items.push(k);
});
}
console.timeEnd(`single-${skillIds}`);
return items;
} catch (err) {
throw err;
}
}
Here are the results from running the code:
single-KS120B874P2P6BK1MQ0T: 1872.735ms
normal: 1873.934ms
single-KS120B874P2P6BK1MQ0T: 3369.925ms
single-KS440QS66YCBN23Y8K25: 3721.214ms
single-KS1226Y6DNDT05G7FJ4J: 3799.050ms
parallel: 3800.586ms