Eliminate repeat entries in MongoDB database

Within our MongoDB collection, we have identified duplicate revisions pertaining to the same transaction.

Our goal is to clean up this collection by retaining only the most recent revision for each transaction. I have devised a script that successfully removes duplicates; however, in certain instances, it fails to preserve the latest revision. For reference, you can access the script here: https://gist.github.com/praveenkrjha/1f02ed675e297a2c85e9020e39320a41

The initial record (with transaction id (tid) and revision number (rv)):

_id: 61b994a48d546d882462da25, tid: b1def2a8-01e0-4f8c-a859-7cd63287617c, rv: 42 
_id: 61b994a48d546d882462da27, tid: b1def2a8-01e0-4f8c-a859-7cd63287617c, rv: 42 
_id: 61b994a48d546d882462da29, tid: b1def2a8-01e0-4f8c-a859-7cd63287617c, rv: 42 
_id: 61b994a48d546d882462da2b, tid: d9b9e197-6f91-480c-8757-2a24330bc4c5, rv: 21 
_id: 61b994a48d546d882462da2d, tid: d9b9e197-6f91-480c-8757-2a24330bc4c5, rv: 22 
_id: 61b994a48d546d882462da2f, tid: df0f1252-51d3-4471-be68-a81e8b9a11e2, rv: 31 
_id: 61b994a48d546d882462da31, tid: df0f1252-51d3-4471-be68-a81e8b9a11e2, rv: 32 
_id: 61b994a48d546d882462da33, tid: bc92c1ee-91b4-4eea-b9ad-752f8dba6c54, rv: 11 
_id: 61b994a48d546d882462da35, tid: bc92c1ee-91b4-4eea-b9ad-752f8dba6c54, rv: 12 
_id: 61b994a48d546d882462da37, tid: 2a131e9a-07f1-413f-a21f-4f1bbbb3d460, rv: 1 
_id: 61b994a48d546d882462da39, tid: e2c7fb60-a1e2-46ea-8d6a-e94e14ded2d3, rv: 1 
_id: 61b994a48d546d882462da3b, tid: 314f7cb2-1aad-4add-b2e1-17e910d97713, rv: 1

The resulting data after cleaning up:

_id: 61b994a48d546d882462da25, tid: b1def2a8-01e0-4f8c-a859-7cd63287617c, rv: 42 
_id: 61b994a48d546d882462da2b, tid: d9b9e197-6f91-480c-8757-2a24330bc4c5, rv: 21 
_id: 61b994a48d546d882462da2f, tid: df0f1252-51d3-4471-be68-a81e8b9a11e2, rv: 31 
_id: 61b994a48d546d882462da33, tid: bc92c1ee-91b4-4eea-b9ad-752f8dba6c54, rv: 11 
_id: 61b994a48d546d882462da37, tid: 2a131e9a-07f1-413f-a21f-4f1bbbb3d460, rv: 1 
_id: 61b994a48d546d882462da39, tid: e2c7fb60-a1e2-46ea-8d6a-e94e14ded2d3, rv: 1 
_id: 61b994a48d546d882462da3b, tid: 314f7cb2-1aad-4add-b2e1-17e910d97713, rv: 1

Upon reviewing the final results, it appears that the script mistakenly deletes the latest revision for some transactions. Any suggestions on how this issue can be addressed?

javascripttypescriptmongodb

Answer №1

Check out this query:

db.collection.aggregate([
   { $sort: { ratingValue: -1 } },
   { $group: { _id: "$taskId", data: { $first: "$$ROOT" } } },
   { $replaceWith: "$data" }
])

or

db.collection.aggregate([
   { $sort: { ratingValue: 1 } },
   { $group: { _id: "$taskId", data: { $last: "$$ROOT" } } },
   { $replaceWith: "$data" }
])

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

There seems to be a problem with the output when trying to display the message "You said ${reply}"

In the following code snippet, vanilla.js is being used with ATOM as the text editor and running on nodejs via terminal: 'use strict'; const Readline = require('readline'); const rl = Readline.createInterface({ input:process.stdin, ...

javascriptnode.jsterminal

How to set up MongoDB on Debian Buster

Is there a way to upgrade MongoDB version from 3.2 to either MongoDB 3.4 or MongoDB 3.6 on Debian Buster? While Ubuntu supports the latest versions, I am currently stuck with MongoDB 3.2 on my server. ...

mongodbdebianstretchsid

Convert the data into a format that is compatible with JavaScript

Having trouble extracting a value from json and placing it into my controller. I want to assign the membership value of 8 to $scope.value = data.membership; Service call in JS: .service('getMembership', function ($http, SERVER_URL) { r ...

javascriptphpjsonencode

The Vue CLI build is displaying a blank page when hosted on an Apache server

I encountered an issue with vue cli. Running npm run serve went smoothly, but when I tried dev mode using npm run build-dev, the build process finished with a blank page displayed. The error message received was "Uncaught SyntaxError: Unexpected token &ap ...

javascriptnode.jsvue.jsnpm

Tips for extracting data from a PHP loop and displaying it in a div element

When I click on a PHP while loop, I want to change the value of a div. Here is my PHP code: <?php $query = mysql_query("select * from tbl_sub_product where product_id='$id'"); while($row=mysql_fetch_array($query)) { ?> <div>< ...

javascriptphpjqueryajax

Guide to including a promise.map (bluebird) within a nested promise chain

I'm currently using a chain to control flow and struggling to make promise.map within step2() wait for all generated promises to be resolved. Here's a visual representation of the flow I aim for: step1() step2() step2() ste ...

javascriptnode.jsasynchronouspromisebluebird

Retrieve the observable value and store it in a variable within my Angular 13 component

Incorporating Angular 13, my service contains the following observable: private _user = new BehaviorSubject<ApplicationUser | null>(null); user$ = this._user.asObservable(); The ApplicationUser model is defined as: export interface ...

javascriptangulartypescript

Filtering JSON array is just as simple as working with XML

During my project, I have transitioned from using XML format to JSON format. The XML structure includes nodes such as: <Creature> <Name>Someone</Name> <Desert>false</Desert> <Woods>false</Woods> < ...

javascriptarraysjsonxml

What is the best way to access an excel file using JavaScript and Protractor?

Is it possible to read an Excel file dynamically in JavaScript and iterate through cell values using row and column counts, without converting it into a JSON object? I have searched through several posts on Stack Overflow but have not found a solution yet. ...

javascriptangularjsjasmineprotractor

"Obtaining a MEAN response after performing an HTTP GET

I'm currently in the process of setting up a MEAN app, however I've hit a roadblock when it comes to extracting data from a webservice into my application. While returning a basic variable poses no issue, I am unsure how to retrieve the result fr ...

javascriptangularjsexpressmean

Error: An unusual symbol "/" was encountered during the compilation of EJS in Node.js with MongoDB

When attempting to use <% include ./partials/messages %>, <%= include ./partials/messages %>, or even <%= include ./partials/messages { %> within my file <!-- REGISTER.EJS --> <div class="row mt-5"> <div class="col-md ...

node.jsmongodbformattingejsweb-development-server

Is the vertex count of a Geometry in Three.js increased when it is converted to a BufferGeometry?

Recently, I've been experimenting with the fromGeometry method to convert regular Geometry objects into BufferGeometry objects. To my surprise, I noticed that the number of vertices increases during this conversion process. For instance, consider the ...

javascriptthree.js

Issues with clearRect() function in HTML5 canvas

Recently, I developed a function with the goal of redrawing a square line block that covers the entire page whenever the window size is adjusted. For reference, you can view my code at http://jsfiddle.net/9hVnZ/ However, I encountered an issue: bgCtx.cl ...

javascripthtmlcanvas

The Express server is set up with CORS enabled, however, it continues to encounter issues when attempting to make asynchronous HTTP requests from

We're currently experiencing an unusual issue and are seeking assistance to resolve it. Initially, our Express server is functioning well as an API when accessed from the same domain. However, we also need to utilize this API on our computers for tes ...

javascriptnode.jsangularexpresscross-domain

The ace.edit function is unable to locate the #javascript-editor div within the mat-tab

Having trouble integrating an ace editor with Angular material Error: ace.edit cannot locate the div #javascript-editor You can view my code on StackBlitz (check console for errors) app.component.html <mat-tab-group> <mat-tab label="Edito ...

javascriptangularangular-materialangular6ace-editor

I encountered a TypeScript error in React Native when attempting to use "className" with TypeScript

Although I've been using React for a while, React Native is new to me. I recently started using tailwind with import { View, Text } from "react-native"; import React from "react"; export default function Navigation() { return ...

reactjstypescriptreact-nativetailwind-css

Is it possible to implement sticky sessions with Node.js and pm2?

Can sticky sessions be implemented using the pm2 node module? Although not supported by Node.js's internal cluster module on purpose, it could still prove beneficial in scenarios like paused media streams. ...

javascriptnode.jssessionstickypm2

Emulate the CSS hover effect with jQuery 코드 희미한

Is there a method in jquery or any other technology that can detect event X and trigger a different event elsewhere? Currently, I am working with an image that has an image map. When a user hovers over a specific area on the map, I would like another part ...

javascriptjquerycsshover

Tips for including a dash or hyphen in an input field after two digits in Angular 4

Struggling to format the date of birth input with dashes manually when entered by the user. The desired output should resemble "08-18-2019," but I'm having difficulty achieving this. public dateOfBirth: { year: number; month: number; day: number }; ...

angulartypescriptmaskdirective

The press of the Enter key does not trigger the button

I'm facing an issue with a form that has just one field for user input and a button that triggers some jQuery to hide the login form when clicked. However, pressing enter after entering text causes the page to refresh... I'm starting to think th ...

javascriptjqueryhtml