Error in pagination when using MAX() function in PostgreSQL query

Here is the query I am using to retrieve the latest message from each room:

SELECT MAX
    ( "Messages"."id" ) AS messageId,
    "Rooms"."id" 
FROM
    "RoomUsers"
    INNER JOIN "Rooms" ON "RoomUsers"."roomId" = "Rooms"."id"
    INNER JOIN "Conversations" ON "Conversations"."roomId" = "Rooms"."id"
    INNER JOIN "Messages" ON "Messages"."conversationId" = "Conversations"."id" 
WHERE
    "RoomUsers"."userId" = 51
GROUP BY
    "Rooms"."id" 

The expected output would look something like this:

+----------+-------+
|messageId |   id  |
+----------+-------+
|    1     |   4   |
|    17    |   5   |
|    7     |   6   |
|    29    |   7   |
|    18    |   8   |
|    19    |   9   |
+----------+-------+

As you can see, handling large datasets efficiently is crucial. To tackle this issue, I plan to implement pagination using OFFSET and LIMIT.

To start off, I set up a simple pagination with 3 records per page:

SELECT MAX
    ( "Messages"."id" ) AS messageId,
    "Rooms"."id" 
FROM
    "RoomUsers"
    INNER JOIN "Rooms" ON "RoomUsers"."roomId" = "Rooms"."id"
    INNER JOIN "Conversations" ON "Conversations"."roomId" = "Rooms"."id"
    INNER JOIN "Messages" ON "Messages"."conversationId" = "Conversations"."id" 
WHERE
    "RoomUsers"."userId" = 51
GROUP BY
    "Rooms"."id" 
    LIMIT 3 OFFSET 0

Upon executing this query, the result for the first page is as follows:

+----------+-------+
|messageId |   id  |
+----------+-------+
|    1     |   4   |
|    17    |   5   |
|    7     |   6   |
+----------+-------+

This seems to work fine for one page. However, when changing the offset to 1 for the second page, the result changes:

+----------+-------+
|messageId |   id  |
+----------+-------+
|    17    |   5   |
|    7     |   6   |
|    29    |   7   |
+----------+-------+

Increasing the offset further to 2 for the third page produces a different outcome:

+----------+-------+
|messageId |   id  |
+----------+-------+
|    7     |   6   |
|    29    |   7   |
|    18    |   8   |
+----------+-------+

There seems to be some confusion in the results as we paginate through them. I need to understand why this is happening and how I can rectify it for proper data pagination.

Answer №1

If each page contains 4 rows, the offsets should be set at 0, 4, and 8.

Furthermore, it's important to note that the ordering of results can be unpredictable without an explicit order by statement. Therefore, include the following:

ORDER BY "Rooms"."id"

Answer №2

PostgreSQL offers the use of non-embedded SQL cursors, specifically designed for this purpose. Take a look at this demo:

DECLARE messages_cursor CURSOR WITH HOLD FOR 
SELECT MAX
    ( "Messages"."id" ) AS messageId,
    "Rooms"."id" 
FROM
    "RoomUsers"
    INNER JOIN "Rooms" ON "RoomUsers"."roomId" = "Rooms"."id"
    INNER JOIN "Conversations" ON "Conversations"."roomId" = "Rooms"."id"
    INNER JOIN "Messages" ON "Messages"."conversationId" = "Conversations"."id" 
WHERE
    "RoomUsers"."userId" = 51
GROUP BY
    "Rooms"."id";

Execute it once, and then start fetching results page by page using FETCH:

FETCH FORWARD 3 FROM messages_cursor;

Once you are done with the cursor, don't forget to close it:

CLOSE messages_cursor;--/*if you want to clean them all up, then*/CLOSE ALL;
  1. Using a cursor prevents the need to recalculate everything, sort repeatedly, and discard unnecessary data like when using offset-based queries.
  2. If sorting is not required, avoiding the use of order by can improve performance.
  3. The cursor maintains stability regardless of any changes made to the table while reading pages, unlike offset/limit pagination which can be disrupted by database modifications.
  4. WITH HOLD feature helps in persisting the cursor even with autocommit enabled or when committing between fetches within the same session. This ensures consistent pagination behavior, especially in repeatable read transactions.

While cursors offer advantages over traditional pagination methods, there are alternative techniques as well - explore other options.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Sending Disabled Form Field Input Value in Angular 2 POST Request

I'm working on a form with an email field that I want to populate using interpolation. However, I also want to prevent users from changing the email address once it's displayed. To achieve this, I tried adding the disabled attribute to the input ...

Looking to incorporate ipcRenderer from Electron into your Angular project? Having trouble accessing variables passed from the preload script?

I am struggling with incorporating ipcRenderer into the 'frontend' code of my electron app. Although I found examples in the documentation that use require, this method is not accessible on the frontend side where I am utilizing Angular. In the ...

ERROR UnhandledTypeError: Unable to access attributes of null (attempting to retrieve 'pipe')

When I include "{ observe: 'response' }" in my request, why do I encounter an error (ERROR TypeError: Cannot read properties of undefined (reading 'pipe'))? This is to retrieve all headers. let answer = this.http.post<ResponseLog ...

Are React component properties enclosed in curly braces?

I have a new component configured like this: type customType = differentType<uniqueType1, uniqueType2, uniqueType3>; function customComponent({q}: customType) When called, it looks like this: <customComponent {...myCustomVar} />, where myCus ...

Error: The property '...' is not found in the ReactElement<any, any> type, but it is required in the type '{...}'

As a beginner in TypeScript, I am currently working on rendering a page by fetching data from getStaticProps. The code snippet I am using for this purpose is: import React, {FormEvent, useState} from "react"; import { InferGetStaticPropsType } fr ...

Allowing the use of a string as a parameter in a Typescript constructor

Currently, I am utilizing TypeScript to create a constructor for a model within Angular. One of the attributes in the model is configured as an enum with specific string values. Everything functions well if an enum value is passed to the constructor. The i ...

The proper method for referencing TypeScript compiled files with the outDir option

I am currently working on a simple app that consists of two .ts files. These files are compiled using the following tsconfig.js file: { "compilerOptions": { "target": "ES5", "module": "commonjs", "sourceMap": true, "emitDecoratorMetadata ...

What is the correct way to trigger an event specified as a string parameter in the emit() function?

My current goal is to pass the emit name as a string (for example, 'showComponent') from child to parent. I then want to trigger another emit in the emitAction(callbackName: string) function, and finally execute the showComponent() function. I&a ...

What is the best way to obtain clear HTTP request data in a component?

My service retrieves JSON data from the backend: constructor(private http: Http) { }; getUsers(): Observable<any> { return this.http.get('http://127.0.0.1:8000/app_todo2/users_list'); }; In the component, this data is processed: ng ...

Seeking assistance with the creation of a PostgreSQL view

I am putting together a new display right now. Here is an illustration. Table1 : Table1.</p> Whether I use WHERE or LEFT JOIN, the result does not match the number of rows in Table1. Please point out where I may be going wrong. ...

Managing the rxjs from an Array of observables when the array has no elements

Looking for a more elegant solution to handle the case where an array of observables is empty in the following TypeScript function. I want the observable to complete when subscribe() is called without the need for an initial check. I've already imple ...

Why isn't useEffect recognizing the variable change?

Within my project, I am working with three key files: Date Component Preview Page (used to display the date component) useDateController (hook responsible for managing all things date related) In each of these files, I have included the following code sn ...

Is it possible to target a specific element using Angular2's HostListener feature? Can we target elements based on their class name?"

Is there a way in Angular2 to target a specific element within the HostListener decorator? @HostListener('dragstart', ['$event']) onDragStart(ev:Event) { console.log(ev); } @HostListener('document: dragstart' ...

Error: The AWS amplify codegen is unable to locate any exported members within the Namespace API

Using AWS resources in my web app, such as a Cognito user pool and an AppSync GraphQL API, requires careful maintenance in a separate project. When modifications are needed, I rely on the amplify command to delete and re-import these resources: $ amplify r ...

What is the best way to execute a function on the output of *ngFor directive in Angular 2?

Imagine having a list of all the users within your system: allUsers = { a: {name:'Adam',email:'<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="39585d5854794d5c4a4d5a56175a56... f: {name:'fred' ...

Having trouble linking tables to Node.js with TypeScriptyntax?

I am facing an issue with mapping multiple entities using sequelize. I keep encountering the error message " Error: Profesor.hasOne called with something that's not a subclass of Sequelize.Model". How can I resolve this issue? Below is the code for t ...

Developing an Angular 2 Cordova plugin

Currently, I am in the process of developing a Cordova plugin for Ionic 2. The plugin is supposed to retrieve data from an Android device and display it either on the console or as an alert. However, I am facing difficulty in displaying this data on the HT ...

Utilizing const as the iteration variable in a for loop

I've grasped the concept of using var and let in a for loop in typescript/javascript, but can someone shed light on how and why a const variable as a loop variable behaves? for (const i = 0; i < 5; i++) { setTimeout(function() { console.log( ...

The interface 'IProduct' does not include several properties found in type 'IProduct[]', such as length, pop, push, concat, and many more

My goal is to transfer data between parent and child components using React and TypeScript. I have defined the following interfaces: export interface IProduct { id: string; name: string; price: string; image: string; ...

What allows the type expression to be considered valid with a reduced amount of arguments?

Currently diving into Typescript and focusing on functions in this unit. Check out the following code snippet: type FunctionTypeForArrMap = (value: number, index: number, arr: number[]) => number function map (arr: number[], cb: FunctionTypeForArr ...