TypeORM reporting duplication error when bulk saving data instead of detecting and ignoring existing records or updating their values

According to the documentation provided by TypeOrm Framework, the Repository.save function is supposed to save/insert new values and ignore/update existing ones.

However, I am currently experiencing an issue where it is throwing a duplication error for an existing value and halting the entire insertion process. (I have a unique column named key)

Definition of My entity:

import { Entity, Column, PrimaryGeneratedColumn, ManyToOne, JoinColumn, PrimaryColumn } from 'typeorm';
import { Source } from '../sources/source.entity';


@Entity({ name: 'articles' })
export class Article {

    @PrimaryGeneratedColumn()
    id: number;

    @Column()
    title: string;

    @Column({
        nullable: true
    })
    image: string | null;

    @Column({
        type: "text",
    })
    url: string;

    @Column({
        type: "varchar",
        unique: true
    })
    key: string;

    @Column({
        type: 'datetime',
        nullable: true
    })
    date: Date | null;

    @ManyToOne(type => Source, source => source.articles, {eager: true})
    @JoinColumn({name: 'source'})
    source: Source;

    @Column({
        type: `text`,
        nullable: true
    })
    description: string | null
}

Definition of My Service:

constructor(
    @InjectRepository(Article) private readonly articleRepository: Repository<Article>,
    private readonly articlesScraper: BlogScraperService
) {

}

async clonningFromScraper() {
    let articles = await this.articlesScraper.articles('1');

    articles = articles.map(article => ({ ...article, key: decodeURIComponent(article.url).substring(0, 255) }));

    return this.articleRepository
        .save(articles);
}

Answer №1

After much deliberation, I successfully resolved this issue by executing a RAW SQL query as shown below:

return this.articleRepository.query(
    "INSERT IGNORE INTO articles ( title, date, url, image, source, description, _key ) VALUES ?", [querableArticles]);

Answer №2

When updating a record with a unique constraint (such as a key), it is important to first locate the existing record in the database. Once you have found the record, you can then associate the ID with the new data that you wish to update using the save method.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Content Mismatch: The webpage was accessed through a secure HTTPS connection, but a request was made for an unsecured resource. Access to this content has been denied as

Error: Mixed Content. The webpage at '' was accessed using HTTPS, but it tried to load an insecure resource ''. This request was blocked as the content needs to be served over a secure connection (HTTPS). ...

What is the process of invoking a JavaScript function from Selenium?

How can I trigger a JavaScript function from Selenium WebDriver when using Firefox? Whenever I am logged into my website, I typically utilize this command in Firebug's Command Editor to launch a file upload application: infoPanel.applicationManager. ...

When res.write is called before res.send, an error occurs

I'm struggling to figure out why I am getting an error with the following code. app.get("/", (req, res) => { res.write("Hello"); res.send(" World!"); }) // Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers afte ...

Looking to retrieve an element from an array of objects based on consecutive object properties, I have a specific value to search for

const levels = [ { indexId: 'A', level: 1, name: 'A', parent: '0', }, { indexId: 'A-A1', level: 2, name: 'A1', parent: 'A', }, { ...

Is there a way for me to compare a string A1 with a value A2 located in index 0 within an array of values?

In my current task, I am attempting to compare two sets of strings: A1 must match A2 when selected. However, if A1 is chosen along with B1, C1, or D1, the comparison should return false. Similarly, selecting B1 should result in a match with only B2, while ...

The React error message refuses to disappear even after attempting to refresh the page

Currently, I am immersed in a React project where I have encountered an issue on the Register Page. All errors are being added to a Message component. Interestingly, even after encountering a React error (such as 'Cannot read properties of undefined&a ...

The Electron application is experiencing difficulties locating the module at /resources/app/index.js

Just started my journey with electron and successfully created my first electron application. It runs perfectly fine with npm start, but I encounter issues when trying to execute it with npm run. (I am using Ubuntu Linux). The command line interface displa ...

Performing numerous queries within codeigniter that are not suitable for sequential execution

In my database, I have a table called "event" that functions as a hierarchical category using the nested set model. Credit to Mark Hillyer for his informative post on this topic at Here is an excerpt of my code: $query = "LOCK TABLE event WRITE; SELECT ...

Deactivate specific choices from a dynamically generated dropdown menu in Angular 8

I am working on a dynamic dropdown feature with multiple fields. https://i.sstatic.net/28iQJ.png By pressing the + button, a new row is generated. Users can add any number of rows. My challenge is to prevent users from selecting previously chosen values i ...

TS2304: 'Omit' is a mysterious being that cannot be located

Encountered an issue while compiling my Angular project. This is a project that has remained unchanged for some time and is built daily by Jenkins. However, today it started failing and I'm struggling to determine the cause. ERROR in [at-loader] ./no ...

Angular input form is throwing an error because it is unable to retrieve the property 'name' of an undefined value

I've been working on creating a simple Angular component following a tutorial I found. The component fetches data from an angular-in-memory-web-api using a service called UserService. I have also added an input form for creating new users. The issue ...

Angular TimeTracker for tracking time spent on tasks

I need help creating a timer that starts counting from 0. Unfortunately, when I click the button to start the timer, it doesn't count properly. Can anyone assist me in figuring out why? How can I format this timer to display hours:minutes:seconds li ...

Passing a service into a promise in Angular 2 using TypeScript

Is there a way to pass a service into a promise? I am currently working on a promise that will only resolve once all the http requests are complete. However, I am facing an issue where this.jiraService is undefined. Is there a method to pass it to the co ...

Separate each element with a time gap when using the .each() function in

Below is the code snippet that I have: $('.action-button').each(function(i, obj) { $(obj).trigger('click') }); I am looking to introduce a delay between each iteration of the loop, ideally a 5-second delay. Is it achievable through se ...

Setting a fixed data value within a div for subsequent retrieval through a function

I found a helpful example that demonstrates how to convert numbers into words. You can check it out here. The function for converting numbers into words is implemented in the following HTML code: <input type="text" name="number" placeholder="Number OR ...

Disabling hover effects in Chart.js: A step-by-step guide

Is there a way to disable hover effects, hover options, and hover links on a chart using chart.js? Here is the code I have for setting up a pie chart: HTML.. <div id="canvas-holder" style="width:90%;"> <canvas id="chart-area" /> </div ...

Alternating the tooltip icon based on the field's condition

Looking for a way to create a tooltip for an input field that involves changing icons based on certain conditions. An example scenario is the password field in a registration form. Seeking advice on the best approach to achieve this. Currently, here' ...

Is there a way to bypass the default layout on app router in Next.js and implement our own custom page layout

Utilizing a default layout in layout.jsx, I passed all my other pages as children through props. In the page router, we typically configure the router path to specify which layout to use. However, in this new system, I am facing challenges with coding it. ...

Autocomplete feature in MUI allows filtering to begin after typing at least 3 characters

I've encountered an issue with the Autocomplete MUI component I'm using to filter a list of checkboxes. The popup with options should remain open at all times, but I only want the filtering to be triggered when the user input is more than 3 chara ...

Exploring the Vertical Tab feature in Java programming and MYSQL database integration

Currently, I am facing a challenge while using Java to insert data containing the Vertical Tab sign into MYSQL Tables. When working with php code, it is as simple as including "\v" in the data. However, when attempting to do the same in Java, I encoun ...