Explore a vast array of items in a collection

I have a massive collection of various objects that I need to sift through.

With over 60,000 elements, the search operation can sometimes be painfully slow.

One typical object in this array has the following structure:

{
  "title": "title"
  "company": "abc company"
  "rating": 13 // internal rating based on comments and interactions
  ...
}

My aim is to search for items based on their title and company information, while prioritizing them by their respective ratings.

This snippet showcases my current search implementation:


onSearchInput(searchTerm) {
    (<any>window).clearTimeout(this.searchInputTimeout);
    this.searchInputTimeout = window.setTimeout(() => {
        this.searchForFood(searchTerm);
    }, 500);
}

searchForFood(searchTerm) {
    if (searchTerm.length > 1) {
        this.searchResults = [];
        this.foodList.map(item => {
            searchTerm.split(' ').map(searchTermPart => {
                if (item.title.toLowerCase().includes(searchTermPart.toLowerCase())
                    || item.company.toLowerCase().includes(searchTermPart.toLowerCase())) {
                    this.searchResults.push(item);
                }
            });
        });

        this.searchResults = this.searchResults.sort(function(a, b) {
            return a.rating - b.rating;
        }).reverse();

    } else {
        this.searchResults = [];
    }
}

Inquiry: Any suggestions on how to enhance the search algorithm for better performance?

Answer №1

Here are some helpful suggestions:

  • Consider moving a portion of the search process to the backend instead of overwhelming the front end with searching through 60,000 items. If you must do it on the front end, try breaking the search into chunks and utilizing setImmediate() to prevent the user's browser from freezing.
  • Perform splitting and lowercasing of the search term outside of the loop for optimization.
  • Rather than using map() without returning values, consider using forEach() or filter() to match items more efficiently.
  • Use some() while iterating over search terms for early return opportunities.
  • No need to re-assign arrays after using sort(), as it mutates the original array.
  • Avoid using reverse() after sort(); swap conditions to b - a instead.
  • For performance testing at this scale, experiment with methods like includes(), indexOf(), custom for loops, and possibly match().

Answer №2

Alex has some great suggestions. Another idea I have is to preprocess the data during idle time (without delaying the initial rendering or interaction). By transforming the data into a modified prefix trie, you can search for items in O(k) time where k is the length of the search term. Currently, you are searching in O(kn) time because you examine every item and then use an includes function which also takes k time (considering the toLowerCase as well).

If you're not familiar with what a trie is, the code snippet below might give you an overview or you can research more about it online. Essentially, a trie is a data structure that maps characters in strings using nested hash maps.

Below is a sample code on how you could build the trie:

function makeTries(data){
    let companyTrie = {};
    let titleTrie = {};
    data.forEach(item => {
        addToTrie(companyTrie, item.company, item, 0);
        addToTrie(titleTrie, item.title, item, 0);
    });
    return {
        companyTrie,
        titleTrie
    }
}

function addToTrie(trie, str, item, i){
    trie.data = trie.data || [];
    trie.data.push(item);

    if(i >= str.length)
        return;
    if(! trie[str[i]]){
        trie[str[i]] = {};
    }
    
    addToTrie(trie[str[i]], str, item, ++i);
}

function searchTrie(trie, term){
    if(trie == undefined)
        return [];
    if(term == "")
        return trie.data;
    return searchTrie(trie[term[0]], term.substring(1));
}

var testData = [
    {
        company: "abc",
        title: "def",
        rank: 5
    },{
        company: "abd",
        title: "deg",
        rank: 5
    },{
        company: "afg",
        title: "efg",
        rank: 5
    },{
        company: "afgh",
        title: "efh",
        rank: 5
    },
];

const tries = makeTries(testData);
console.log(searchTrie(tries.companyTrie, "afg"));

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Error encounter in JSP is nested within another JSP file, both of which utilize jQuery

Before proceeding, it is important to note that I am in the process of selecting a month using a datepicker and a specific meter (by its serial number). Once this information is selected, a query will be sent to a MySQL database to retrieve data for plotti ...

Incorporate a fresh label for a function utilizing AngularJS

I want to insert a new HTML tag with an event attached to it. Here is an example of what I am trying to achieve: <html ng-app="module"> <head> <script src="http://code.jquery.com/jquery-latest.min.js" type="text/javascript"></script&g ...

Getting latitude and longitude from Google Maps in a React Native appHere are the steps to

Looking to retrieve the latitude and longitude from Google using React Native. As a newcomer to React Native, I have been unable to find any resources to address this issue. Any assistance would be greatly appreciated. Thanks! Thanks ...

Encountered an error while trying to install Drivelist: Module 'C:Program Files odejs ode_modules pm ode_modules ode-gypin' not found

My electron project relies on the drivelist dependency. However, when I attempt to run "npm install," I encounter an error indicating that the node-gyp\bin folder is missing. Surprisingly, I do have the node-gyp\bin in my node modules and even in ...

Attempting to showcase a collection of values, specifically focusing on highlighting the even numbers within the array

const sampleArray = [ 469, " " + 755, " " + 244, " " + 245, " " + 758, " " + 450, " " + 302, " " + 20, " " + 712, " " + 71, " " + 456, ...

SonarLint versus SonarTS: A Comparison of Code Quality Tools

I'm feeling pretty lost when it comes to understanding the difference between SonarLint and SonarTS. I've been using SonarLint in Visual Studio, but now my client wants me to switch to the SonarTS plugin. SonarLint is for analyzing overall pr ...

Retrieve the maximum numerical value from an object

My goal is to obtain the highest value from the scores object. I have a global object called "implementations": [ { "id": 5, "project": 'name project', "scores": [ { "id": 7, "user_id": 30, "implement ...

How can I keep the cursor in place while editing a phone number field on Sencha ExtJS?

After one backspace move, the cursor on the phone number field automatically moves to the end which can be inconvenient if the user only wants to edit the area code. Unfortunately, I am unable to post images at the moment due to insufficient reputation. B ...

Developing a real-time form that syncs with a MYSQL database for automatic updates

I am currently working on developing a form with multiple drop-down menus. The first dropdown is populated with 'Customer Name' data retrieved from my MYSQL database. Upon selection, the second dropdown menu below it should display the available ...

Tips for troubleshooting a timeout issue in electron-html-to with node.js

Encountering a timeout issue while attempting to convert an html file to pdf using electron. The js app is being run through node. `{ Error: Worker Timeout, the worker process does not respond after 10000 ms at Timeout._onTimeout (C:\Users\Owner ...

Accessing data from a CD-ROM or DVD through a web browser

I am currently working on developing a web application for a friend that will enable users to simply click a button to upload all the content from the CD-ROM or DVD they have inserted directly to a server. It's not feasible to rely on the standard br ...

Maximizing the potential of $urlRouterProvider.otherwise in angular js

I am encountering an issue with redirecting to the login screen when there is no UABGlobalAdminId in my storage. I have tried using $urlRouterProvider.otherwise but it doesn't seem to be functioning properly. When I attempt to debug, the console does ...

Retrieve the current URL of an IFRAME

How can I easily retrieve the current URL from an embedded iframe? The user will be navigating through various web pages. I assume that some JavaScript code would be needed for this task. ...

The art of positioning images and creatively cropping

Seeking advice on allowing users to dynamically position and clip an image on a webpage. I've tried using CSS and JavaScript for clipping and resizing, but it's not working as expected. If PHP could provide a solution, that would be ideal for my ...

Retrieving Information from API using Vue.js

In the code snippet below, I am displaying data from an API for all flats on a single page. However, I am facing difficulty in showing the floor number for each flat. The JSON body is as follows: { "response": [ { "fl ...

Creating a dynamic image carousel using jQuery

Today, I completed the jQuery course on Code Academy and am now trying to create an image slider using it. While I have a general idea of how it should work, I feel like I'm missing a key element. My goal is to have the slider continuously running whe ...

Problem with Layering in Javascript Canvas Game using Kinetic JS

Currently, I am attempting to replicate the game found at and delving into html5 canvas and kineticJS for my study. Here is the link to my fiddle: http://jsfiddle.net/2WRwY/7/ The issues I am facing are as follows: The tail part of the player in the f ...

"Ajax validation is causing the duplication of the HTML page content within an HTML

My PHP username validation with Ajax code seems to be duplicating my HTML page inside an HTML div element, which is used for displaying Ajax errors. Despite trying various solutions and searching on Google, I have not been able to find a resolution yet. Th ...

Have Vue props been set to null?

Currently, I have a component within my vue.js application that looks like this: export default { props: ['forums'], methods: { increment(forum, index) { ForumService.increment(forum) .then(() => { ...

Executing code on the server-side (back-end) using Javascript: A step-by-step guide

Imagine wanting to execute JavaScript on the server side instead of exposing the code to the client. For example: Starting with the JS native window Object from the browser. Performing all JavaScript processing in the backend by passing the window Object ...