JSON / jQuery performance with lots of data

You should use additional library, like backbone & underscore stack (underscore has fenomenal filtering & other utilities, and templating, and the stack is perfect for your purpose), or you should consider angular.js, which also has much higher level of operation on lists, objects, data & html. In angular you can achieve complex filtering & dynamic html updating probably with 2-3 lines of code.

In any case, regardless of performance, don't do it with html <li> items, do it with data. These libraries I suggested will make it much simpler & faster.

Worth learning.

as a general tip for performance try to recude DOM manipulation, if you generate a nel HTML code and change the content of your container tag using $().html() it's a lot faster than traversing 2000 tag elements and change the visibility of 1000 of them.

You can also listen to the click event on the container avoiding 2000 event listeners

Hope this helps

instead of filtering this on the client side, why not just generate the list on the server side and send it to the client?

html

<div id="bigAssList">
   <ul>
      <li></li>
      A billion rows...
   </ul>
</div>

js

$.ajax( {
   url: 'someurl.php',
   data: someFilterCriteria,
   success: function( response ) {
      $( '#bigAssList' ).html( response );
   }
} );

php

// query based on filter criteria
// output list in ul html form

And even better than that, pagenate your data. I doubt anyone needs 2000 rows on one page. And why use Ajax to do this at all? Wouldn't a submit/page refresh be just as effective? Might be unnecessarily complicating this.

I have an HTML+JavaScript web app that has a page that contains as many as 50,000 rows, each with unique data. In the original code that I inherited, the server would generate the full markup (approximately 30MB of HTML), and then generate a filtered list (again on the server) every time you changed some filter constraint. The performance was so bad that the page would default to the most filtered (and least usful) view and include a warning that expanding the results only worked on modern computers. Here is how I fixed it and why I made each choice.

First to address the volume of data being sent I created a lean JSON representation (take a look at Google Charts Datasource API for a solid example of this) (this reduced the data down to about 1 MB when I combined it with GZip compression)

Then the next problem was that a table with 50,000 rows (each with 4 cells) creates 250,000 DOM Nodes (each <tr> and <td> is a node), that is a huge burden on Selector engines and the rendering pipeline. To fix this I fetched the JSON via AJAX and stored it's results in a variable that was outside of the AJAX fetch function. Then I wrote code to use this cached data and render it in pages (so page change events don't go back to the server). I played with the best number of rows per page (500 seemed the most compatible for my needs). As for hiding verses rebuilding, I say rebuild. The performance gains of hiding specific cells are quickly lost when the number of things you are going to hide increases.

Finally because every node isn't visible on the page at once things like Ctrl+F to find are broken. For this I created a filtered render function so table filtering can be handled with a in page UI (which lets me do things like find based on specific columns or even with data that isn't rendered in the table but is passed over the JSON). The other challenge is printing, again because no single page has a visual representation of everything. To address this I did something a little more complicated. Using MemcacheD I stored the filtered results on the server and added a export to PDF button that would take the cached copy of the result set (you have to adjust MemCacheD to store things larger than 1MB). Form a user's perspective the PDF (which is generated by a server that I can control) can take more punishment than a browser that I don't control.

In the end I'm very satisfied with the results. the page loads in seconds now, users like that their browsers don't crash, and the solution is generic enough that it is easy to apply it to all of our other reports.