How to improve Client Side DataTable performance in C# ASP.NET MVC Project?

In our test environment, it takes about 55 seconds to load approximately 5,500 records. We are using the following DataTable script:

    <script>
        $(document).ready(function() {
            $("#DataTable").DataTable({
                stateSave: true,
                stateSaveCallback: function(settings, data) {
                    localStorage.setItem(`DataTables_${settings.sInstance}`, JSON.stringify(data));
                },
                stateLoadCallback: function(settings) {
                    return JSON.parse(localStorage.getItem(`DataTables_${settings.sInstance}`));
                },
                stateDuration: 0,
                searching: true,
                columnDefs: [
                    {
                        orderable: false,
                        searchable: false,
                        targets: @ViewBag.NoSort
                    }
                ],
                order: [[@ViewBag.InitSort, "desc"]]
            });

            $(".dataTables_length").addClass("form-inline pb-2");

            $(".dataTables_length select").addClass("mx-1");

            $(".dataTables_filter").addClass("form-inline float-right pb-2");

            $(".dataTables_paginate").addClass("float-right");

        });
    </script>

Not sure if ~5,500 records call for a server side DataTable script.

How can this DataTable script be updated to improve response time/performance?