AT Issues: Resource Handling

We have run into a significant performance issue at the half-way juncture in our legacy data migration efforts. We have found that Resources are taking much too long to load--small ones are taking at least 15 seconds; medium resources are taking up to 3 minutes; our largest ones are taking upwards of 5 minutes. Again, since we're only half-finished migrating our legacy data, the problem is likely to get much worse, especially as we begin to create/describe the thousands of digital objects we've been waiting to use the AT for.

We think the reason for the problem is how the AT loads Resources, i.e. recursively issuing SQL commands for each component, rather than issuing a single command for all resource components at once. Hence, given a large, complex finding aid (e.g. one with several levels of nested components and associated instance and location information), the AT grinds to a stand-still as it tries to systematically open all of the hierarchy. The main issue here appears to be the number and depth of nested components. Resources with only one or two levels of nested components do seem to open right up, the exception being two-level resources that have larger numbers of sub-components. However, resources that have three or more levels of nested hierarchy always open slowly, even if they don't have many actual associated instances.

We have tried to address the problem by maximizing the memory assigned to the client (see AT Wiki), boosting the database server's use of system memory, increasing the key buffer size, and engaging Memory Lock to discourage swapping--all to no avail. These steps however did boost performance of the MySQL database administration tool Navicat, which we use to access the MySQL database. This therefore points to the problem as lying in the AT and its Java encoding. This hypothesis was further supported when we set up a slow query log (i.e. anything over 2 seconds) on the database and tested resource loading, finding that no such queries were logged. As a result, all of our efforts seem to suggest that while we can indeed speed up operation of the MySQL database, we cannot speed up the performance of the Java that hits or interacts with the database. Since this however lies in the core of the AT code, it would have to be addressed either by the AT development team and put into the next/final release, or developed externally and then somehow offered back to the AT community. We are not sure yet how to proceed. An alternative that we and perhaps others might be happy with would be for only the top-level components (i.e. c01s) be built and displayed within the main Resources window, with further sub-components built when clicked. After all, you don't need to see/retrieve the whole Resource to make an edit to a specific part.

One of the questions this problem has raised for us is why hasn't anyone noticed/reported this performance issue? Was this borne out in scale testing? Surely there are other repositories who have loaded large finding aids, or who are working with complex resources or digital objects. Are they not using/editing these Resources and therefore not noticing the lag in performance? Perhaps they haven't ingested the amount of instance and location information that we have? Or, is there an upper data limit to the AT's functionality? That seems short-sighted. Regardless, others are bound to run into the problem sooner or later as they begin to ingest more and more data into the AT. As a result, we hope that progress can be made to address this critical problem.