Wednesday, April 22, 2009

AT Issues: Large Collections

Two issues that we have encountered during testing are load time and slow performance. Given MSSA's 100,000+ instances, 2600+ resources, 7000+ accessions, and multiple locations, we've encountered lengthy start-up times and slow performance on even just a single client, let alone the dozen or so clients that will eventually be using the AT at a given time. Since we have not fully deployed the AT at this time, so we cannot yet accurately test how the AT will perform when running across several clients. Two things we will be doing to speed up performance is to move the AT to a dedicated server and to simplify our location assignments. Concerning the latter, in a previous iteration involving writing location information to the tables directly we created an enormous locations index, including many duplicate values, which significantly slowed client performance.

A third performance issue encountered concerns report generation. Again given the amount of data we have in the AT, running reports across our resources and accessions can take a considerable amount of time. Rather than running reports in the AT, which also aren't customizable, we use the database administration tool Navicat for MySQL to query the AT tables. Aside from being much faster, it's customizable (provided you know a little MySQL) and allows for batch editing.

1 comment:

  1. It is important to note that our performance may be further impacted by two different issues as we seek to implement the AT across Yale special collections:

    1. There will be an even larger base of data if we store all data in the same instance of the AT. This is the situation for the Collections Collaborative AT setup (see the previous posting for more information).

    2. As the AT moves from development to production, it will most likely move away from a dedicated server to a shared server administered by the Yale Library Information Technology Office.

    These performance issues will need to monitored as our development progresses.

    ReplyDelete