Performance Testing

Performance Testing Results

• Fiz Karlsruhe Fedora Performance and Scalability Wiki
• Fedora Repository 2.2.4 Performance Tests
• Sun Testing

Performance Testing Initiative

The is the beginning of a new performance testing initiative for the Fedora Repository and components integrated into Fedora Commons Framework. This work builds upon the superb performance tests and analysis performed by Fiz Karlsruhe and the performance testing included in each release of the Fedora Repository. We hope this will provide a common performance testing framework and begin a community effort to characterize Fedora performance to document best practices for configuration and identify bottlenecks for subsequent software development.

Testing Framework

Performance testing will be accomplished using testbeds kindly provided by a number of community sources. Testing will also be accomplished using Cloud infrastructure such as Amazon's offerings. Details of the test configurations will be documented in this Wiki as the configurations are implemented.

In most cases a test configuration will always consists of at least four parts:

1. A load injector
2. An instance of a Fedora Repository
3. A relational database
4. A collection point for the results

Other typical components include:

1. A resource index (RDF database)
2. An OIA-PMH provider
3. A search engine
4. Ingest components
5. Client software
6. Other framework components or services
7. Fedora Repository backend services

A number of frameworks are used by the community including several home-grown test platforms. Two well supported open-source frameworks are Apache JMeter and The Grinder. Either are well suited for this purpose and have their proponents. In many cases tests written for one framework can easily be re-purposed for another. Hopefully, experience with more than one framework and tests from all will be shared here. Just as important is publication of test results and analyses.

Testing Frameworks

• Using The Grinder
• Using Apache JMeter (Call for Participation)

Test Resources

To be valid, a performance test must minimize or isolate external influences. This can be done with a dedicated platform (and network if a multi-host configuration is used). Unfortunately, cloud platforms are shared and the performance allocated to any given set of processes varies with time. To utilize a ahared platform, you must monitor it and factor its load into the performance calculations. This can be complicated since there are so many potential factors to consider but "rough order of magnitude" results are still quite useful. This is still useful and may be the most practical approach for interim performance test.