TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status
May 2003

Overview

The TPC held a General Council meeting on April 17 in Durham, North Carolina. The main focus of the work was on refining existing benchmarks and laying the foundation for new benchmarks. On the OLTP front, work continues on a modification of the TPC-C benchmark. In parallel, the TPC has begun work on a new OLTP benchmark. With respect to decision support, a solution was found for the random number generator which will be used for large scale factors for TPC-H and R. Progress continues on the new DS benchmark with the refinement of the metric and execution rules and schema. As for the transactional web e-Commerce benchmark, development continues on the next version of TPC-W.

 

TPC-C and OLTP

The TPC-C Maintenance Subcommittee continued the exploration of a modification to the current version of the TPC-C benchmark. The objectives for this follow-on version continue to be:

  • Extend the life of the TPC-C benchmark
  • Reduce the number of disk spindles per tpmC
  • Have a fair comparison/upgrade path for existing version 5 results
  • Accomplish this with minimal changes to the exiting Version 5.1 benchmark
  • Add a requirement for data loss protection on the data drives (e.g., RAID)

The subcommittee reviewed prototype data and revised the list of experiments to be conducted. Prototyping will continue in an effort to refine the contents and ramifications of the changes proposed to the existing TPC-C benchmark.

The General Council formed an OLTP Technical Development Subcommittee to develop a new OLTP benchmark based upon a proposal received in response to the TPC’s benchmark development guidelines.

 

TPC-H, R and DS

To address the shortcomings of the random number generator that were identified at the last meetings, the TPC-H/R subcommittee decided on a 64 bit linear congruential precision random number generator (PRNG) to be used for larger scale factors in TPC-H/R. Tests have shown that the random numbers produced by this PRNG have very similar characteristics compared to the existing 32 bit PRNG. Members are currently ensuring that the new PRNG runs on all member platforms and, most importantly, yields the same results on all platforms. This research is scheduled to be finished by the June TPC meeting. At this meeting the TPC-H/R subcommittees will vote on a new specification including 30,000 and 100,000 scale factors supported by a new dbgen.

The TPC-DS development subcommittee continues its work on a new data warehouse benchmark. The main focus of this meeting was to refine the metric/execution rules and schema. Further sensitivity analysis has revealed that one can take unfair advantage of the current metric/execution rules. This has been addressed. Various members have started to build prototypes. During this process they have discovered problems with the data generator (dbgen2). These problems have been addressed by the subcommittee. Until the next meeting our focus will be on converting queries into templates and to further test the data generator to ensure data consistency.

 

TPC-W

The TPC-W subcommittee continued its development of the next version of TPC-W. The benchmark is a transactional web service benchmark consisting of a commercial application server and database interactions displaying ACID properties.

The TPC-W subcommittee is in the prototyping phase of the web services version of the benchmark. The benchmark will provide for the measurement of the performance and throughput of a commercially available web/application server that provides web services to clients. The primary metrics are SIPS (Service Interactions Per Second), Price Performance which is $/SIPS, and the system availability date. An industry standard database product running on a second server will be used to provide data storage and integrity of business information. The business model addressed by this benchmark is focused on the business to business scenario. The SUT will be driven by web service requests made through the industry standard and vendor neutral protocols of SOAP and XML. The business logic of the web services must be implemented such that it runs in the context of a managed environment. The benchmark incorporates common B2B functionality such as distributed transactions and message queuing utilizing durable queue.

Currently the benchmark defines the following B2B interactions:

  • New Customer
  • Create Order
  • Shipping Process
  • Stock Management
  • Order Status
  • New Products List
  • Product Detail
  • Change Payment Method
  • Product Update

The processing requirements for these operations have been defined. All of the core implementation requirements for these interactions have been reviewed and accepted by the subcommittee. The subcommittee has also defined the majority of the benchmark requirements in regards to the Test Run.

The subcommittee has defined the requirements for the environment in which the benchmark is measured. The environments must be either compliant with the Java Runtime Environment (JRE) 1.3 or greater as published by Sun Microsystems, or ECMA-335 as published by the European Computer Manufacturers Association.

The subcommittee is reviewing proposals for additional implementation requirements such as adhering to the WS-I basic profile for web service interactions and requiring a standard random number generation algorithm. The schedule for submitting the benchmark for TPC General Council vote is set for December 2003.

 

Public Relations Committee

The PR Committee continues its work on the campaign to promote the TPC and the TPC benchmarks to the industry. An RFP for PR services was drafted and distributed, and the responses to the RFP are currently being reviewed.

 


All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!