TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status
January 2003

Overview

The TPC held a General Council meeting on December 11, in Dana Point, California. The main focus of the work was on refining existing benchmarks and laying the foundation for new benchmarks. On the OLTP front, version 5.1 of the TPC-C benchmark was completed and approved. With respect to decision support, enhancements to TPC-H and R v2.0 were considered. Work also continues on the new DS benchmark with the completion of two prototype databases. As for the transactional web e-Commerce benchmark, development continues on the next version of TPC-W. The business to business operations as well as the processing functionalities of these operations have been defined.

 

TPC-C and OLTP

The TPC-C Maintenance Subcommittee finished work on the proposed Version 5.1 benchmark specification. The revisions for v5.1 consist primarily of editorial, audit, and runtime changes. The workload components of the benchmark remain unchanged.

The changes are:

  • Increasing the throughput required for the durability tests. (Clause 3.5.4)
  • Definition of the Checkpoint Duration as used in the benchmark. (Clause 5.5.2.2)
  • Removed the requirement that the Measurement Interval must be an integral multiple of the Checkpoint Interval. (Clause 5.5.2.2)
  • Editorial correction to example SQL for Delivery transaction (Appendix)
  • Added wording to address the use of Profile-Directed Binary Optimizations (Clause 6.6.7)
  • Typographical correction to Consistency test 11 (Clause 3.3.2.11)
  • Minor update to reported numerical quantities for checkpoints (Clause 8.1.1.3)
  • Defined additional durability tests for clustered database configurations. (Clause 3.5)
  • Defined additional requirements for software maintenance as priced in the benchmark. (Clause 7.3.3)
  • Defined the pricing requirements of network interface cards in the Client System(s). (Clause 7.1.2)

The approved V5.1 specification will become effective on February 9th, 2002. All TPC-C v5.0 results are comparable to the new v5.1 results and will be removed at the same time as v5.1 results.

 

TPC-H, R and DS

The group continued working on wording changes to allow for 30,000GB and 100,000GB scale factors in TPC-H and TPC-R. The new version will be TPC-H/R 2.0.1. Approval by the General Council of these versions is pending research regarding the line-count for the LINEITEM table at these large scale factors. The target date for a final vote is February 2003.

On the TPC-DS front, the group enhanced the data and query generator. The data generator with the corresponding SQL is now available on the TPC-DS web site. IBM and Oracle have built prototype databases. Other members have announced that they will build prototype databases next year.

A simulator to emulate query execution assuming the current execution model has discovered comparability problems. During this meeting the group defined modifications to the execution model and tasked a group of members to develop specification wording by the next meeting.

The next steps are to convert the queries into templates and to conduct a query sensitivity analysis and to finalize the execution model. The group is validating the output of dbgen to ensure it reflects the specification definition.

 

TPC-W

The TPC-W subcommittee is currently developing the next version of TPC-W. The benchmark is a transactional web service benchmark consisting of a commercial application server and database interactions displaying ACID properties.

The TPC-W subcommittee is in the development phase of the web services version of the benchmark. The benchmark will provide for the measurement of the performance and throughput of a commercially available web/application server that provides web services to clients. The primary metrics are SIPS (Service Interactions Per Second), Price Performance which is $/SIPS, and the system availability date. An industry standard database product running on a second server will be used to provide data storage and integrity of business information. The business model addressed by this benchmark is focused on the business to business scenario. The SUT will be driven by web service requests made through the industry standard and vendor neutral protocols of SOAP and XML. The business logic of the web services must be implemented such that it runs in the context of a managed environment.

Currently the benchmark defines the following B2B interactions:

  • New Customer
  • Create Order
  • Stock Management
  • Order Status
  • New Products List
  • Product Detail
  • Get New Order ID
  • Change Customer Payment Method

The processing requirements for these operations have been defined. Most of the requirements for these interactions have been reviewed and accepted by the subcommittee. The subcommittee has also defined the majority of the benchmark requirements in regards to the Test Run as well as reporting requirements.

The subcommittee will be defining the details of the remaining interactions to prepare for a prototyping phase expected to begin in February 2003. The subcommittee expects the prototype phase to last approximately six months, during which the remaining clauses will be completed. The subcommittee has set the target schedule for benchmark approval vote for October of 2003.

 

Public Relations Committee

The TPC-H and R version 2 results are now available on the web site. Beginning January 22, the version 2 results will be featured on the primary TPC-H and R result pages, with links from there to the version 1 results pages. The older versions of the TPC-H and R results will remain on the web site until April 30, 2003.

Several updates to the web site are being deployed this year. A new search feature has been implemented to enable search and retrieval of all the documents that are publicly available. A more flexible query page is also in development to replace the current advanced sorting options page.


All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!