TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status
November 2004

Overview

The TPC held a General Council meeting on October 7th in Houston, Texas. The main focus of the work again was on refining existing benchmarks and laying the foundation for new benchmarks. On the OLTP front, wording changes were proposed for TPC-C version 5.3. Work also continues on the new TPC-E specification, with progress on increasing the size of the initial database population. With respect to decision support, work has begun to address the large scale factor issue on TPC-H and R. Also, development of the DS benchmark continues in the areas of queries, query generator, and data generator. As for the transactional web e-Commerce benchmark, work continues on the freshly renamed TPC-App specification with substantial time and resources spent on testing and reviewing modifications to the interaction parameters, mix, and workload in order to increase the application server to DBMS processing ratio. Finally, the Pricing Subcommittee focused on completing the Pricing specification and working with the TPC benchmark subcommittees to coordinate changes that will be compatible with the specification.

 

Current Benchmarks

TPC-C

The TPC-C Maintenance Subcommittee passed a wording change to Clause 5.3.3 which allows sponsors to cache the menu requests provided the component(s) used for this caching are included in the system under test and therefore priced.  This wording will be included in the next version of the TPC-C specification.

 

TPC-H and R

The TPC-H/R Maintenance Subcommittee addressed the open issues with scale factor extension to 100,000. The committee intends to resolve these over the next couple of weeks.

 

New Benchmarks

TPC-E (OLTP)

Over the past month, the TPC-E subcommittee has continued its efforts in the development of a new OLTP benchmark.

During this time a new draft of the specification has been approved. Work has completed on increasing the size of the initial database population with data that has characteristics similar to data generated at runtime. A new transaction profile to work against this data has been implemented. This adds an I/O component to the runtime workload. Prototyping with the new transaction has started.

Moving forward, work will focus on prototyping and adjusting the existing runtime workload.

 

TPC-DS

The TPC-DS subcommittee continued its work on developing TPC-DS in the areas of queries, query generator, and data generator. Data Output from the Data Generator has been verified to assure correctness of column distributions that are used in the current query set as selectivity predicates.  In order to assure query soundness, including semantic correctness, number of rows returned, elapsed time, the subcommittee reviewed prototype data.  Column distributions were reviewed to assure that queries impose comparable workload.  Qgen has been improved from the last meeting to cover all substitution types necessary for current queries.  It is now able to generate both multiple copies of the same query and permutations of multiple queries with different substitution parameters.  Queries were converted into templates so that sensitivity analysis can be conducted.  The group also started working on cleaning up the specification.

 

TPC-App (TPC-W)

The TPC-W subcommittee completed all changes incorporated from the comments received during the public review phase of TPC-W Version 2. The benchmark has been renamed TPC-App to better reflect the focus of the benchmark, the application server. TPC-App is a web service benchmark consisting of transactions involving database interactions, durable message queue operations, and displaying ACID properties running against a commercial application server. The primary metrics are Total Throughput (SIPS, Service Interactions Per Second), Throughput per Application Server (SIPS / App Server), Price Performance (for example $/SIPS), and the system Availability Date.

The subcommittee has spent the last two months testing and reviewing modifications to the interaction parameters, mix, and workload in order to increase the application server to DBMS processing ratio.  Previous prototype data indicated the application to DBMS ratio was around 2 : 1.  The subcommittee has incorporated some new ordering, shipping, and stock management business logic that improved this ratio.  Additionally, image serialization for the Product Detail Web Service Interaction has been included.  The current ratio is now approximately 6 : 1.   

The subcommittee is finalizing the wording to incorporate these changes as well as determining the final interaction mix.

 

Other TPC Activities

Public Relations Committee

The PR Committee continues its work on the campaign to promote the TPC and the TPC benchmarks to the industry. A TPC overview article has been submitted to a periodical publisher. Presentations for TPC speakers are being prepared and reviewed. A prototype of a new customized information system for subscribers of TPC lists and reports has been completed.

 

Pricing

The TPC Price Subcommittee drafted a pricing specification that can be applied to all TPC benchmarks. The specification does not substantially change what is priced, but the requirements for reporting and ordering are enhanced and strengthened, allowing improved verifiability of pricing. If approved, the pricing specification will be referenced by all existing and future TPC specifications for a consistent methodology.

The subcommittee’s focus areas are to complete the specification and to work with the TPC benchmark subcommittees to coordinate changes that will be compatible with the pricing specification.

 

All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!