TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status
October 2003

Overview

The TPC held a General Council meeting on August 14 in Redmond, Washington. The main focus of the work was on refining existing benchmarks and laying the foundation for new benchmarks. On the OLTP front, work continues on the specification of a brand new OLTP benchmark with a focus in two general areas: TPC-provided source code and new/modified features of the benchmark. The TPC-C benchmark will be updated with a few definitions. With respect to decision support, progress continues on the new DS benchmark in the areas of ACID tests and data maintenance. The TPC-H and TPC-R benchmarks have been revised to version 2.1.0 by extending the scale factors to 30 and 100 terabytes. As for the transactional web e-Commerce benchmark, development on the next version of TPC-W continues with the definition of several key elements. The benchmark is in the prototyping phase of the web services version.

 

Current Benchmarks

TPC-C

The TPC-C Maintenance Subcommittee discussed the definition of a third-party and the branding of system components in the context of the TPC-C benchmark. The subcommittee will include these definitions in a future revision of the specification. The subcommittee also discussed and agreed to revise the TPC-C Auditor Certification Exam. The content will be compiled and the exam will be completed by October, 2003. This is part of the continuous effort the TPC undertakes to maintain the integrity of the audit process.

 

TPC-H and R

The General council voted to extend the allowable scale factors in TPC-H and TPC-R to 30 and 100 terabytes. The updated specifications, TPC-H 2.1.0 and TPC-R 2.1.0, become mandatory on October 13th, 2003 and take effect immediately. The new specifications are minor revisions allowing for benchmark publications to be fully comparable to the 2.0.0 revisions. The updated specifications will be posted on the TPC web site. Dbgen has been modified to use a 64 bit random number generator (RNG) for the new scale factors. The existing scale factors are not affected by the new 64-bit RNG since they use the existing 32-bit RNG.

 

New Benchmarks

OLTP

Over the past two months, the OLTP subcommittee work has been focused in two general areas: TPC provided source code for portions of the benchmark, and features of the benchmark. The OLTP benchmark will require use of TPC provided source code for portions of the benchmark implementation. This code provides functionality for database loading, runtime transaction input generation, runtime driver functionality, and simplified middle tier software. Major portions of this have been implemented and ported to a variety of platforms.

The subcommittee has also received and reviewed position papers proposing features for consideration in the benchmark. The topics covered include the following.

  • Middle Tier Software Functionality
  • Non-Linear Database Scaling with respect to Performance
  • Interactions between Client Middle Tier Software and Database Server Software
  • Data Types used in the Schema
  • Isolation Levels used by all Transactions
  • Publish/Subscribe Methodology for Implementing Interrelated Transactions

Draft revisions of the specification continue to be developed as members of the subcommittee review existing content and develop new content. Moving forward, the subcommittee will focus on reviewing and solidifying transaction profiles and preliminary development of prototype environments.

 

TPC-DS

The TPC-DS subcommittee continued its work on developing TPC-DS in the areas of ACID tests and data maintenance wording. The data maintenance model is now ready for prototyping. Member companies will be extending their prototypes with the data maintenance portion of the benchmark. The ACID test in TPC-DS will test ACID properties on the same level as TPC-H with the exception of schema changes.

 

TPC-W

The TPC-W subcommittee continued its development of the next version of TPC-W. Version 2 is a transactional web service benchmark consisting of database interactions displaying ACID properties running against a commercial application server. The primary metrics are SIPS (Service Interactions Per Second), Price Performance which is $/SIPS, and the system availability date.

The processing requirements, database schema and population, test run requirements, and reporting requirements for the web service interactions have been defined. The TPC-W subcommittee is in the prototyping phase of the web services version of the benchmark. All of the implementation details have been reviewed and accepted by the subcommittee. The ACID tests have been developed and approved. These tests demonstrate ACID compliance not only for the database system, but for the distributed transactions and message queues as well. The subcommittee has also defined the benchmark pricing rules.

The requirements for the environment in which the benchmark is measured have been defined. The environments must be either compliant with the Java Runtime Environment (JRE) 1.3 or greater as published by Sun Microsystems, or ECMA-335 as published by the European Computer Manufacturers Association -- the ECMA-335 standard defines the Common Language Infrastructure (CLI) in which applications written in multiple high level languages may be executed in different system environments without the need to rewrite the application to take into consideration the unique characteristics of those environments. Requirements for random number generators and seeding methods have been defined.

The subcommittee will be reviewing prototype data to adjust the benchmark profile through October 2003. The subcommittee is adopting additional requirements such as adhering to the WS-I basic profile (BP1.0) for web service interoperability.

 

Other TPC Activities

Public Relations Committee

The PR Committee continues its work on the campaign to promote the TPC and the TPC benchmarks to the industry. The committee has selected the PR firm of Owen Media for the campaign and will begin implementing its plan.

 


All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!