TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status
March 2003

Overview

The TPC held a General Council meeting on February 12 in Cypress, Texas. The main focus of the work was on refining existing benchmarks and laying the foundation for new benchmarks. On the OLTP front, work has begun on a modification of the TPC-C benchmark. With respect to decision support, work continues on large scale factors for TPC-H and R. Progress continues on the new DS benchmark with the completion of execution rules. As for the transactional web e-Commerce benchmark, development continues on the next version of TPC-W. The business to business operations as well as the processing functionality of these operations have been defined.

 

TPC-C and OLTP

The TPC-C Maintenance Subcommittee began the exploration of a modification to the current version of the TPC-C benchmark. The majority of the work consisted of defining the overall objectives for a follow-on version.

The objectives are:

  • Extend the life of the TPC-C benchmark
  • Reduce the number of disk spindles per tpmC
  • Have a fair comparison/upgrade path for existing version 5 results
  • Accomplish this with minimal changes to the exiting Version 5.1 benchmark
  • Add a requirement for data loss protection on the data drives (e.g., RAID)

Two proposals were reviewed along with several additional ideas for enhancements to TPC-C. Seven member companies volunteered to assist with or perform prototype work to evaluate the proposals.

The TPC received a proposal for a new OLTP benchmark. The General Council approved the formation of an OLTP Working Group to continue work on the proposal and, specifically, itemize known but not yet completed tasks that were identified in the proposal, develop early prototypes, and look at porting issues in the code that was provided.

 

TPC-H, R and DS

The TPC-H/R subcommittee has finished the wording changes to allow for 30,000 and 100,000 scale factors in TPC-H and TPC-R. Approval by the General Council of these versions was initially scheduled for February 2003. However, since the last meeting problems in dbgen’s data generator for large scale factors were discovered. The precision of the current data generator in dbgen is not sufficient to create correct data distributions of some columns for large scale factors. Therefore, the approval of larger scale factors in TPC-H/R is postponed until the data generator issues have been resolved. Target approval date is June 2003.

The TPC-DS development subcommittee continues its work on a new data warehouse benchmark. At the December meeting results of a simulator to emulate query execution assuming the current execution model discovered comparability problems between runs. At that meeting the group drafted a modified version of the execution rules. Since the last meeting the group has worked on specification wording for those execution rules. The execution rules and metric were discussed and a vote on the metric was taken. The next steps are to convert the queries into templates, conduct a query sensitivity analysis, and continue work on prototyping within each member company. The group is also validating the output of dbgen to ensure it reflects the specification definition.

 

TPC-W

The TPC-W subcommittee is currently developing the next version of TPC-W. The benchmark is a transactional web service benchmark consisting of a commercial application server and database interactions displaying ACID properties.

The TPC-W subcommittee is in the development phase of the web services version of the benchmark. The benchmark will provide for the measurement of the performance and throughput of a commercially available web/application server that provides web services to clients. The primary metrics are SIPS (Service Interactions Per Second), Price Performance which is $/SIPS, and the system availability date. An industry standard database product, which may be running on a second server, will be used to provide data storage and integrity of business information. The business model addressed by this benchmark is focused on the business to business scenario. The SUT will be driven by web service requests made through the industry standard and vendor neutral protocols of SOAP and XML. The business logic of the web services must be implemented such that it runs in the context of a managed environment.

Currently the benchmark defines the following B2B interactions:

  • New Customer
  • Create Order
  • Stock Management (background task)
  • Order Status
  • New Products List
  • Product Detail
  • Change Customer Payment Method
  • Change Item

The processing requirements for these operations have been defined, reviewed and accepted by the subcommittee. The subcommittee has also defined the database schema and the majority of the benchmark requirements in regards to the Test Run as well as reporting requirements.

The subcommittee is now ready to begin the prototyping phase. The subcommittee expects the prototype phase to last approximately until August 2003, during which the remaining clauses will be completed. The subcommittee has set the target schedule for benchmark approval process for October of 2003.

 

Public Relations Committee

The PR Committee is working on a campaign to promote the TPC and the TPC benchmarks to the industry. In this process, the TPC will be defining its key messages, addressing key industry influencers, and developing materials for distribution.

Activities under consideration include development of a press kit, article and speaker placement, and a possible press and analyst tour.

 


All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!