TPC photo
The TPC defines transaction processing and database benchmarks and delivers trusted results to the industry
    Document Search         Member Login    
     Home
About the TPC
Benchmarks
      Newsletter
      Join the TPC
      Downloads
      Technical Articles
      TPCTC
      Performance-Pulse

TPC Benchmark Status July 2014

Overview

Published about 3 times per year, the TPC Benchmark Status Report is a digest of the activities of the TPC and its technical subcommittees.

The TPC held meetings in February, April, and June in Austin, Raleigh, and Portland respectively. A new class of benchmarks was launched with the TPCx-HS "Express" benchmark. The TPC-E and TPC-H specifications were updated following enhancements and minor corrections and clarifications. The TPC’s sixth Technology Conference, TPCTC 2014, will be held in conjunction with the annual conference on Very Large Databases (VLDB), in Hangzhou, China.

 

Current Benchmarks

TPC-C

The TPC-C Maintenance Subcommittee had no outstanding issues to discuss. The subcommittee continues to monitor the benchmark, along with any TAB interpretations and rulings, for discussion and possible inclusion in future versions of the TPC-C specification.

 

TPC-DS

The TPC-DS Maintenance Subcommittee is discussing changes necessary to create a version of TPC-DS that can be run on Hive/Hadoop platforms. That new version will not require the database product to enforce Atomicity, Consistency and Isolation. But it will be required to enforce Durability. The new version will also not require the database product to run the DS workload using the SQL text as provided. The SQL text may be modified as long as the modification is functionally equivalent to the SQL text.

 

TPC-E

The TPC-E Maintenance Subcommittee released a new revision of the TPC-E benchmark standard (v1.13.0) that adds many clarifications to the Response Time, Test Run, Required Reporting, Durability, Data Accessibility, and Full Disclosure Requirements clauses of the specification, and further extends the unit testing framework and resolves numerous portability issues, minor bugs and code cleanups in EGen.

This revision of the standard was approved and is available immediately, and v1.12.0 becomes obsolete at 11:59 PM PT on June 23, 2014. Prior results and Version 1.13.0 results are comparable. Prior results do not need to be upgraded to Version 1.13.0 results.

 

TPC-H

The TPC-H Maintenance Subcommittee released Version 2.17.0 of the TPC-H specification, which corrects issues with the answer set and updates correspondingly dbgen and qgen.

 

TPC-VMS

The TPC-VMS Maintenance Subcommittee had no outstanding issues to discuss. TPC-VMS specifies the rules and methodology for measuring and reporting TPC-C, TPC-E, TPC-H and TPC-DS metrics running in a Virtualized Environment.

 

TPC-DI

Data Integration (DI), also known as ‘ETL’, is the analysis, combination, and transformation of data from a variety of sources and formats into a unified data model representation. Data Integration is a key element of data warehousing, application integration, and business analytics solutions. The variety and volume of data is always increasing and performance of data integration systems is critical. However, there has been no industry standard for measuring and comparing the performance of DI systems.

The TPC-DI benchmark subcommittee is continuing refinement of the specification.

 

TPCx-HS

TPCx-HS is the TPC’s first Express Benchmark™ – developed to provide verifiable performance,  price / performance, availability, and optional energy consumption metrics of big data systems. Express-class benchmarks are being developed in response to overwhelming demand for a turnkey alternative to enterprise-class benchmarks, which have distinct advantages but are also substantially more time-intensive and costly to run.

TPCx-HS is the industry’s first objective specification enabling measurement of hardware, OS (operating system) and commercial Apache Hadoop File System API compatible software distributions

 

Benchmarks Under Development

TPC-V

The TPC-V development subcommittee has been making progress in developing an end-to-end reference benchmarking kit, which can now run transactions on multiple databases in multiple VMs on a virtualized server. A beta version of the end-to-end, multi-VM express benchmarking kit is available to TPC members. The subcommittee is working on completing the specification for an Express version of the benchmark.

 

Other TPC Activities

TPC Technology Conference

The TPC remains committed to developing new benchmark standards to keep pace, and one vehicle for achieving this objective is the sponsorship of the Technology Conference on Performance Evaluation and Benchmarking (TPCTC). Over the last five years we have held TPCTC successfully in conjunction with VLDB.

With the TPCTC 2014 conference, in conjuction with with VLDB 2014, the TPC encourages researchers and industry experts to submit novel ideas and methodologies in performance evaluation, measurement, and characterization. The conference will be held on September 5, 2014 in Hangzhou, China.

Topics will include:

  • Big Data Appliances
  • Cloud Computing
  • In-memory databases
  • Social media infrastructure
  • Business intelligence
  • Complex event processing
  • Database optimizations
  • Green computing
  • Disaster tolerance and recovery
  • Energy and space efficiency
  • Hardware innovations
  • Data Integration
  • Hybrid workloads
  • Virtualization
  • Lessons learned in practice using TPC workloads
  • Enhancements to TPC workloads

 

All Benchmark Status Reports
 

Valid XHTML 1.0 Transitional Valid CSS!

  

Valid XHTML 1.0 Transitional Valid CSS!