Thirteenth TPC Technology Conference on Performance Evaluation & Benchmarking
(TPCTC 2021)

in conjunction with




Conference Program

August 20th, 2021




All times are local times for
Copenhagen, Denmark


Start Time End Time Paper Information



Local chair for session 1 (8:00 AM - 10:30 AM): Prof. Tilmann Rabl
08:00 AM 08:30 AM Opening Remarks and Welcome.
Raghunath Nambiar.
08:30 AM 09:00 AM CH2: A Hybrid Operational/Analytical Processing Benchmark for NoSQL:
Michael Carey, M. Muralikrishna, Dmitry Lychagin, Vijay Sarathy and Till Westmann.
09:00 AM 09:30 AM Tell-tale tails: Pitfalls and Perils in Database Benchmarking:
Michael Fruth, Stefanie Scherzinger, Wolfgang Mauerer and Ralf Ramsauer.
09:30 AM 10:00 AM IoTDataBench: Extending TPCx-IoT for Compression and Scalability:
Yuqing Zhu, Yanzhe An, Yuan Zi, Yu Feng and Jianmin Wang.
10:00 AM 10:30 AM A Survey On Big Data, HPC and Machine Learning Benchmarks:
Nina Ihde, Paula Marten, Ahmed Eleliemy, Gabrielle Poerwawinata, Pedro Silva, Florina Ciorba and Tilmann Rabl.



10:30 AM 11:00 AM Break



Local chair for session 2 (11:00 AM - 12:30 PM): Rafael Moczalla
11:00 AM 11:30 AM EvoBench: Benchmarking Schema Evolution in NoSQL:
André Conrad, Mark Lukas Möller, Tobias Kreiter, Jan-Christopher Mair, Meike Klettke and Uta Störl.
11:30 AM 12:00 Noon Quantifying Cloud Data Analytic Platform Scalability with Extended TPC-DS Benchmark:
Guoheng Chen, Timothy Johnson and Miso Cilimdzic.
12:00 Noon 12:30 PM A YCSB Workload for Benchmarking Hotspot Object Behaviour in NoSQL Databases:
Casper Claesen, Ansar Rafique, Dimitri Van Landuyt and Wouter Joosen.



12:30 PM 1:30 PM Lunch Break.



Local chair for session 3 (2:30 PM - 4:00 PM): Ilin Tolovski
1:30 PM 2:30 PM Invited Talk.
Benchmarking Cloud-Native Database Systems in the Cloud Era:

Feifei Li
2:30 PM 3:00 PM Everyone is a Winner: Interpreting MLPerf Inference Benchmark Results:
Miroslav Hodak, David Ellison and Ajay Dholakia.
3:00 PM 3:30 PM Orchestrating DBMS Benchmarking in the Cloud with Kubernetes:
Patrick Erdelt.
3:30 PM 3:45 PM Upcoming Artificial Intelligence Benchmark, TPCx-AI:
Hamesh Patel.
3:45 PM 4:00 PM Closing Remarks
Meikel Poess.




Call For Papers

The Transaction Processing Performance Council (TPC) is a non-profit organization established in August 1988. Over the past two decades, the TPC has had a significant impact on the computing industry’s use of industry-standard benchmarks. Vendors use TPC benchmarks to illustrate performance competitiveness for their existing products, and to improve and monitor the performance of their products under development. Many buyers use TPC benchmark results as points of comparison when purchasing new computing systems. The information technology landscape is evolving at a rapid pace, challenging industry experts and researchers to develop innovative techniques for evaluation, measurement and characterization of complex systems. The TPC remains committed to developing new benchmark standards to keep pace, and one vehicle for achieving this objective is the sponsorship of the Technology Conference on Performance Evaluation and Benchmarking (TPCTC). Over the last nine years we have held TPCTC successfully in conjunction with VLDB.


With the thirteenth TPC Technology Conference on Performance Evaluation and Benchmarking (TPCTC 2021) proposal, we strive to excel the success of previous workshops by encouraging researchers and industry experts to present and debate novel ideas and methodologies in performance evaluation and benchmarking for emerging technology areas. Authors are invited to submit original, unpublished papers that are not currently under review for any other conference or journal. We also encourage the submission of extended abstracts, position statement papers and lessons learned in practice. The accepted papers will be published in the workshop proceedings, and selected papers will be considered for future TPC benchmark developments.
 

Topics of interest include, but are not limited to:

  • Blockchain
  • Big Data and Analytics
  • Complex event processing
  • Database Optimizations
  • Data Integration
  • Artificial Intelligence
  • Emerging storage technologies (NVMe, 3D XPoint Memory etc)
  • Hybrid workloads
  • Energy and space efficiency
  • In-memory databases
  • Internet of Things
  • Virtualization
  • Disaster tolerance and recovery
  • Enhancements to TPC workloads
  • Lessons learned in practice using TPC workloads
  • Collection and interpretation of performance data in public cloud environments

Submission Guidelines

Authors are invited to submit original, unpublished papers that are not currently under review for any other conference or journal. We also encourage the submission of extended abstracts, position statement papers and lessons learned in practice. The length of a paper should not exceed 16 pages. Papers should follow the LNCS format. The title page must contain a short abstract. All papers should be submitted electronically in PDF format to the review web site at: Easychair

Important Dates

Abstract due: June 14th, 2021
Papers due: June 21st, 2021
Notification of acceptance: July 10th, 2021
Camera-ready copies: July 20th, 2021
Conference day: August 20th, 2021                 


Conference Venue and Registration
Please visit the VLDB2021 conference web site at: http://vldb.org/2021

Proceedings
Proceedings will be published by Springer-Verlag as Lecture Notes in Computer Science (LNCS). Selected papers may be considered for future TPC benchmark developments.

LNCS logo




TPCTC 2021 Organization

General Chairs and Contacts
Raghunath Nambiar, AMD, USA, raghu.nambiar@amd.com
Meikel Poess, Oracle, USA, meikel.poess@oracle.com

Program Committee
Dippy Aggarwal, Intel, USA
Daniel Bowers, Gartner, USA
Michael Brey, Oracle, USA
Ajay Dholakia, Lenovo, USA
Dhabaleswar Panda, The Ohio State University, USA
Tilmann Rabl, TU Berlin, Germany
Swapna Raj, Intel, USA
Anil Rajput, AMD, USA
Reza Taheri, VMWare, USA

Publicity Committee
Meikel Poess, Oracle, USA
Andrew Bond, Red Hat, USA
Paul Cao, HPE, USA
Gary Little, Nutanix, USA
Raghunath Nambiar, AMD, USA
Reza Taheri, VMware, USA

Michael Majdalany, L&M Management Group, USA
Forrest Carman, Owen Media, USA
Andreas Hotea, Hotea Solutions, USA



About the TPC
The Transaction Processing Performance Council (TPC) is a non-profit organization that defines transaction processing and database benchmarks and distributes vendor-neutral performance data to the industry. Additional information is available at: tpc.org.