| Eighteenth TPC Technology Conference on Performance Evaluation & Benchmarking (TPCTC 2026) |
|||||||||||||||||
| August 31, 2026 |
|||||||||||||||||
| in conjunction with VLDB 2026 |
|||||||||||||||||
 |
|||||||||||||||||
Conference Program |
|||||||||||||||||
August 31, 2026 |
|||||||||||||||||
|
Abstract: AI systems are rapidly evolving from single-turn generative models into closed-loop agents that plan, invoke tools, coordinate with other agents, maintain state, and operate within simulated or learned environments. This shift challenges conventional benchmarking methods that focus on isolated prompts, model accuracy, latency, or throughput. For agentic systems, the natural unit of evaluation is no longer a query, but a trajectory: the end-to-end execution path of a task, including model calls, tool use, memory updates, inter-agent communication, retries, artifacts, costs, and final outcomes. This panel will examine how performance evaluation must evolve for such systems, with emphasis on workload design, trace-based measurement, token economics, run-to-run variance, reliability, reproducibility, and energy-aware cost-performance. The discussion will also consider the emerging role of world models, both as benchmark targets and as controlled environments for evaluating autonomous agents. Panelists from academia and industry will debate what metrics, reporting standards, and audit mechanisms are needed for fair comparison of closed-loop AI systems. The goal is to identify practical directions toward a TPC-style benchmarking framework, including the possibility of a standardized agent transaction log for reproducible evaluation. |
||||||||||||||||
Panelist: | |||||||||||||||||
|
|||||||||||||||||
Call For Papers |
|||||||||||||||||
| The Transaction Processing Performance Council (TPC) is a non-profit organization established in August 1988. Over the past two decades, the TPC has had a significant impact on the computing industry’s use of industry-standard benchmarks. Vendors use TPC benchmarks to illustrate performance competitiveness for their existing products, and to improve and monitor the performance of their products under development. Many buyers use TPC benchmark results as points of comparison when purchasing new computing systems. The information technology landscape is evolving at a rapid pace, challenging industry experts and researchers to develop innovative techniques for evaluation, measurement and characterization of complex systems. The TPC remains committed to developing new benchmark standards to keep pace, and one vehicle for achieving this objective is the sponsorship of the Technology Conference on Performance Evaluation and Benchmarking (TPCTC). Over the last seventeen years we have held TPCTC successfully in conjunction with VLDB. | |||||||||||||||||
With the eighteenth TPC Technology Conference on Performance Evaluation and Benchmarking (TPCTC 2026) proposal, we strive to excel the success of previous workshops by encouraging researchers and industry experts to present and debate novel ideas and methodologies in performance evaluation and benchmarking for emerging technology areas. Authors are invited to submit original, unpublished papers that are not currently under review for any other conference or journal. We also encourage the submission of extended abstracts, position statement papers and lessons learned in practice. The accepted papers will be published in the workshop proceedings, and selected papers will be considered for future TPC benchmark developments. |
|||||||||||||||||
Topics of interest include, but are not limited to: |
|||||||||||||||||
|
|||||||||||||||||
Submission Guidelines |
|||||||||||||||||
| Authors are invited to submit original, unpublished papers that are not currently under review for any other conference or journal. We also encourage the submission of extended abstracts, position statement papers, and lessons learned in practice.
The length of a paper should not exceed 16 pages. Papers should follow Springer's Formatting Guidelines for LNCS All papers should be submitted electronically in PDF format to: Easychair (TPCTC26) |
|||||||||||||||||
|
|||||||||||||||||
|
|||||||||||||||||
| Conference Venue and Registration | |||||||||||||||||
| Please visit the VLDB2026 conference web site at: http://vldb.org/2026 |
|||||||||||||||||
| Proceedings | |||||||||||||||||
| Proceedings will be published by Springer-Verlag as Lecture Notes in Computer Science (LNCS). Selected papers may be considered for future TPC benchmark developments. | |||||||||||||||||
 |
|||||||||||||||||
TPCTC 2026 Organization (not finalized yet) |
|||||||||||||||||
|
General Chairs and Contacts Raghunath Nambiar, AMD, USA, raghu.nambiar@amd.com Meikel Poess, Oracle, USA, meikel.poess@oracle.com Program Committee Andrew Bond, redhat, USA Michael Brey, Oracle, USA Paul Cao, HPE, USA Ajay Dholakia, Lenovo, USA Rodrigo Escobar, Univ. Texas at San Antonio, USA Ahmad Ghazal, SingleStore, USA Shahram Ghandeharizadeh, University of Southern California, USA Miro Hodak, AMD, USA Swathi Kovvuri, Intel Corporation, USA Klaus-Dieter Lange, Hewlett Packard Enterprise, USA Hanumath Maduri, Workday, USA Raghunath Nambiar, AMD, USA Meikel Poess, Oracle Corporation, USA Tilmann Rabl, Hasso Plattner Institute, Germany Steve Shaw, HammerDB, United Kingdom Pengcheng Zheng, Timecho Europe GmbH, Germanyy Publicity Committee Meikel Poess, Oracle, USA Paul Cao, HPE, USA Rodrigo Escobar, Intel, USA Gary Little, Nutanix, USA Nirmala Sundararajan, Dell, USA Michael Majdalany, SBIMS, USA Forrest Carman, Owen Media, USA Andreas Hotea, Hotea Solutions, USA |
|||||||||||||||||
About the TPC |
|||||||||||||||||
| The Transaction Processing Performance Council (TPC) is a non-profit organization that defines transaction processing and database benchmarks and distributes vendor-neutral performance data to the industry. Additional information is available at: tpc.org. | |||||||||||||||||
