OOPSLA 2004 Workshop: Component and Middleware Performance.
Deadline September 2, 2004 - Conference October 24-28, 2004, Vancouver, canada
The goal of the workshop is to help advance the current practice of gathering performance characteristics of component systems and middleware implementations through benchmarking. Positions are solicited especially from people with previous or impending benchmarking experience.
July 16th 2004: 2nd ObjectWeb Benchmarking Workshop
A workshop on benchmarking was held in Prague, Czech Republic on july 7th 2004.
The slides of the presentations given during the workshop are available.
June 15th 2004: RUBBoS Servlets v1.2 now available.
Version 1.2 of the Java Servlets implementation of RUBBoS, the bulletin board benchmark, is now available
on the JMOB website. Thanks to Niraj Tolia this version fixes the folowing bugs:
- Fixed connection pooling
- Use of the executeUpdate() method instead of executeQuery() to execute updates
- Fixed bug in ModerateComments that tried to load servlet/classname instead of rubbos/servlet/classname
- Added two indices missing from the comments and old_comments table
- Fixed a couple of grammatical errors in the client
Download RUBBoS Servlets version.
April 14th 2004: Evaluation of JGroups performance and scalability.
An evaluation of JGroups performance and scalability in the context of clustered
J2EE application servers is available. JGroups configuration used by popular software such as the Tomcat JSP server
or JBoss J2EE server are evaluated with different network technologies, protocol stacks and cluster sizes.
Have a look at the results.
JMOB: Java Middleware Open Benchmarking
The main objectives of JMOB are:
- to make available open-source middleware benchmarks such as
- to share benchmarks and results on ObjectWeb middleware, and
in particular on application servers (in a first step on JOnAS), including
- to exchange experience on
client load generation, and to contribute a distributed client
- to investigate impact of hardware configurations on
performances, and in particular of new architectures (in a first
Discussions around the JMOB project are supported by the JMOB mailing
RUBiS (Rice University Bidding System)
is an auction site prototype modeled after eBay.com. RUBiS is
currently used to evaluate design patterns, application
servers and communication layers scalability.
Reports and articles about RUBiS are available on the RUBiS
Web site Visit also the RUBiS's
ObjectWeb Forge Web site for the latest code
CLIF (CLIF is a Load Injection
Framework) is a starting development activity aiming at
providing a generic load injection platform for
More information can be found on the CLIF
Stock-Online is the CSIRO application used to
stress middleware performance and scalability. It is a
simulation of a simple on-line stockbroking system. It
enables subscribers to buy and sell stock, inquire about
the up-to-date prices of particular stocks, and get a
holding statement detailing the stocks they currently
own. It was designed to be implemented in multiple
technologies to provide a rapid means for evaluating and
comparing different distributed infrastructures and
products. The workload is intended to be representative of
a "typical" web based application (80% read transactions),
and stress the middle-tier rather than the database.
The open-source version is implemented according to the
Enterprise Java Beans (EJB) 1.1 specification, and has
been used to evaluate in excess of six J2EE 1.2
Application Servers as part of the CSIRO
Middleware Technology Evaluation (MTE) project. It
uses a 3-tier architecture consisting of a Java (or J2EE)
client, EJB middle-tier, and Oracle database. Two
different EJB design approaches are provided: Stateless
Session Beans with JDBC, and Stateless Session Beans with
Container Managed Persistence (CMP) Entity Beans.
Visit the Stock-Online's
ObjectWeb Forge Web site to checkout the source
RUBBoS is a bulletin board benchmark modeled after an online news forum. As RUBiS,
RUBBoS was designed to to evaluate design patterns, application
servers and communication layers scalability. We provide a PHP version of RUBBoS.
More information can be found on the RUBBoS
TPC-W is a standard benchmark that models an online bookstore. We just
provide a Java servlets implementation based on the version from the University of Wisconsin that works with MySQL as database.
Download this implementation here.
JVM benchmarks: IBM, BEA and Sun JVM
are evaluated using various benchmarks such as Volano,
RUBiS or SPECjbb2000. Available
CORBA benchmarks: benchmarking suites for CORBA and results are available from Charles University Distributed System Group's web page.