CppCMS Blog :: Benchmarks

Asp.Mono in Linux? Not Yet...

During development of CppCMS I have always wanted to do some benchmarks against one of the most popular web development technologies: Asp.Net. I had found a blogging system: BlogEngine.Net that in its latest version had full support of mono under Linux, support of MySQL — all I need to run benchmarks against CppCMS.

The beginning was promising. The instructions were simple, there was an actual blog running BE.Net under Apache mod_mono on Ubuntu. There were Linux specific instructions as well.

So, first of all I had installed mono 1.9.1 from Etch backports. The first problem I had to deal with was an installation problem — the version of C# compiler and mono environment were different. This was solved quite simply. So, I could finally see BE.Net running under Mono using XML as data storage backend.

Then, I wanted to add MySQL storage backend. The problems had come very soon. The MySQL database script was written under assumption that table names are case insensitive. That was not true for MySQL under Linux. This problem was fixed.

more...

CppCMS vs WordPress

Setup

I had compared two blog systems: this one and WordPress 2.5 with a patched WP-Cache-2 addon. I used following configuration:

Web Server lighttpd 1.4.13
Interface FastCGI
PHP 5.2
Bytecode cacher: XCache 1.2.1
Database MySQL 5.0
Caching for WP: WP-Cache-2 with an additional performance patch
Hardware: AMD Athlon XP 64bit, 1G RAM
OS: Linux, Debian Etch 64bit.

I prepared two blogs that were filled up with 1000 articles each. Each article had 10 comments, all the articles were organized in 10 categories in each blog.

more...

Patch For WP-Cache-2 plugin

I'm going to run a heavy benchmarks comparing WordPress – the blog system I know very well, with CppCMS based blog – the system I had written.

The new caching system that was developed for CppCMS is quite smart, it stores the entry pages twice: original and gzip compressed. On heavy loads, this allows serving pages significantly faster because only thing that should be done is to push html or compressed html page directly from the cache. Otherwise, gzip compression (even fastest) would take lots of resources and reduces a preformace of the system.

When it comes to benchmarks, I had discovered that WP-Cache-2 plugin does the job well, but it caches only html version of the file, thus, even if the page is cached it still must pass a compression by Apache’s mod_deflate or by PHP engine itself.

I had patched this plugin and now it stores two versions of same page: an original and compressed. and was able to get 60% performace improvement.

WordPress native plugin: 450 requests per second
WordPress patched plugin: 720 requests per second

So after this patch I can feel that the benchmarks would be proper, because without it this would be incorrect to compare time required for fetching a cache with the time required for compressing entry page.

Links:

N.B.: The full benchmarks coming soon

more...

New Templates System

New templates system was introduces to the CppCMS framework. It is based on ideas of dynamic typed languages inside static typed C++.

The original template system had several problems:

The each template variable was referenced by and integer key that was generated during compilation of templates.
The rendering process required from the developer some kind of activity – update content values according to requests from rendering engine.
The values of the entries where limited to string, integer and boolean values.

In any case, the design of the first template system was just unacceptable, thus new template system was build.

It introduced following features:

Dynamic typed variable values using boost::any. They allow assigning of any kind of objects to the variables and rendering them to the templates using custom engines.
All the variables are references by their names.
Content now has hierarchical structure when each variable can include list of items or callbacks that allow one step template rendering.
The design of the engine is now much more modular.

Additional features I'm still working on them are:

Support of different filters like "html escaping", "urlizing" etc.
Support of custom filters, including filter chains.
Support of localization and translation.

more...

SOCI or DBI, stage 2

Discussing soci performance with its developers I had found that soci is compiled without any optimizations by default.

So, after recompiling soci with -O2 option I've got much better results. Simple comparison of dbixx and soci had given very close result. I had run my tests once more and got following results – pages per second (no gzip):

DB        soci dbixx
--------------------
MySQL      710   800
Sqlite3    550   410
PgSQL      385   430

We can clearly see that for MySQL and PostgreSQL dbixx is still faster in about 10% however in case of Sqlite3 dbixx is significantly slower (25%).

So it seems to me that both solutions are quite reasonable to use without clear advantage of one over another.

more...

SOCI or DBI

One of the problematic issues in writing cross-SQL code is an API that differs from one SQL to another.

There are two open source libraries that provide unified API

soci – C++ library.
dbi – C library.

At the first point I had chosen soci as native solution of C++ programmer. After running some benchmarks on the new version of this blog I had found 20% performance reduction for MySQL database. But I also remembered that there should be negligible difference between MySQL and Berkeley DB. This was mostly due to incorrect design of my BDB database layout I had done.

That had seem to be strange and I stared benchmarking the system more and more.

more...

The Cost of SQL - First Measurements

There are the preliminary benchmarks comparing different DB backends. I've measured pages per second ratio for fetching different pages from the blog that includes 5,000 articles and 50,000 comments to them. The client and the server had run on the same PC.

The settings and the database is the same that is used in this article.

Backend         gzip no-gzip   gzip   no-gzip %
------------------------------------------------
Berkeley DB     565  830       N/A    N/A
MySQL InnoDB    475  645       -16%   -22%
Sqlite3         410  515       -27%   -38%
PostgreSQL      305  360       -46%   -57%

We can see:

There is not negligible cost of using SQL Data Bases. However the price is not too high with fast data bases like MySQL.
PostgreSQL had badly surprised me with its real performance. Maybe I'm doing something wrong?

more...

BDB or not BDB - that is the question...

At the beginning of the project I had several options for president storage for CppCMS:

Use standard SQL databases like MySQL, PostgreSQL.
Use embedded SQL data base like Sqlite3
Use Berkeley DB
Implement my own storage model.

At the beginning I wanted to use MySQL. At certain point I decided to switch to Berkeley DB, and now I come back to the original question: "what data storage to use?"

In order to make a proper design I had run lots of micro benchmarks and finally I had chosen Berkeley DB. However I never used queries from real DB. And now, when I had implemented a simple blog I decided to run benchmarks on the situation that is closer to real one.

I had reimplemented an operation of fetching single post using MySQL and imported the database that included 5000 articles with 10 comments each one and tested.

more...

Benchmarks, First Results

Updated 28/12/07, 17:00

In order to show that all the time I had spend on this system wasn’t wasted I had run a simple benchmark tests on following setup:

Server

Hardware: AMD Athlon XP 3000+ 64 bit, 1GB RAM.
OS: Debian Etch 64 bit.
Web server: Lighttpd 1.4.13
CMS: This blog software, configured with 5 working threads.
Data base of 85MB: 5,000 articles with 10 comments to each one.

Network devices

100Mbit network cards
Edimax $75 home router

Client

Hardware: Dell Latitude C400, PIII 750MHz, 256MB RAM
OS: Debian Etch 32 bit

more...