CppCMS Blog :: Progress

MSVC9 is now supported by CppCMS

MSVC 2008 had successfully compiled CppCMS 1.x.x branch and run several examples. So it looks like that Windows with MSVC would definitely be one of the officially supported platforms. (Note: 0.0.x stable branch would not support Windows besides Cygwin)

The biggest problems I had with MSVC were lack of support of C99 and some POSIX functionality like: snprintf, cstdint or gmtime_r… but with couple of ifdefs it mostly solved.

To be honest, Windows development can be quite unpleasant, especially with lack of basic widely available libraries like zlib. But finally I must admit that it is possible to work with Mircosoft compiler, I could even admit, it could be very fine compiler if it was supporting C99.

Also, more time I try supporting Windows platform, more I understand: gcc is not as polished under Windows as it is under Unix platforms. That makes development of cross platform software even harder.

more...

CppCMS 1.x.x moves to CMake

No, I don’t think that CMake is better then autotools. In fact I still think that CMake is total "crap". It has terrible cache policy, it has broken configuration_file support. It is crappy documentation and many broken configuration tools like CheckTypeSizeOf… and much more.

But it supports MSVC (that I may think supporting in future) and has a better Windows support… So I announce that next version of CppCMS would use CMake (and it already uses in re-factoring branch).

Autotools build system is no longer supported and will be removed from the CppCMS 1.x.x branch, because I do not really like supporting two build systems.

I hope CppCMS users would understand this terrible move.

more...

Introducing Boost.Locale

After a long period of hesitating I had understood – standard C++ locale facets are no-go and started developing localization tools based on ICU that work in C++ friendly way. Thus, Boost.Locale was born. It had just been announced to the Boost community for preliminary review.

Boost.Locale provides case conversion and folding, Unicode normalization, collation, numeric, currency, and date-time formatting, messages formatting, boundary analysis and code-page conversions in C++ aware way.

For example in order to display a currency value it is enough to write this:

cout << as::currency  << 123.34 << endl;

And currency like "$124.34" would be formatted. Spelling a number?

cout << as::spellout << 1024 << endl;

Very simple. And much more. This library would be the base for CppCMS localization support and I hope it would be also accepted in Boost at some point in future.

I've tested this library with:

Linux GCC 4.1, 4.3. with ICU 3.6 and 3.8
Windows MSVC-9 (VC 2008), with ICU 4.2
Windows MingW with ICU 4.2
Windows Cygwin with ICU 3.8

Documentation

Full tutorials: http://cppcms.sourceforge.net/boost_locale/docs/
Doxygen reference: http://cppcms.sourceforge.net/boost_locale/docs/doxy/html/

Source Code

Is available from SVN repository.

svn co https://cppcms.svn.sourceforge.net/svnroot/cppcms/boost_locale/trunk

Building

You need CMake 2.4 and above, ICU 3.6 and above, 4.2 recommended So checkout the code, and run

cmake /path/to/boost_locale/libs/locale
make

Inputs and comments are welcome.

more...

CppCMS meets Comet

One of the major requirements for framework refactoring was support of Comet. Now, with introduction of asynchronous request handling and persistent application servers it becomes reality.

Client Side

There is a HTML source of simple chat client, that uses Dojo toolkit. It does following:

Submits new messages to the server application by posting form using XHR:

function send_data() {
        var kw = {
                url : "/chat/post",
                form : "theform"
        };
        dojo.xhrPost(kw);
        dojo.byId("message").value="";
        return false;
}

Receives new messages from the server using long poll via XHR:

var message_count = 0;
function read_data() {
        dojo.xhrGet( {
                url: "/chat/get/" + message_count,
                timeout: 120000,
                handleAs: "text",
                load: function(response, ioArgs) {
                        dojo.byId("messages").innerHTML =
                                response
                                + '<br/>'
                                + dojo.byId("messages").innerHTML;
                        message_count++;
                        read_data();
                        return response;
                },
                error: function(response,ioArgs) {
                        read_data();
                        return response;
                }

        });
}
dojo.addOnLoad(read_data);

So, the client side is quite simple (however error handling should be quite better).

Server Side

First we create our long running asynchronous application, that receives two kinds for requests: "/post" – with new data, and "/get/NN" – receive message nuber NN, we assign these calls to two member functions post and get.

class chat : public cppcms::application {
public:
    chat(cppcms::service &srv) : cppcms::application(srv)
    {
        dispatcher().assign("^/post$",&chat::post,this);
        dispatcher().assign("^/get/(\\d+)$",&chat::get,this,1);
    }

Now, this class includes two data members:

private:
    std::vector<std::string> messages_;
    std::vector<cppcms::intrusive_ptr<cppcms::http::context> > waiters_;

The history of all chat messages – messages_ and all pending get requests that can’t be satisfied, because the message still not exists – waiters_

Each, "waiter" is actually pointer to request/response context that can be used for message transport.

Now, when new message arrives, post member function is called:

void post()
{
    if(request().request_method()=="POST") {
        if(request().post().find("message")!=request().post().end()) {
            messages_.push_back(request().post().find("message")->second);
            broadcast();
        }
    }
    release_context()->async_complete_response();
}

If the requested message was found, it is added to messages_ list and all waiters are notified using broadcast() member function.

At the end, the current request context is released and completed.

The broadcasting is done as following:

void broadcast()
{
    for(unsigned i=0;i<waiters_.size();i++) {
        waiters_[i]->response().set_plain_text_header();
        waiters_[i]->response().out() << messages_.back();
        waiters_[i]->async_complete_response();
        waiters_[i]=0;
    }
    waiters_.clear();
}

For each pending request the last message is written and the request closed. After that, all pending request are cleaned.

When get request arrives, it is handled by get(std::string no) member function, first of all we check if requested message exists, if so we just return it to user.

unsigned pos=atoi(no.c_str());
if(pos < messages_.size()) {
    response().set_plain_text_header();
    response().out()<<messages_[pos];
    release_context()->async_complete_response();
}

Otherwise, if the requested message is the last one, that does not exists, we add the request context to pending list waiters

else if(pos == messages_.size()) {
    waiters_.push_back(release_context());
}

If requested message it too late – probably client error, we just set status to "404 Not Found" and return the response.

else {
    response().status(404);
    release_context()->async_complete_response();
}

No, all we need to do is to add application to the main running loop under script name "/char" and start the service.

cppcms::service service(argc,argv);
cppcms::intrusive_ptr<chat> app=new chat(service);
service.applications_pool().mount(app,"/chat");
service.run();

Summary

So, the simple chat service was written with about 50 lines of C++ code and about same amount of JavaScript code.

I must admit, that it is too simplistic and not efficient, for example: if new client connects it receives all messages one by one and not as bulk (can be easily fixed), I do not handle timeouts and disconnects. But the general idea is quite clear:

Asynchronous long running application that handles all request is created.
It manages all outstanding request and uses them for server side push.

This is actually a base for future development of tools like XML-RPC and JSON-RPC that allow client to call asynchronously server side objects, it can be used for implementation of any other Comet protocols.

more...

Progress Report on CppCMS v1

Its quite long time that most of the work is done in new refactoring branch… Meanwhile trunk stays silent. So, I decided to open a window and show some new changes:

Dependencies:

I had removed almost all dependencies with a big exception of Boost libraries.

Because of internal structure changed — mostly introduction of asynchronous event handling I could not use existing implementations of FastCGI because of its synchronous API. Also I decided to remove CgiCC that was very problematic in terms of installation, portability and most important the quality of implementation and ability to communicate with its primary developer.

So, at this point you only need latest boost library… Thats all. When the job would be complete it would be very easy to create deb/rpm packages for most popular distributions.
Server APIs:

In addition to supported FastCGI and SCGI protocols, direct HTTP protocol is supported, so you do not need to use external web server for debug purposes any more. It is also useful for embedding web applications.
Localization is now fully integrated with C++ std::locale and allows using correct facets for each supported language and translation
Windows is now would be one of the officially supported platforms.

There is still lot of work to make new version as useful as current CppCMS stable version:

Integrate all template system back.
Integrate cache and sessions management back.
Rewrite forms classes that currently work with CgiCC.
Rewrite support of CGI API for embedded systems.

But there are many good points that are already visible.

more...

What's Next?

The road map of the project includes two important milestones:

CppCMS core components refactoring including following:
- Removal of dependency on CgiCC – today there is about 5% of CgiCC library is used, many features are not supported by it or are not supported well. For example: file upload handling in CgiCC is very primitive, limited and error prone, support of cookies buggy and so on.
- Using of Boost.Asio as internal event handler, because:
  1. It provides transparent synchronous and asynchronous event handling allowing future implementation of server push technologies.
  2. It provides efficient timer based event handling.
- Removal dependency of libfcgi and writing Boost.Asio friendly implementation of FastCGI/SCGI connectors. Implementation of HTTP connectors as well.
- Support of plug-in applications in CppCMS framework.
- Improving compilation speed by representing more pimpl idioms and removal of unnecessary classes.
Better support of i18n and and l10n:
- Transparent support of std::wstring with forms including automatic encoding testing and conversion.
- Support of std::locale for localization for outputs like numbers, dates, monetary, translation and so on.
- Optional support of ICU and icu::UnicodeString and icu::Locale that would add unsupported features by std::locale and allow replacement std::locale features with more correct implementations provided by ICU.

These changes will significantly break API backward compatibility, but it would be possible to adopt the code almost "mechanically" to the new API.

more...

CppCMS 0.0.4 Released

Version 0.0.4 of CppCMS had released.

It includes optimizations required for using it in embedded systems.

Normal Embedded Build:

Caching is completely removed. Small memory footprint is very important for embedded system thus, caching stuff in memory is quite useless.
Zlib compression are removed – it removes dependency on boost::iostreams, zlib and bzip2 libraries.
Removed mod-prefork.
Removed dynamic templates loading — this feature requires export of symbols to binary and increases its size in order to make RTTI work. Thus, all templates should be statically compiled into the binary.

Embedded CGI Mode:

FastCGI and SCGI APIs are removed
Mod-thread and mod process are removed including all thread pool facilities
Changes in files based session backend to work properly with CGI mode including garbage collection (sessions that had time-out).

Downloads are avialable from Sf Project Page.

more...

CppCMS Going Embedded... or I need your help!

Recently I start understanding that embedded market may be very important market for CppCMS. So I did several tweaks to the code in order to support embedded systems:

When the library is compiled for embedded system, caching system is totally removed, removed dependency on boost::iostreams and compression library.
Mod-prefork is removed as much more memory consuming.
Removed dynamic loading of templates.
Option for cgi-only library given — no scgi and and fastcgi APIs are compiled in, all remaining worker "mods" are removed.
Session storage backend is simplified and optimized for cgi when it is compiled in embedded mode.

Everything was cross compiled for arm and tested with qemu-arm, but unfortunately I do not have an access to real ARM hardware with installed Linux system, so…

I need your help:

Download following test_arm_embed.tar.gz
Copy it to your ARM system and extract files.
Try to run cgi script from command line:
```
./hello.dynamic -c config.txt
```
This is "Hello World" that is dynamically linked with libstdc++. If it does not work try
```
./hello.static -c config.txt
```
This is full statically compiled version of "Hello World" cgi script.

You should get as output CGI headers and HTML output.
Please, try to run it several times with:
```
time ./hello.dynamic -c config.txt
```
And tell the average run time.
If you have a web server in your system I'd be very glad if you could test cgi script – please edit it — test.cgi to reflect correct executable and path.

I all works fine I'd be glad (if it is possible) to prepare a set of scripts that would test various aspects of cppcms as templates, various session backends and so on.

Thanks

more...

Release Announcement - CppCMS 0.0.3

CppCMS 0.0.3 Released

Changelog:

Forms API cleanup.
Fixed configuration with latest autoconf/automake
gcc 4.3 compilation fixes
Added an option to disable gzip compression
Added an option for manual output stream control
Fixed incorrect use of readdir_r with dirent on Solaris that caused crash
Changed cppcms_make_key now written in C instead of unportable bash/od
Fixed – widgets are default constructable
Added help to templates compiler, BSD bugfix
Now non-default session timeout type/limit is preserved withing session.
Added "notriggers" to store cache functions as well
Fixed incorrect triggers behavior in cache_iface.

DbiXX 0.0.3 Released

Changelog:

Fixed compilation issues with gcc-4,3

WikiPP 0.0.2 Released

Changelog:

Updates according to API changes in CppCMS 0.0.3

Examples 0.0.3 Released

No changes, but package location on sourceforge release system had changed

Documentation

User Documentation is complete. It covers most of aspects required for developing with CppCMS.

Still, documentation of internals is missing.

more...

CppCMS 0.0.2 Released!

After long period of development CppCMS 0.0.2 was released!

Downloads
Documentation — (not 100% complete yet).
Installation Instructions

It includes following components:

CppCMS Web Development Framework.
DbiXX Database independent SQL library.
WikiPP – CppCMS powered wiki engine.
Examples – a set of various examples of using CppCMS.

Policy Changes

There are important release policy changes:

Beta versions are available to public via svn on trunk branch.
The frequency of releases is expected to grow, instead of long development periods inside trunk.
Most of CppCMS architecture is stabilized, however API is still subject to change.
No full backward compatibility for future releases is granted, however, preserving backward compatibility and stabilizing API are important goals of future development.

Major Changes

New, statically typed template system was introduced. It is based on compilation of templates to native, dynamically loadable shared objects.
Old, dynamic typed template system had deprecated and not supported any more
Various enhancements to template system where introduced like:
1. Powerful "Django style" templates inheritance
2. Extending templates using direct injection of C++ code to templates.
3. Templates are integral part of CppCMS project and not additional library.
Introduced form processing and validation sub-framework deeply integrated with templates system.
Introduced powerful, scalable session management system.
Improved cache API for better supporting non-trigger based caching.
Documentation had moved to CppCMS powered Wiki – WikiPP
FreeBSD and OpenSolaris are supported in addition to Linux and Cygwin.
CppCMS powered Wiki software – WikiPP was released under GPLv3.

more...