After reading How fast is your framework I was rather intrigued to compare CakePHP, CodeIgniter (Also added Symfony). As both frameworks are very similar and one of the best comparisons I’ve heard is “Cake is like a strict teacher while CodeIgniter is your favorite sports teacher” but I can’t remember where I found it. For me this sums the two very similar frameworks in that Cake has very strict naming conventions etc while CodeIgniter allows you more leeway. This is a particular advantage if your using databases that you do not control or have permission to modify etc. Anyway back to the subject of this post the bench mark of both these frameworks.
Update (11 Dec 2006):
After some kind pointers in the right direction from Larry E. Masters (aka PhpNut) and Nate from CakePhp.org to help showing me small modification and oversights in my testing procedures, I was happy to correct. (Thanks for the help guys). You will notice on line 07 of the helloworld controller the addition of var $helpers = null; and also some modifications to the app/config/code.php setting the AutoSession value to false greatly improved the performance. After some discussion the results of this quick benchmark aren’t really in a real world environment and shortly a more complete set of benchmarks for a simple app will be added. These results will not only be more useful but also be able to test the more advanced features of each framework. The change to production mode also cleared up the failed requests
System:
- System: 3.0GHz Intel 512 Ram
- Os: Ubuntu Edgy
- Webserver: Apache 2.0
- PHP 5.16
Framework:
- CakePHP 1.1.11.4064
- CodeIgniter 1.5.1
- Symfony 1.0 beta1
Test:
Each framework is required to have the output “HelloWorld!” produced from a view and a controllers will obviously have to be created. The code I have used is shown below along with the benchmarking results. There are no caching available logging and no databases are connected. I’ve used ApacheBench to test and have also shown the results with the call for 10 concurrent users and testing for 60 secs.
ab -c 10 -t 60 http://framework/helloworld
CakePHP
Controller: helloworld_controller.php
02. class HelloWorldController extends AppController {
03. var $layout = null;
04. var $autoLayout = false;
05. var $uses = array();
06. var $helpers = null;
07. function index()
08. {
09. }
10. }
CodeIgniter
Controller: helloworld.php
02. class Benchmark extends Controller {
03. function Index()
04. {
05. $this->load->view('benchmark_view');
06. }
07. }
Symfony
Controller: action.class.php
02. class mymoduleActions extends sfActions
03. {
04.
05. public function executeIndex()
06. {
07. }
08. }
Also for Symfony the layout.php was modified to contains
< ?php echo $sf_data->getRaw('sf_content') ?>'
as it was adding a large amount of HTML content.The views are identical just containing ‘Hello World!’
Results:
I have moved the results to another page to aid reading because as I add each framework it’s starting to get quite long. For the full results: Web Framework Benchmarking Results
Requests per second for each framework:
CodeIgniter: 58.51 CakePHP: 37.4629.67Symfony: 22.78
So the results leave no doubt that CodeIgniter is much faster than CakePHP some concern must be present about CakePHP in that 213 requests failed. Symfony comes last but as it’s still in beta maybe this will improve, when it’s a stable release I will run the benchmark again.
If you would like some other frameworks tested leave a comment and I will try to add them later. I will try to add Djanjo later this week then and RoR.
December 8, 2006
Very interesting. Thanks for taking the time. I think it’s important to run repeated test and show the results then. Sometime it’s the TREND that important instead of the BEHAVIOUR.
Just to clarify, I am a CodeIgniter developer and have nevr developed with Cake.
It would be interesting to see these tests under load. For example when they are both connected to a database and loading models and libraries. The trick might be to develop a app that’s almost equal, perhaps hello world to all client’s in a database table.
December 8, 2006
very interesting. I’m a fan of both CI and Cake, but for reasons I mention in a certain comparison thread, I never trust packages out of the box - I reckon you need to know what tools you’re using. Taking a look under the bonnet of both CI and Cake shows why the CI is much slower - its much heavier than CI and does alot more between the initial parsing of the URI to the point the script terminates. CI relies on plugins for everything. Take away the plugins (libraries, helpers ,etc), and you’re left with a VERY empty framework. Cake includes much of this functionality whether you use it or not. I love Cake, but I do kinda hope that CI stays on its own path and doesnt become as bloated over time. The plugin architecture works much better, IMO.
As for the ‘headmaster vs sports teacher’ analogy, i posted it here in the Cake vs CI debate:
http://www.codeigniter.com/forums/viewthread/750/P45/ (poster: redbullmarky).
Good article+good insite
Cheers
Mark
December 8, 2006
very interesting.. but it’s not enough to me to change framework :)
I use cake, and I haven’t time to study another framework.
for the brilliant analogy, I think that sometimes it’s better strict conventions, for examples for multi developer or for reusing code. but it’s only my opinion.
December 8, 2006
The question is what is happening if some libraries are loaded and you are working with the DB. Actually you want to use CI libs of course. Would be interesting to develop a simple application that does the same on both frameworks.
December 8, 2006
Good article. I’d be interested to see how Qcodo compared to Cake and CI.
December 8, 2006
aleinside: I agree with your point that the strictness of Cake is a a major plus for team development but when connecting to DBs which are not modifiable for example it can be a negative. Both frameworks are excellent examples of RAD frameworks using the MVC model.
John & Pradesh: Yes this simple ‘HelloWorld’ test does not really prove that much, only how fast the views can be loaded. If I have the time I will extend the tests further to include a simple model, view and controller with very similar code as this will provide even more insight.
Edward: If I get time I’d add Qcodo as well.
December 11, 2006
The reason requests are failing in Cake is because you’re running it in debug mode. Go into app/config/core.php and set DEBUG to 0, then you’ll be running in production.
December 11, 2006
I would be interested to know if the test you performed using these other frameworks have sessions enabled by default, and what other “helpers” are loaded.
While your test looks impressive there are many things behind the scenes in CakePHP that is automatic.
The code you used to test CakePHP is creating a session file on each request unless you turned them off, and the HtmlHelper is also loaded by default.
In config/core.php set DEBUG to 0 as nate suggested this will stop the failed request, then in the same file you will find around line 137 define(’AUTO_SESSION’, true); set this to false.
And since you do not need to load the HtmlHelper add var $helpers = null; to your HelloWorldController
Could you post the results of this after you have made these changes to the CakePHP version of your test?
If I can find time, I may put together a real world example that people can convert to other frameworks and use for benchmarking.
Hello World! is far from a real test, since you are not using most things that you would in a true application (sessions, database connections, etc.)
December 11, 2006
Luci3n: Could you explain this comment? “…but when connecting to DBs which are not modifiable for example it can be a negative”
CakePHP has no problem working with legacy databases. CakePHP promotes conventions over configuration, that does not mean it will not work if you do not follow conventions.
December 18, 2006
did you use eAccelerator? cake’s performance increases a lot (something around factor 3). all the overhead produced by the framework itself (mostly includes and defines) is stepping in the background. use a profiler like apd to go a little deeper. this is really interesting.
i haven’t done this with codeigniter. maybe this is something for the holidays.
December 25, 2006
Glupiy benchmark, luche bi v na peregruzkah testirovali. Avtor baran.
January 12, 2007
I would like to point a thing.
Not only cakephp is slower, it’s slower to understand too.
Code igniter is not only faster, but you read the (great) manual(15 minutes) and you are ready to go.
January 14, 2007
I really think without APC/EAccelerator and turning sessions of, this test is quite useless…
CakePHP has a way more includes and APC will simply get rid of all IO that overhead. And, writing/reading a file for every request just makes it IO bound, hence you are not really measuring framework performance but rather dealing with IO bottlenecks.
January 15, 2007
cake is a dog its that simple… just amazing how you guys get ya back up when someone says something about cake and you all coming running to its defence.
take the time and test yourself and you will see the codeigniter is truely the best framework currently available.
ive tested and used them all and ALL my projects now use codeigniter PERIOD!
January 30, 2007
Larry… what you say may be true, but CodeIgniter was intentionally designed to use optional library/plugin/helper resources either globally across an entire application via an autoload.php configuration file, on call separately within indivdual controllers, or by using a combination of the two approaches. CodeIgniter does automatically load a number of core libraries required for basic framework operation.
In effect, it uses an approach that could be compared somewhat to the dynamic link libraries used in a C++ desktop application–where the DLLs are dynamically loaded as required by the C++ application. If you only use the libraries you need to load at a specific point in time when your CI application controllers require them, application memory requirements become more controllable and application performance can be optimized to a finer degree.
February 6, 2007
Having used both - I would rather operate inside the CakePHP framework where when I ask a question on a forum or google group I actually get a useable answer from someone who knows what they are doing.
As far as speed goes, testing frameworks with a hello world app is silly. If you strip out ( from all the frameworks ) all the overhead except what is *needed* to display hello world ( see http://www.sellersrank.com/php/cakephp-codeigniter-benchmark/#comment-36 ) you will find that cake is comparable. Not that such a test measures anything useful. Do a more full featured ( Read CRUD ) for one table and one related table and then check benchmarks.
February 7, 2007
@abba the results of the benchmarks have been updated according to Larry’s suggestions and the updated results published after removing various unused overhead (see update paragraph). But I do agree that this is a in accurate assessment of the frameworks speed and additional usage such as db calls, sessions etc would be far more interesting.
February 27, 2007
Waiting for RoR results
April 19, 2007
I would really love to see is a database based benchmark. The rationale is that the database component is just as key to a framework. The goal is to try to find out how much time requests get “stuck” in the framework especially if you’re using ORM.
Thanks! And thanks for your work and contribution:)
April 22, 2007
I really liked CodeIgniter a lot… Just found out about CakePHP and will give it a try, the first thing that seems interesting to me is that CakePHP would support Firebird (adodb-drivername says in the manual I guess it should work) and Codeigniter not yet.. For me it´s a must so I´ll try Cake and see how it goes.
May 15, 2007
I start to develop applications with CI like a week ago but CakePHP was always on my mind so your test is really helped me that i am in the right track.
Thanks
May 18, 2007
after playing around with CI, I decided to switch to Cake, because it’s a mature framework and more comfortable. And in my opinion, CakePHP got the better documentations through it’s good 15min blog-tutorial, which covers almost all basic functions while the CI Tutorial scratches only the surface.
The performance issue causes in the amount of features delivered with the framework, because CI lacks basic features like authentification, automatic (HTML-)table- or a convenient form-creation.
Don’t get me wrong: CI is a promising framework, but needs further development.
May 18, 2007
markus: I think your right that CI needs further development it seems to have been stagnant for quite a period now with the only updates being maintenance releases for quite some time. This problem seems to be that although CI is open source the devlopment is controlled by ellis labs and they are currently focusing on expression engine before any new features are added to CI. CI needs some features such as official authentication and Ajax integration are missing and while these are not part of the official distribution will turn many developers away although in CI Forge there are many unofficial project to resolve these problems.
In the next few weeks I will be re-running the benchmarks with the latest release builds with a few more php frameworks to compare.
May 25, 2007
I have a few questions regarding the Symfony benchmark:
1. Did you turn all default helpers off as you did with cake (settings.yml)?
2. Did you turn the layout off as you did with cake (view.yml)?
3. Did you use the production environment?
In general: Are you planning any tests where the model layer is used? After all it’s called MVC-framework, not VC-Framework ;)and the model layer is, where I see the biggest challenge for these frameworks.
May 28, 2007
I totally agree with Andreas ;)
I’d love to see how the ORM compares between cake and symfony. Does CI have an ORM?
I’d also like to see the developers of the other frameworks getting involved to verify the config’s such as Nate and Larry. This should put everyone at ease as well as on the same playing field!
June 1, 2007
@Andreas Next week i hope to have some benchmarks for using the model layer as I agree this would be more useful. I will also make the code used available to ensure that any errors I have made can be identified and are open to scrutinty by everyone. I
@Beth I will be rerunning the benchmarks with the three frameworks and also some other frameworks I will try to contact the principal developers before I post these results so they can examine the results and advise on any oversights I have made. This may delay the posting somewhat but it’s better to be late and more accurate than have any bad results and then have many corrections.
June 6, 2007
Hi Luci3n,
I really appreciate you taking the time in developing and running these tests ;)
I’m kinda of old school when it comes to writing SQL and haven’t totally bought into ORM. Well I buy into the objective but as with anything else it comes down to the actual implementation. Having to learn a new “language” aka ORM to find out I’m losing out on performance for database independence would tick me off to no end! BTW, how many companies switch DB’s anyway? The DB is a core infrastructure IMHO.
Most of the code I’ve seen deals with mickey mouse examples of querying a single table. In the real world that only happens if you’re going to be populating a drop down list or a bunch of check boxes. The real meat is when you have to perform left joins with group by and order by on like 3 or more tables.
Enough of my babel.
Thanks again!!!
June 14, 2007
My two cents..
comparing APPLICATIONframeworks with a “hello world” test is for sure a funny thing, but please! dont be serious about this test! Its just playing around, but holds no information about the usefullness of a framework for a software developer.
Just a few more important things: documentation, debugging enviroments, security and so on.
No deveolper needs or makes a framework for just printing out “hello world” or other static content.
July 6, 2007
My 1 cent.
Database benchmark would be a way appreciated involving series of nice JOIN’s; also making them heavier with their out of the box libs, plugs, mugs, comps and whatnot loaded.
I love Cake and CI, but I found CI more and more getting under my skin. It’s clean, *lightweight* and easy for coding and building apps.
Anyway, thanks for your benchmarks Luci3n! Please re-run them again ;)
(why everyone says: “…great CI documentation.” What’s so great there?)
August 9, 2007
Good post, been wondering about those two. But I still prefer cakePHP then CI and I would recommend cakePHP for those casual Rails programmers
August 20, 2007
Very interesting. It answered some of the questions I was asking for a while, and it seems interesting. Although, I work with CodeIgniter and love it and happy to see CI running front, I feel CI still lacks some issues with some of its libraries like Database, Scaffolding etc.
December 16, 2007
Both CakePHP and CI are nice PHP framework. Nice findings there.
January 9, 2008
I hated cake for it’s steep learning curve, it may be useful for the not so savvy developers, who prefer setting things up rather than write code.
I will try CI for sure, it’s more friedly, so it caters to the more tech savvy.
I don’t consider cakePHP as a RAD tool.
I can set a custom MVC structure in a matter of 30 minutes if I have a good design.
CakePHP takes away the fun in programming.
Now I’m feeling better.
January 25, 2008
I have been using CI most of my projects and i find it comfortable; On CakePHP, i have never try on it; maybe i should try to get one cake and taste it.
March 8, 2008
I’ve used CAKE since 2005. But CI is my favourite now.
March 11, 2008
I’d love to see CakePHP 1.2 tested too. Also, (and this was true in 1.1) you can use other databases and break the conventions in CakePHP. It doesn’t really limit you all that much when you really get into it.
I’ve used it within other CMS’ (notably: Drupal) and carried the Drupal login info, using Drupal’s user/roles tables over to work with the CakePHP app.
Pretty cool stuff you can do with CakePHP - and I’ve heard there is some more overhead than CI, but it’s only slight and the trade off is development time apparently. I personally have not used CI enough to be able to make that decision on my own and I would rather see numbers like you have here…just revised ones now that 1.2 is out.