Apache::Session benchmark – fighting a Web bottleneck

March 25th, 2011

Apache::Session is easy to use and works great – but it could be a big bottleneck. I developed a huge web application for my company which is – over all requests – slower than it should be. Some earlier analysis told me that the reason might be in the sessioning part. Adding a big ajax-loaded page with many sub-requests during the last days, I started to look at the speed issue again this morning.

Everybody knows that memory is faster than a hard disk. The application is using Apache::Session::SharedMem and even if it’s “shared memory” (which is slower than normal memory due to locking), it must be faster than a hard disk – because it’s memory!

Benchmarking the sessioning is easy:

use Apache::Session::SharedMem;
use Time::HiRes('time');
my $t = time; # Store the start time
for (1..1000) {
   my $sid; # session-id
   for (1..2) {
      # Create a new session (1) or open an existing (2)
      tie %Session,"Apache::Session::SharedMem",$sid,{ expires_in => 60 };
      $sid = $Session{'_session_id'}; # Store the sessionid for next loop run
      for (1..10) {
         $X = $Session{$_}; # Read from the session
         $Session{$_} = $_; # Write to the session
      }
      untie %Session; # Close the session
   }
}
print $t - time; # time needed for this run

Perl has powerful benchmarking modules, but “time” is sometimes easier to use. The test will run 1000 times, each run creates a new session (because $sid is undef), save the session id, read some (not existing) values from the session, write some values into the session and close the session. Next the same session is opened again (inner loop run #2) and the same read/write actions are performed – because a session usually exists and I didn’t want to mix up results if the creation of a new session has some delay.

The test started – and nothing happend. After reducing the outer loop to 10, the test took 7,95 seconds to run (on the busy server running the application)! That is nearly 0,8 seconds per request for having a session – way too much.

I changed the test script to use Apache::Session::File – and it ran 0,02 seconds on the same machine. Switching back to 1000 loops made the test taking 1,63 seconds which is about 2 ms per request.

I like to use the Linux shared memory path /dev/shm for files which aren’t required after a reboot, so I tried to place the session data there – and got 1,44 seconds. I don’t think that these 0,19 ms per request really make any difference and I try not to waste any memory – but the sessions of this application will go to /dev/shm from now on.

Switching was easy:

  • Add a mkdir for the session cache directory to my mod_perl startup file which is also preloading the modules
  • Change the “use” (startup and session module) and “tie” (session module only) lines from Apache::Session::SharedMem to Apache::Session::File
  • Add a “Directory => ‘/dev/shm/session_cache’” to the tie’s options hash

My ajax page is much faster now – and all other functions and scripts, too.

Bookmark and Share

Unknown power

July 8th, 2010

So, you’re using Perl? For web applications? You’re not alone, many people do the same, but…

no one knows!

So many people use Perl for so many web projects, but only very few people actually show or tell it.

We (the Padre IRC channel) shortly discovered that there are plenty more or less “official” icons and buttons for showing this to the world:

The Perl foundation (TPF) gives us

and  

And O’Reilly licensed some camels for free usage:

and    and   and  

(and some others)

All of them have their licenses but as a summary one could say that you could put them on your website if you

  • Run a non-commercial project (most but not all are also approved for commercial projects)
  • and make the graphic a link to http://www.perl.org (Perl foundation buttons)  or http://www.perl.com (O’Reilly buttons)

Please follow the links above and read the full license before using them.

Gabor Szabo colltected some Powered-by-Perl icons here but I don’t know about the sources an licensing. You might violate TPF, O’Reilly or others rights when using them, so add them on your own risk.

I like this one . If you got any rights on it, please tell be and I’ll remove it.

Why should you promote Perl?

Why are you using Perl?

It’s a great language but too few people are using it.

(Nearly) every PHP project is using “.php” – URLs which tell people: This (maybe great) site is running PHP!

But Perl based sites? They use mod_perl (which has no file extension by default) or CGI-scripts (which aren’t recognized as Perl scripts).

It would be great if you’ld add one of the buttons to your Perl projects. If you don’t want it on every page, simply add it to the about, imprint or contact page.

If you’re not allowed to add a button (maybe it’s not your project but you’re managing it), you might be allowed to add it to the Perl Foundation list of Perl users – it’s free and it’s free advertising for your company or project.

You don’t have any Perl based website?

But you might know someone who runs a Perl based project or website! Tell him (or her) to add a button or point her/him to this blog post.

Bookmark and Share

Hannover.pm – creation

March 22nd, 2010

Another non-Padre post, apologies for this:

Talking to Gabor during CeBit made me trying to start a new Hannover.pm, a Hannover Perl Mongers Group.

Anybody interested is invited to join the Mailinglist or visit the Hannover.pm homepage.

Bookmark and Share

Perl::Staff 0.02

March 7th, 2010

Perl::Staff has just been released as version 0.02 which includes updated and new blog links to CeBit reports and fixes some POD char and formatting errors.

Look at it on CPAN or grab and perldoc it :-)

Bookmark and Share

Things I showed to people not knowing Perl but PHP

March 7th, 2010

Thank you for reading this article. If you continue, please read it completly before judging or writing about it and expecially before using any information written here!

We were talking to many PHP guys on CeBit 2010. I tried showing the huge power of Perl but – as far as I remember – I never said anybody “forget PHP”. Here are the most important items (odered by usual order of appereance):

1. You don’t need to learn Perl

Neither PHP not Perl people like that fact, but PHP was originally based on Perl. There are still many similarities between both considering keywords and syntax. But most PHP people agree if you say: “If you know PHP, you basically know a good amount of Perl.”

2. CPAN

There are about 20.000 modules for all kinds of things on CPAN, like database access, file parsing, templating and math and all of them are free. All CPAN modules could be installed by a CPAN client which usually comes with your Perl.

Please tell me if you know any other language which has something comparable.

3. DBIx::Class

Most of the PHP users also use MySQL but for the full time of CeBit 2010, I met only one person (who actually was a database expert with some PHP knowledge) who answered the question “Do you like to write SQL statements, expecially SELECT JOINs over some tables?” with yes, all others answered between “no”, “not really” and “less than writing PHP”.

Basically DBIx::Class abstracts the SQL statement layer: Your source dosn’t need to contain one SQL statement any longer. The results are:

  • No need to SELECT JOIN, just point DBIx::Class to some property of the table row you need, maybe using a referenced table in just one line of source.
  • Once you got your object, you could use all referenced tables as simple childs. Sample: $user->customer->company gets the company name out of the customer table starting from the user table pointing to the customer table.
  • DBIx::Class reduces database load by fetchting the object when it’s first actually used, not at the time the object is created.
  • DBIx::Class makes SQL injection impossible because it’s encapsulated by DBIx::Class without any additional development.
  • Reduces development time of database actions by about 50% compared to writing SQL statements (depending on the developer, of course)

Nothing written above has anything to do with comparing PHP and Perl. It’s just comparing DBIx::Class to $dbh->select… (or other ways sending SQL statements in other languages). There is something on PHP which might be compareable to DBIx::Class even (up to me) it isn’t nearly as powerful as DBIx::Class, but most PHP – people I met on CeBit 2010 were still writing plain SQL statements – as I did about eight month ago.

Also remember that using DBIx::Class will introduce a slight part of OOP even to people who don’t actually use it. Up to me this would be the preferred way of starting OOP development because it’s a slow start – you could add more and more OOP as time goes by and you feel ready to look into another world.

4. Catalyst and Template::Toolkit

There are technologies for PHP which are really better in many situations than these two. Catalyst is a complete Web framework which reduces the typical programming amount for weblications and Template::Toolkit (TT) provides a very powerful template engine, but I was working together with some webdesigners in the past which can’t handle it because it was too programmish.

If you pass a DBIx::Class object to TT, the one writing the HTML code could easily access all data of the table row including any child objects and any future changes to the table structure without any additional application source.

5. Coming back to CPAN

I wrote about three examples of the more than 20.000 CPAN modules, but if you ask people what else do they want to do and live-search for it on CPAN, it usually gives you some results. Please recheck this if/when the CPAN search engine (which shows modules even on very slow relations to the search) is being fixed.

Good samples to show are

  • Parse::CSV for parsing CSV files in two minutes development time instead of ten
  • Creating or modifiing PDF files
  • Accessing external things like network servers of any kind

6. How to start

Padre, the free Perl IDE comes with some few lessons within the examples, not enough for really learning Perl but good for looking into the syntax.

The next step would be http://learn.perl.org for many more resources how to learn Perl.

If you want to meet the community, look at the PerlMongers groups who meet maybe also in your area.

If you don’t even want to look at Perl…

…you should at least try Padre which also has a PHP plugin for working on PHP files. You’re welcome to improve it by testing or changing the source.

Last notice: Writing Padre Plugins in PHP should theoretically be possible using the Parrot engine, if you want to try it, please add your experiences to the Padre Trac wiki.

Bookmark and Share