Tag Archives: php

Offloading time consuming PHP work outside your application

It is natural that as business applications become more complex they often have to capture and process more information. With web-based applications in particular, this can become a problem.

Let’s suppose we have built a CRM application for our company to store information about our customers. In its first release, the application was designed to capture basic information about a customer,  and store it in the database – simple. This is bread and butter work for most applications and should be fairly light work. User’s of this system would find their interactions with this software to be fairly slick.

However, six months down the line, the company owners think it would be really useful to allow emails to be sent to the customers from the CRM – Quotes, Order Invoices and Progress Updates. Whilst this seems a perfectly legitimate addition to the CRM, adding even simple email connectivity to an application can cause a performance degradation, particularly when using SMTP servers to send the emails. The SMTP protocol is a very ‘chatty’ protocol, and to just send 1 email, requires several conversations between the application and the server to send an email. I won’t go into the specifics of this conversation but in a nutshell these conversations discuss CONNECTION, AUTHENTICATION, SENDING, DELIVERY and RESPONSE. Each of these steps can take a while to happen, and sending an email can very easily amount to an additional three or four seconds of delay that the user would experience waiting for the page to reload and allow them to continue. This is known as IO Blocking Behaviour.

Offloading.

The user of the software wants to click the “send message” button, and then move on to the next customer or activity, but they have to wait for the message to be sent before they can. Three or four seconds per message, per customer, per day can amount to quite a lot of waiting time. So how to we overcome this.

There are several ways you could achieve this, but in this article I am going to explore offloading the “Email sending” process to be handled at a later time. The idea behind this is that the user still clicks the “send message” button, but instead of actually sending the email at this time, we just log a request that the user has requested to send an email. This request could be stored in a log file, or a database table, or even sent to a Message Broker server (more on that later). Regardless of where and how the request to send the email is stored, the user is not blocked by waiting for the email to send. They are notified that the email has been queued for delivery and can carry on their work without the three or four second delay.

Who actually sends the email then..?

Good question. We are going to build a PHP page which scans the queue, or log for email requests and then processes them one at a time. This process can take place on the same, or even a different server, and can be running in the background all day. The only purpose of this process is to send emails that have been requested. Whilst this process is happy chugging away in the corner, the user’s are interacting with their CRM quite happily, raising more and more email requests. The user’s don’t actually care if their email email is delivered seconds, or even minutes after they clicked the “Send Message” button, they just want to know that it will be sent. We can call this partnership between the CRM and the Email Processor a non-blocking relationship. The activity of one system is not being held up by another.

Look out for the next part of this article where I will look at a few techniques to achieve offline processing of these emails, first by using CRON Jobs and then using a more robust messaging system such as RabbitMQ and the AMQP protocol.

Thanks for reading.

James

SEO friendly urls in codeigniter – the challenge

Codeigniter is generally quite good at providing friendly urls, mainly due to its url segment approach. Instead of the traditional querystring approach where parameters are passed as follows:

index.php?account=james

codeigniter uses segments instead.

index/account/james

So the segmented approach is much friendlier but still not perfect. What I want to achieve is a much more descriptive url such as “view-my-account.html”. In order to achieve this I decided to generate a ‘slug’ (usually a hyphenated string in lower case) for each post as I create or edit them in the cms. The slug is a friendly version of the blog post title as can be seen below. This is stored in a column called “slug” in the database for use later.

slug

The intention is to have my controller perform a database or file-cache lookup for the slug, and then route it to the correct controller.

routing

One of the first challenges is to tame the Codeigniter routing system, which allows urls to be directed to the correct controllers and methods. Routes can be managed inside of “/application/config/routes.php”. In order to dynamically route the first step is to add some extra code in the routes.php to handle dynamic routes:

// include the dynamic routes from the cache file
require_once("application/cache/routes.php");

We also need some way of actually generating these routes, so what I decided to do was generate them when posts are created or updated. After the new post has been written to the database, a new record is created that contains the friendly url, the Slug, and the route that it should be directed to. In my blog this model is quite straight forward and comprises of the segments “blog/category/postid”. So in my add/edit posts I have some code like this

//////////////////////////////////////////////////////////////////
//update the record for the dynamic route
$dynamic_route = "blog/index/" . $category . "/false/" . (int)$id;
$route = array(
    'post_id' => $id,
    'route' => $dynamic_route,
    'slug' => $slug
);

$result = mysql_query("SELECT * FROM routes WHERE post_id='{$id}'");
$num_rows = mysql_num_rows($result);

if($num_rows > 0) {
    //update the record
    $this->db->where('post_id', $id);
    $this->db->update('routes', $route);
} else {
    //insert a new record
    $this->db->insert('routes', $route);
}

Now that the route has been generated we have to make it accessible to Codeigniter somehow. To save on database queries I have decided to write all of the routes for the site into a file cache – this lives inside “application/cache/routes.php” and effectively contains an array of all the routes for the site. I did this with a new model called “Router_m” which has a method called “cache_routes()” which looks as follows:

class Router_m extends CI_Model {

    function __construct() {
        parent::__construct();
    }   

    //grab all of the routes from the database, and cache to a file
    public function cache_routes()
    {
        $this->db->select("*");
        $query = $this->db->get("routes");

        foreach ($query->result() as $row)
        {
            $data[] = '$route["' . $row->slug . '"] = "' . $row->route . '";';
            $output = "load->helper('file');
            write_file(APPPATH .  "cache/routes.php", $output);
        }
    }
}

This cache file is loaded by “config/routes.php” after the static routes have been defined. The file looks like this:

$route["new-single-whore-left-me.html"] = "blog/index/musician/false/27";
$route["content-management-system.html"] = "blog/index/blog/false/3";
$route["inline-ajax-image-uploader-for-tinymce.html"] = "blog/index/developer/false/45";
$route["jumping-on-the-bandwagon-youtube-videos.html"] = "blog/index/blog/false/41";
$route["test-post.html"] = "blog/index/blog/false/46";
$route["seo-friendly-urls-in-codeigniter-the-challenge.html"] = "blog/index/developer/false/44";
$route["big-brother-is-watching-google-analytics.html"] = "blog/index/blog/false/42";
$route["new-image-uploader-plupload.html"] = "blog/index/developer/false/43";
$route["sublime-text-2-video-intro.html"] = "blog/index/developer/false/40";
$route["multiband-compression.html"] = "blog/index/musician/false/39";
$route["store-your-snippets-with-github-gist.html"] = "blog/index/developer/false/38";
$route["on-fire-with-sparks.html"] = "blog/index/developer/false/37";
$route["sublime-coding.html"] = "blog/index/developer/false/35";
$route["my-new-guitar.html"] = "blog/index/musician/false/36";
$route["a-moment-of-clarity-literally.html"] = "blog/index/musician/false/33";
$route["song-writing-basics.html"] = "blog/index/musician/false/15";
$route["codeigniter-language-packs.html"] = "blog/index/developer/false/29";
$route["are-you-listening-to-your-speakers-or-your-room.html"] = "blog/index/musician/false/34";
$route["the-importance-of-gain-staging.html"] = "blog/index/musician/false/32";
$route["the-studio.html"] = "blog/index/musician/false/30";
$route["the-subjective-part-of-mixing-your-own-vocals.html"] = "blog/index/musician/false/28";
$route["implementing-tinymce-as-your-rich-text-editor.html"] = "blog/index/developer/false/2";
$route["welcome-to-jamesstoddernnet.html"] = "blog/index/home/false/26";
$route["new-single-out-now-smile.html"] = "blog/index/musician/false/14";

Of course, to make use of all this routing, I have had to change the links on the site, so that in the “recent posts” panel for example, instead of building a traditional codeigniter controller/method style link,  the slugs can be used directly. As soon as you navigate to the slug, its correct location is loaded from the routes cache. Of course the user doesn’t know this and just sees the friendly title. I have also got the page title to be set now from the post’s title as well to make everything even more friendly.