It is natural that as business applications become more complex they often have to capture and process more information. With web-based applications in particular, this can become a problem.
Let’s suppose we have built a CRM application for our company to store information about our customers. In its first release, the application was designed to capture basic information about a customer, and store it in the database – simple. This is bread and butter work for most applications and should be fairly light work. User’s of this system would find their interactions with this software to be fairly slick.
However, six months down the line, the company owners think it would be really useful to allow emails to be sent to the customers from the CRM – Quotes, Order Invoices and Progress Updates. Whilst this seems a perfectly legitimate addition to the CRM, adding even simple email connectivity to an application can cause a performance degradation, particularly when using SMTP servers to send the emails. The SMTP protocol is a very ‘chatty’ protocol, and to just send 1 email, requires several conversations between the application and the server to send an email. I won’t go into the specifics of this conversation but in a nutshell these conversations discuss CONNECTION, AUTHENTICATION, SENDING, DELIVERY and RESPONSE. Each of these steps can take a while to happen, and sending an email can very easily amount to an additional three or four seconds of delay that the user would experience waiting for the page to reload and allow them to continue. This is known as IO Blocking Behaviour.
The user of the software wants to click the “send message” button, and then move on to the next customer or activity, but they have to wait for the message to be sent before they can. Three or four seconds per message, per customer, per day can amount to quite a lot of waiting time. So how to we overcome this.
There are several ways you could achieve this, but in this article I am going to explore offloading the “Email sending” process to be handled at a later time. The idea behind this is that the user still clicks the “send message” button, but instead of actually sending the email at this time, we just log a request that the user has requested to send an email. This request could be stored in a log file, or a database table, or even sent to a Message Broker server (more on that later). Regardless of where and how the request to send the email is stored, the user is not blocked by waiting for the email to send. They are notified that the email has been queued for delivery and can carry on their work without the three or four second delay.
Who actually sends the email then..?
Good question. We are going to build a PHP page which scans the queue, or log for email requests and then processes them one at a time. This process can take place on the same, or even a different server, and can be running in the background all day. The only purpose of this process is to send emails that have been requested. Whilst this process is happy chugging away in the corner, the user’s are interacting with their CRM quite happily, raising more and more email requests. The user’s don’t actually care if their email email is delivered seconds, or even minutes after they clicked the “Send Message” button, they just want to know that it will be sent. We can call this partnership between the CRM and the Email Processor a non-blocking relationship. The activity of one system is not being held up by another.
Look out for the next part of this article where I will look at a few techniques to achieve offline processing of these emails, first by using CRON Jobs and then using a more robust messaging system such as RabbitMQ and the AMQP protocol.
Thanks for reading.