4) Logging: If it wasn’t logged, it never happened We need to be careful about what we log and we should log everything that can help us figure out what went wrong. When working with the cloud it’s normal to experience failure every so often. Never build an application without thinking about how you will recover from a fault and how long will it take. Sometimes, when you start on a new project, you only have time to plan for the straightforward cases. This means that we can learn a lot from a brand new application with real users. In order to facilitate this learning process and support it properly you need logging in place. 5) Prepare for failure “The cloud never fails” – that’s what the cloud provider wants us to believe! The reality is different (and has been proven several times): even well-managed clouds will fail. The problem is not that they fail, but that most people are unprepared for such failures, because they believe the cloud is an indestructible silver bullet. Cloud providers do not explicitly plan for the failover of your services, they just provide the platform and the tools, and it’s your job to plan and implement your own failover system. Cloud services are known for their accessibility, but they are still bound to Murphy’s law: “Anything that can go wrong will go wrong”. Amazon’s AWS, Microsoft’s Azure and Google Mail, amongst others, have all failed in the past and most of them will fail again in the future. An important step for us was to lessen the reliance on our 20+ third party integrations. Our working assumption is that any third party system will fail, and if handled badly, a third party slowdown could quickly escalate and become our slowdown and impact our systems. To mitigate this, the vast majority of our services run through an out of band message bus. Messages sent to the bus are sent in a “fire and forget” fashion. Messages sent to the bus takes an overage of 2ms, regardless of the state of the 3rd party. This mechanism allows us to handle requests in a fashion that does not impact the user’s experience. All of our emails are handled by a specialist email provider, which provides an API that we use to send emails. This service has proven to be highly reliable, however if they have performance issues or their service becomes unavailable, the user experience is not affected because failed messages are stored and placed in the queue to be resent later. This mechanism allows us to handle a complete outage from a range of providers using the same principle without having to worry about our users being impacted. Once a provider resumes service, we simply pick up the previously failed messages. David Kavanagh is technical director at hybrid estate agency Purplebricks.com.
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.