March 2009 Archives

Keeping your database simple and fast is often difficult if you use higher level frameworks such as ActiveRecords in Ruby or Java object persistence technologies such as Hibernate. There is a lot of magic that is happening out of sight that you have no control over. If you then have to scale your application it is often the relational database that these technologies require that becomes the performance and scaling bottleneck. Often requiring complex custom implementations of partitioning and sharding to make it work.

The AWS services Amazon S3 and Amazon SimpleDB were designed to handle the dominant storage usage patterns within Amazon and they greatly reduced our need to rely on relational storage for scaling our systems. But it is almost never the case that a single storage technique is used in applications and services that need to operate at enterprise scale. For example it is a common pattern that objects stored in S3 using a primary key, have a collection of secondary keys (e.g. metadata) stored in SimpleDB. SimpleDB provides very fast indexing for querying of the metadata that will return primary keys of objects located in S3.

At SXSW Interactive there was a great panel/presentation by Mike Subelsky, co-founder of AWS customer OtherInbox , about their experiences with scaling Ruby-on-Rails applications in the Cloud. They demonstrated that with Amazon EC2 and Amazon S3 Ruby/Rails scales just fine. The room was packed and there was some great Q&A.

During the Q&A presentation co-founder and CEO of OtherInbox Joshua Baer gave some great insight in the changing role of relational databases and some really good advice about how they were able to keep their database simple and fast. After the session I asked Joshua to explain it once more for the readers of this weblog.

.

Flexibility is a key advantage of using Amazon Web Services; you can obtain resources instantaneously without the headache of owning them. If you no longer need the resource, you release it and only pay for what you have used. This is a very powerful model that has helped many of our customers drive capital expense out of their IT operation. It has helped both enterprises and startups reduce the risk that comes with developing new products and businesses.

While this on-demand flexibility is ideal for a whole range of scenarios, some Amazon EC2 customers who have more predictable workloads have asked us for even greater flexibility in the cost model through the ability to reserve capacity. To address this need we have introduced Amazon EC2 Reserved Instances, which provides customers who have predictable usage patterns with a way to even further reduce costs.

Using Amazon EC2 Reserved Instances is really simple: you make a low one-time payment for each instance you want to reserve and in turn you receive a significant discount on the hourly charges for that instance. Furthermore, you don't pay hourly charges at all during periods when you have the instance turned off. Reserved Instances give customers more flexibility to reduce their IT costs. As these instances work exactly the same as On-Demand Instances, customers have the power to seamlessly extend their reserved base with scalable on-demand capacity.

Many of our customers find levels of predictability in many of their workloads. And not just those with major computational tasks such as HPC and Data mining, but many different types of customers, from enterprises running ERP and CRM applications, to media companies running portals and media streaming, to young businesses serving Web based applications are able to find a predictable base level of usage that can now be served through Amazon EC2 Reserved Instances.

Reserved Instances also give customers who already have an existing infrastructure in place a transition model that is closer to their current strategy, but at significant cost saving. Customers who currently own their own hardware to meet capacity needs have a total cost of ownership that does not only include the server, network equipment, rack space, etc., but also includes operation costs such as power, cooling, system administration, etc. which has to be paid regardless of how much of the capacity is being used. Reserved Instances at first glance appear similar, as you make a single upfront payment to reserve capacity, but the operational cost is only paid if the capacity is indeed used, allowing for the best of worlds, resulting in a significant cost reduction but maintaining the benefits of elastic computing.

For more details see the Amazon EC2 detail page and the AWS Developer Blog.

Today we have some important news for our Amazon EC2 customers who are running Windows Server and Windows SQLServer instances and who have been looking to extend their coverage for fault-tolerance and locality reasons. Starting today Windows instances can be launched in an additional Availability Zone is the US and they can also be launched in two Availability Zones in Europe. This allows developers who use our Windows instances to build solutions that can tolerate various failure and recovery scenarios. It also puts Windows Server into the hands of developers who want low latency for their European customers.

We have also integrated these features into the Amazon AWS console, such that you now can use the console to launch instances in any of our regions, regardless whether it is Europe or the US. Popular third party tools such as Elastic Fox also have been upgraded and provide additional functionality such as advanced naming an tagging.

Rolling out services and features globally is very important to us as it allows our customers to take a uniform approach to serving their customers world-wide.