Today, Amazon Web Services took very an important step in unlocking the advantages of cloud computing for a very important application area. Cluster Computer Instances for Amazon EC2 are a new instance type specifically designed for High Performance Computing applications. Customers with complex computational workloads such as tightly coupled, parallel processes, or with applications that are very sensitive to network performance, can now achieve the same high compute and networking performance provided by custom-built infrastructure while benefiting from the elasticity, flexibility and cost advantages of Amazon EC2
During my academic career, I spent many years working on HPC technologies such as user-level networking interfaces, large scale high-speed interconnects, HPC software stacks, etc. In those days, my main goal was to take the advances in building the highly dedicated High Performance Cluster environments and turn them into commodity technologies for the enterprise to use. Not just for HPC but for mission critical enterprise systems such as OLTP. Today, I am very proud to be a part of the Amazon Web Services team as we truly make HPC available as an on-demand commodity for every developer to use.
HPC and Amazon EC2
Almost immediately after the launch of Amazon EC2, our customers started to use it for High Performance Computing. Early users included some Wall Street firms who knew exactly how to balance the scale of computation against the quality of the results they needed to create a competitive edge. They have run thousands of instances of complex Monte Carlo simulations at night to determine how to be ready at market open. Other industries using Amazon EC2 for HPC-style workloads include pharmaceuticals, oil exploration, industrial and automotive design, media and entertainment, and more.
Further computationally intensive, highly parallel workloads have found their way to Amazon EC2 as businesses have explored using HPC types of algorithms for other application categories, for example to to process very large unstructured data sets for Business Intelligence applications. This has led to strong growth in the popularity of Hadoop and Map Reduce technologies, including Amazon Elastic Map Reduce as a tool for making it easy to run these compute jobs by taking away much of the heavy lifting normally associated with running large Hadoop clusters.
As much as Amazon EC2 and Elastic Map Reduce have been successful in freeing some HPC customers with highly parallelized workloads from the typical challenges of HPC infrastructure in capital investment and the associated heavy operation lifting, there were several classes of HPC workloads for which the existing instance types of Amazon EC2 have not been the right solution. In particular this has been true for applications based on algorithms - often MPI-based - that depend on frequent low-latency communication and/or require significant cross sectional bandwidth. Additionally, many high-end HPC applications take advantage of knowing their in-house hardware platforms to achieve major speedup by exploiting the specific processor architecture. There has been no easy way for developers to do this in Amazon EC2... until today.
Introducing Cluster Compute Instances for Amazon EC2
Cluster Computer Instances are similar to other Amazon EC2 instances but have been specifically engineered to provide high performance compute and networking. Cluster Compute Instances can be grouped as cluster using a "cluster placement group" to indicate that these are instances that require low-latency, high bandwidth communication. When instances are placed in a cluster they have access to low latency, non-blocking 10 Gbps networking when communicating the other instances in the cluster.
Next, Cluster Compute Instances are specified down to the processor type so developers can squeeze optimal performance out of them using compiler architecture-specific optimizations. At launch Cluster Computer Instances for Amazon EC2 will have 2 Intel Xeon X5570 (also known as quad core i7 or Nehalem) processors.
Unlocking the benefits of the cloud for the HPC community
Cluster Compute Instances for Amazon EC2 change the HPC game for two important reasons:
Access to scalable on-demand capacity
Dedicated High Performance Compute clusters require significant capital investments and their procurement often has longer lead times than many enterprise class server systems. This means that most HPC systems are constrained in one or more dimensions by the time that they become operational. As a result, HPC systems are typically shared resources with long queues of jobs waiting to be executed and many users have to be very careful not to exceed their capacity allocation for that period. Cluster Compute Instances for Amazon EC2 bring the advantages of cloud computing to this class of High Performance Computing removing the need for upfront capital investments and giving users access to HPC capacity on demand at the exact scale that they require for their application.
Commoditizing the management of HPC resources
Traditionally we have thought about HPC as the domain of extreme specialism; the kind of research that only happens at places such as the US National Research Labs. These labs have access to some of the world's fastest supercomputers and have dedicated staff to keep them running and fully loaded 365 days a year. But there is a much larger category of potential HPC users beyond these top supercomputer specialists, including even within the more mid-range HPC workloads of these labs such as Lawrence Berkeley National Labs where they have been early successful users of the new Cluster Compute Instances.
Many organizations are using mid-range HPC systems for their enterprise and research needs. Given the specialized nature of these platforms, they require dedicated resources to maintain and operate and put a big burden on the IT organization. Cluster Compute Instances for Amazon EC2 removes the heavy lifting of the operational burden typically associated with HPC systems. There is no more need for hardware tinkering to keep the clusters up and running (I spent many nights doing this; there is no glory in it). These instance types are managed exactly the way other Amazon EC2 instances are managed which allows users to capitalize on the investment they have already made in that area.
More information
If you are looking for more information on Cluster Compute Instances for Amazon EC2 visit our HPC Solutions page. Jeff Barr in his blog post on the AWS developer blog has additional details and there are some great testimonials of early Cluster Compute Instances customers in the press release.
Today a new storage option for Amazon S3 has been launched: Amazon S3 Reduced Redundancy Storage (RRS). This new storage option enables customers to reduce their costs by storing non-critical, reproducible data at lower levels of redundancy. This has been an option that customers have been asking us about for some time so we are really pleased to be able to offer this alternative storage option now.
Durability in Amazon S3
Amazon Simple Storage Service (S3) was launched in 2006 as "Storage for the Internet" with the promise to make web-scale computing easier for developers. Four years later it stores over 100 billion objects and routinely performs well over 120,000 storage operations per second. Its use cases span the whole storage spectrum; from enterprise database backup to media distribution, from consumer file storage to genomics datasets, from image serving for websites to telemetry storage for NASA.
Core to its success has been its simplicity: no matter how many objects you want to store, how small or big those objects are, or what object access patterns you have to deal with, you can rely on Amazon S3 to take away the headaches of dealing with the hard issues typically associated with big storage systems. All the complexities of scalability, reliability, durability, performance and cost-effectiveness are hidden behind a very simple programming interface.
Under the covers Amazon S3 is a marvel of distributed systems technologies. It is the ultimate incrementally scalable system; simply by adding resources it can handle scaling needs in storage and performance dimensions. It also needs to handle every possible failure of storage devices, of servers, networks and operating systems, all while continuing to serve hundreds of thousands of customers.
The goal for operational excellence in Amazon S3 (and for all other AWS services) is that it should be "indistinguishable from perfect". While individual components may be failing all the time, as is normal in any large scale system, the customers of the service should be completely shielded from this. For example to ensure availability of data, the data is replicated over multiple locations such that failure modes are independent of each other. The locations are chosen with great care to achieve this independence and if one or more of those locations becomes unreachable, S3 can continue to serve its customers. Some of the biggest innovations inside Amazon S3 have been how to use software techniques to mask many of the issues that would easily have paralyzed every other storage system.
The same goes for durability; core to the design of S3 is that we go to great lengths to never, ever lose a single bit. We use several techniques to ensure the durability of the data our customers trust us with, and some of those (e.g. replication across multiple devices and facilities) overlap with those we use for providing high-availability. One of the things that S3 is really good at is deciding what action to take when failure happens, how to re-replicate and re-distribute such that we can continue to provide the availability and durability the customers of the service have come to expect. These techniques allow us to design our service for 99.999999999% durability.
Relaxing Durability
There are many innovative techniques we deploy to provide this durability and a number of them are related to the redundant storage of data. As such, the cost of providing such a high durability is an important component of the storage pricing of S3. While this high durability is exactly what most customers want, there are some usage scenarios where customers have asked us if we could relax the durability in exchange for a reduction in cost. In these scenarios, the customers are able to reproduce the object if it would ever be lost, either because they are storing another authoritative copy or because they can reconstruct the object from other sources.
We can now offer these customers the option to use Amazon S3 Reduced Redundancy Storage (RRS), which provides 99.99% durability at significantly lower cost. This durability is still much better than that of a typical storage system as we still use some forms of replication and other techniques to maintain a level of redundancy. Amazon S3 is designed to sustain the concurrent loss of data in two facilities, while the RRS storage option is designed to sustain the loss of data in a single facility. Because RRS is redundant across facilities, it is highly available and backed by the Amazon S3 Service Level Agreement.
More details on the new option and its pricing advantages can be found on the Amazon S3 product page. Other insights in the use of this option can be read on the AWS developer blog.
Today Amazon Web Services has taken another important step in serving customers worldwide: the AWS Asia Pacific (Singapore) Region is now launched. Customers can now store their data and run their applications from our Singapore location in the same way they do from our other U.S. and European Regions.
The importance of Regions
Quite often "The Cloud" is portrayed as something magically transparent that lives somewhere in the internet. This portrayal can be a desirable and useful abstraction when discussing cloud services at the application and end-user level. However, when speaking about cloud services in terms of Infrastructure-as-a-Service, it is very important to make the geographic locations of services more explicit. There are four main reasons to do so:
- Performance - For many applications and services, data access latency to end users is important. You need to be able to place your systems in locations where you can minimize the distance to your most important customers. The new Singapore Region offers customers in APAC lower-latency access to AWS services.
- Availability - The cloud makes it possible to build resilient applications to make sure they can survive different failure scenarios. Currently, each AWS Region contains multiple Availability Zones, which are distinct locations that are engineered to be insulated from failures in other Availability Zones. By placing instances in different Availability Zones, developers can build systems that can survive many complex failure scenarios. The Asia Pacific (Singapore) region launches with two Availability Zones.
- Jurisdictions - Some customers face regulatory requirements regarding where data is stored. AWS Regions are independent, which means objects stored in a Region never leave the Region unless you transfer them out. For example, objects stored in the EU (Ireland) Region never leave the EU. Customers thus maintain control and maximum flexibility to architect their systems in a way that allows them to place applications and data in the geographic jurisdiction of their choice.
- Cost-effectiveness - Cost-effectiveness continues to be one of the key decision making factors in managing IT infrastructure, whether physical or cloud-based. AWS has a history of continuously driving costs down and letting customers benefit from these cost reductions in the form of reduced pricing. Our prices vary by Region, primarily because of varying costs associated with running infrastructure in different geographies; for example, the cost of power may vary quite a bit across different regions, countries, or even cities. We are committed to delivering the lowest cost services possible to our customers based on the cost dynamics of each particular Region.
Worldwide uniform application deployment
Regions have become a very important tool for worldwide rollout of applications. The uniformity of the environment allows customers who have built applications for one Region to easily launch the application in a different Region. For example, there is a large European Insurance company that is looking to expand their EU-based product offerings to the Asia Pacific market. With some minor configuration changes, they can simply move the software running in the AWS EU Region to the AWS Singapore Region and rapidly begin serving Asia Pacific customers. Before AWS, the cost and complexity of moving these applications in a traditional IT model would have prohibited or delayed the company's entry into the Asia Pacific market.
You can begin using Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3), Amazon SimpleDB, Amazon Relational Database Service (Amazon RDS), Amazon Simple Queue Service (Amazon SQS), Amazon Simple Notification Service (Amazon SNS), Amazon CloudWatch, and Amazon CloudFront from the Singapore Region beginning today. Go to http://aws.amazon.com/products for pricing.
You can also find more information on the AWS developer blog. All the RightScale services are available for the new region as well, read more on their blog.
The ecosystem of new application and platform services in the cloud is the future of application development. It will drive rapid innovation and we'll see a wealth of mobile, web and desktop applications arrive that we couldn't dream about a few years ago, and these building blocks are the enablers of that. These services will be delivered not only by new startups but also by enterprises looking to capitalize on their IP.
As examples of such services I always use Twillio (voice &sms) and Simplegeo (location), but it is time to start building out my knowledge of all the different services that are in the ecosystem. If you run such a service or know of one that I should be checking out, please leave the info in the comments below. I'll be using that information in presentations and some future writings on this topic.
I am excited that Amazon Cloudfront has been selected as one of the 10 Editor's pick of 2010 by Streaming Media. Amazon Cloudfront is the Content Delivery Network (CDN) that is dead simple to use both from a technology and a business point of view. From a technology point it literally takes a single button click in the console or a single API call to have your content CDN enabled. You can find several websites marveling about how simple it is to CDN enable your content with amazon Cloudfront. On the business side Amazon Cloudfront revolutionized the CDN business by providing a true pay-as-you-go service, no longer requiring upfront commitments that are commonplace in the CDN business and which opens up CDN for everyone to use. Customers can switch on Cloudfront whenever they want and disable it when no longer needed and only get charged for what they need.
The Streaming Media editors singled out Cloudfront Streaming Content service as possibly truly disruptive. Here are the laurels given by the editors:
Debuting in November 2008, Amazon's entry into the CDN market quickly became a major player. It's still not a threat to Akamai or Limelight, but the addition in December 2009 of Flash streaming to its offerings could truly disrupt the market. Even without Flash, though, CloudFront was winning customers based on its pay-as-you-go rate structure and its self-service model; take a look at Larry Bouthillier's "How to Get Started With Amazon CloudFront Streaming" to get an idea of just how easy the service is to use. As Dan Rayburn wrote on his Business of Video blog, "Amazon will be in the driver's seat to own the market for small and medium sized content owners who need simple delivery at a great price."
For more information on Cloudfront visit their product page.
Today Amazon Web Services takes another big step in making it easier to migrate legacy storage systems to the cloud through AWS Import/Export support for ingesting Punch Cards. AWS Import/Export accelerates moving large amounts of data into and out of AWS using portable storage media for transport. Punch cards are paper-based storage media that represent data using the presence or absence of holes in specific positions. With AWS Import/Export for Punch Cards, enterprises can begin using the service to preserve and unlock the large volumes of data that have accumulated over the last century on what was the first broadly adopted digital storage medium.
"Enterprises spent the better half of the 20th century accumulating critical data on punch cards," said Alyssa Henry, General Manager of Amazon S3. "Now that data can be easily transferred into Amazon S3, making it instantly accessible to customers and applications around the world."
AWS Import/Export for Punch Cards initially supports importing 80 and 96 column punch cards that conform to ANSI X3.26 and are encoded using ISO 6586 compliant 7-bit or 8-bit character sets. Customers may send up to 572 sorted cards per package and may send as many packages to AWS as they like. AWS Import/Export loads each card into Amazon Simple Storage Service (Amazon S3) as a distinct object, uniquely identified by batch and card number.
"We have decades of valuable data stored on punch cards that sit idle in storage cabinets, readable only by specialized, aging systems," said Matt Hunter, CTO of Hollerith Inc. "With AWS Import/Export for Punch Cards, not only are we eliminating large amounts of physical storage space, but we're actively leveraging that data for the first time in years."
For general information on AWS Import/Export, see http://aws.amazon.com/importexport. For details on supported punch card configurations, boxing requirements, and pricing of AWS Import/Export for Punch Cards, see http://aws.amazon.com/importexport/punchcards.
Revisiting the Consistency Challenges
Architecting distributed systems that need to reliably operate at world-wide scale is not a simple task. There are many factors that come into play when you need to meet stringent availability and performance requirements under ultra-scalable conditions. I laid out some of these challenges in an article explaining the concept of eventual consistency. If you need to achieve high-availability and scalable performance, you will need to resort to data replication techniques. But replication in itself brings a whole set of challenges that need to be addressed. For example updates to data now needs to happen in several locations, so what do you do if one or more of those locations is (temporarily) not accessible?
A whole field of computer science is dedicated to finding solutions for the hard problems of building reliable distributed systems. Eric Brewer of UC Berkeley summarized these challenges in what has been called the CAP Theorem, which states that of the three properties of shared-data systems--data consistency, system availability, and tolerance to network partitions--only two can be achieved at any given time. In practice the possibility of network partitions is a given, so the real trade-off is between availability and consistency: In a system that needs to be highly available there are a number of scenarios under which consistency cannot be guaranteed and for a system that needs to be strongly consistent, there are a number of scenarios under which availability may need to be sacrificed. These trade-off scenarios generally involve edge conditions that almost never happen, but in a system that needs to operate at massive scale serving trillions and trillions of requests on a daily basis, one needs to be prepared to handle these cases.
Next to this fundamental tension between availability and consistency, there is also a trade-off between concurrency, performance and consistency. Achieving strict consistency can come at a cost in update or read latency, and may result in lower throughput. Developers should be aware of these trade-offs, as they are not simply an esoteric concept only impacting the internals of the system. Rather, the consistency model of the underlying service will have explicit downstream effects on the developer's application.
Data Consistency Models in the Amazon Services
Within Amazon we strongly favor the use of services that are highly-available and as such we are willing to deal with a relaxed form of consistency called eventual consistency. Our applications are designed to handle this consistency model and there are many cases where this has allowed us to architect services and applications that can meet the most stringent reliability requirements.
Amazon S3 and Amazon SimpleDB are examples of AWS services that are designed using these high-availability principles using an eventual consistency model. Our customers have been able to handle this consistency model as well and have built scalable and reliable applications on top of these services. Now that these services have been in operation for quite some time we have received feedback from a number of our advanced customers asking whether they could get more control over the trade-offs made by the service architects, specifically over the consistency of the read operations. Under the eventual consistency model there is the possibility of stale reads, while in a strongly consistent model reads always returns the latest update, though with the potential for a performance penalty and lower availability under certain rare partition scenarios.
Two new features released today by Amazon SimpleDB put control over the consistency model in the hands of the customers. The first is the ability for customers to perform a Consistent Read, which will prevent the read operation from possibly returning stale data. The second is a Conditional Put and Delete which will ensure that these operations are only performed if certain attribute values are matched.
These new features will make it easier for traditional database application scenarios such as concurrency control, item counters, conditional update/delete, etc., to be implemented using Amazon SimpleDB.
Consistent Read
Until recently SimpleDB only supported eventually consistent reads. This meant that under certain conditions a read, for a small period of time, may not reflect the result of a recently completed write. Yet there is a class of applications that would be much easier to implement if reads would reflect the outcome of writes performed prior to the read. In order to facilitate building such applications SimpleDB now also supports a strong consistency option called consistent read which will reflect all writes successfully returned prior to that read. Especially those applications that originally were designed with a traditional database in mind will be easier to adapt to Simple DB.
Select & GetAttributes request parameters now include an optional Boolean flag ConsistentRead which is set to false by default. If ConsistentRead is absent or set to false, SimpleDB will default to an eventually consistent read. If ConsistentRead is set to true, SimpleDB will return a consistent read.
The table below summarizes the characteristics of the two SimpleDB read consistency options:
| Eventually consistent read | Consistent read |
|
Stale reads possible Lowest read latency Highest read throughput |
No stale reads Higher read latency Lower read throughput |
Since a consistent read may incur higher latency and lower read throughput it should only be used when an application scenario mandates returning the very latest value of an item or attribute. For all other scenarios an eventually consistent read will yield the best performance.
Conditional Put and Delete
Many developers have asked for primitives in SimpleDB for implementing optimistic concurrency control, item level transactions, locks, counters, etc. The new conditional puts and deletes feature in SimpleDB enables building these primitives.
Conditional Puts allow inserting or replacing values of one or more attributes for a given item if the expected consistent value of a single-valued attribute matches the specified expected value. Conditional puts are a mechanism to eliminate lost updates caysed by concurrent writers writing to the same item as long as all concurrent writers use conditional updates.
Conditional deletes allow deleting an item, an attribute or an attribute's value for a given item if the expected consistent value of a single-valued attribute of an item matches the specified expected value. If the current value does not match the expected value, or if the attribute is gone altogether, the delete is rejected.
What about availability
Even though SimpleDB now enables operations that support a stronger consistency model, under the covers SimpleDB remains the same highly-scalable, highly-available, and highly durable structured data store. Even under extreme failure scenarios, such as complete datacenter failures, SimpleDB is architected to continue to operate reliably. However when one of these extreme failure conditions occurs it may be that the stronger consistency options are briefly not available while the software reorganizes itself to ensure that it can provide strong consistency. Under those conditions the default, eventually consistent read will remain available to use.
Summary
By providing developers control over the consistency model, they are now empowered to make the consistency and performance trade-offs. This flexibility is important in scenarios where they need to adapt existing applications that have been designed with a traditional database operational model in mind. Developers can now choose to use a consistent read approach if their application can accept the trade-offs. And with conditional put and delete, developers have the ability to use an optimistic concurrency control approach to managing their database.
You can find more details on the Amazon SimpleDB product pages and Jeff Barr's posting at the Amazon Developer's blog.
Today we launched a new option for acquiring Amazon EC2 Compute resources: Spot Instances. Using this option, customers bid any price they like on unused Amazon EC2 capacity and run those instances for as long their bid exceeds the current "Spot Price." Spot Instances are ideal for tasks that can be flexible as to when they start and stop. This gives our customers an exciting new approach to IT cost management.
The central concept in this new option is that of the Spot Price, which we determine based on current supply and demand and will fluctuate periodically. If the maximum price a customer has bid exceeds the current Spot Price then their instances will be run, priced at the current Spot Price. If the Spot Price rises above the customer's bid, their instances will be terminated and restarted (if the customer wants it restarted at all) when the Spot Price falls below the customer's bid. This gives customers exact control over the maximum cost they are incurring for their workloads, and often will provide them with substantial savings. It is important to note that customers will pay only the existing Spot Price; the maximum price just specifies how much a customer is willing to pay for capacity as the Spot Price changes.
Spot Instances are ideal for Amazon EC2 customers who have workloads that are flexible as to when its tasks are run. These can be incidental tasks, such as the analysis of a particular dataset, or tasks where the amount of work to be done is almost never finished, such as media conversion from a Hollywood's studio's movie vault, or web crawling for a search indexing company. For most of these tasks their completion is not time critical and as such they are ideal targets for additional cost savings.
Economies of scale
Spot Instances are an innovation that is made possible by the unparalleled economies of scale created by the tremendous growth of the AWS Infrastructure Services. The broad Amazon EC2 customer base brings such diversity in workload and utilization patterns that it allows us to operate Amazon EC2 with extreme efficiency. True to the Amazon philosophy, we let our customers benefit from the economies of scale they help us create by lowering our prices when we achieve lower cost structures. Consistently we have lowered compute, storage and bandwidth prices based on such cost savings.
This massive scale also enables new innovative purchasing models such as Spot Instances that empower our customers to gain even more control over the cost-effectiveness of their IT infrastructure. A highly efficient purchasing model such as Spot Instances is another way in which Amazon EC2 customers benefit from the unique economies of scale found in AWS Infrastructure Services.
Different Purchasing Models
The three different purchasing models Amazon EC2 offers give customers maximum flexibility in managing their IT costs; On-Demand Instances are charged by the hour at a fixed rate with no commitment; with Reserved Instances you pay a low, one-time fee and in turn receive a significant discount on the hourly usage charge for that instance; and Spot Instances provide the ability to assign the maximum price you want for capacity with flexible start and end times.
- On-Demand Instances - On-Demand Instances let you pay for compute capacity by the hour with no long-term commitments or upfront payments. You can increase or decrease your compute capacity depending on the demands of your application and only pay the specified hourly rate for the instances you use. These instances are used mostly for short term workloads and for workloads with unpredictable resource demand characteristics.
- Reserved Instances - Reserved Instances let you make a low, one-time, upfront payment for an instance, reserve it for a one or three year term, and pay a significantly lower rate for each hour you run that instance. You are assured that your Reserved Instance will always be available in the Availability Zone in which you purchased it. These instances are used for longer running workloads with predictable resource demands.
- Spot Instances - Spot Instances allow you to specify the maximum hourly price that you are willing to pay to run a particular instance type. We set a Spot Price for each instance type in each region, which is the price all customers will pay to run a Spot Instance for that given hour. The Spot Price fluctuates based on supply and demand for instances, but customers will never pay more than the maximum price they have specified. These instances are used for workloads with flexible completion times.
Managing Spot Instance Applications
There is one technological requirement for the applications that run as Spot Instances: they need to be able to handle that their computation can be stopped and restarted based on the Spot Price in relation to the customer's pricing limit. Ideally these applications will periodically save their state into, for example, EBS or Amazon S3 and upon restart read the last saved state and continue their work. This snapshot-restart technique is a well known methodology already available to many batch oriented applications.
There are three features that help customers manage their Spot Instances and the pricing.
- Persistent Requests: Spot Instance requests can be one-time or persistent. A one-time request will only be satisfied once; a persistent request will remain in consideration after each instance termination. A persistent request is useful when you have a large amount of computing that you want to get done but only below a certain price. By using a persistent request, customers can launch instances any time the Spot Price is below the target price and steadily work through the tasks.
- Launch Groups and specifying Availability Zones: Customers can request a cluster of instances to always launch and terminate simultaneously by specifying a Launch Group. They can also specify the Availability Zone these Instances should be launched in.
- Price History: Amazon EC2 provides a history of the Spot Price for each instance type in each Region via the AWS Management Console and the APIs. Spot Price history is a valuable tool in helping customers use what-if scenarios to determine right pricing level for a particular workload.
Spot instances are a great innovation that, as far as I know, has no equivalent in the IT industry. It brings our customers a powerful new way of managing the cost for those workloads that are flexible in their execution and completion times. This new customer-managed pricing approach holds the power to make new areas of computing feasible for which the economics were previously unfavorable.
For more details and background information visit the Amazon EC2 Spot Instance detail page, the AWS developer blog and the good folks at RightScale.
Today a powerful new feature is available for our Amazon EC2 customers: the ability to boot their instances from Amazon EBS (Elastic Block Store).
Customers like the simplicity of the AMI (Amazon Machine Image) model where they either choose a preconfigured AMI or upload their own AMI into Amazon S3. A wide variety of operating systems and software configurations is available for use. But customers have also asked us for more flexibility and control in the way that Amazon EC2 instances are booted such that they have finer grained control over for example what software configurations and data sets are available to the instance at boot time.
The ability to boot from Amazon EBS gives customers very powerful control over the boot configuration of the Amazon EC2 instances. In the traditional boot process, the root partition of the image will be the local disk, which is created and populated at boot time. In the new Amazon EBS boot process, the root partition is an Amazon EBS volume, which is created at boot time from an Amazon EBS snapshot. Other Amazon EBS volumes beyond the root disk can also made part of the instance before it is booted. This allows for a very fine-grain control of software and data configuration. An additional advantage of using the Amazon EBS boot process is that root partitions are no longer constrained by the size of the local disk and can be up to 1TB in size. And the new boot process is significantly faster because a local disk no longer needs to be populated.
With this new boot process another powerful feature is available to our Amazon EC2 customers: the ability to stop an instance and restart it at a later time with the disk configuration intact. When an instance is restarted, the customer can choose to use a different instance type (e.g., with more memory or CPU), a different operating system (e.g., with new security patches installed), or add new user data. While the instance is stopped it does not accrue any usage hours and customers are only charged for the storage associated with the Amazon EBS volume. The ability to stop and restart an instance is a very powerful mechanism that makes management of instances much easier; many scenarios related to adaptive instance sizing and software management have now become much simpler.
The new boot from Amazon EBS feature is an important step in our continuing quest to remove more and more of the heavy lifting that comes with today's computer environments.
For more details on the new boot features visit the Amazon EC2 detail page and the posting on the AWS developer blog. RightScale's perspective is also worth reading.
We have expanded the AWS footprint in the US and starting today a new AWS Region is available for use: US-West (Northern California). This new Region consists of multiple Availability Zones and provides low-latency access to the AWS services from for example the Bay Area. In the US, AWS customers now can choose between the US-East (Northern Virginia) Region and the new US-West (Northern California) Region. In addition, the EU (Ireland) Region is available to customers who want local access to services from Europe to address their performance or jurisdiction requirements.
As we announced earlier this month a Region with multiple Availability Zones will come online in Singapore in the first half of 2010, with other regions in Asia to follow later in 2010.
AWS is committed to making its services available at low cost. We use the cost-following principle in pricing the services and several times we have lowered our pricing in response to the cost savings we were able to achieve. Operation costs are often different based on location and, as such, the pricing for services may vary somewhat between Regions, giving our customers the power to make trade-offs between, for example, cost and latency.
Subscribe to this weblog's