I always enjoy looking for solutions to difficult challenges in non-obvious places. That is probably why I like using probabilistic techniques for problems that appear to be hard, or impossible to solve deterministically. The probabilistic approach may not result in the perfect result, but it may get you very close, and much faster than deterministic techniques (which may even be computationally impossible).
Some of the earliest approaches using probabilities in physics experiments resulted in the Monte Carlo methods. Their essential idea is using randomness to solve problems that might be deterministic in principle. These are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results.
The Monte Carlo methods can be traced back to Stanislaw Ulam, John van Neuman, and Nick Metropolis at the Los Alamos Scientific Laboratory in the late 40s. The Monte Carlo methods were crucial in the simulations of the Manhattan Project, given the limited computational power available in those days
The paper I will be reading this weekend is the original paper from 1949, by Metropolis and Ulam. For fun, I’ve also decided to add a second paper by Herbert Anderson, who was a member of the Manhattan project. Anderson’s paper describes the use of Monte Carlo methods, and the computers in the Manhattan project.
“The Monte Carlo Method”, Nicholas Metropolis, S. Ulam, Journal of the American Statistical Association, Vol. 44, No. 247. (Sep., 1949), pp. 335-341.
"Metropolis, Monte Carlo and the MANIAC", Anderson, Herbert L., Los Alamos Science, (1986) 14: 96–108.
Listening to the "Algorithms to Live By" audio on my commute this morning, once again I was struck by the beauty of Bloom Filters. So, I decided it is time to resurrect the 'Back-to-Basics Weekend Reading' series, as I will be re-reading some fundamental CS papers this weekend.
In the past, I have done some weekend reading about Counting Bloom Filters, but now I am going even more fundamental, and I invite you to join me.
Bloom Filters, conceived by Burton Bloom in 1970, are probabilistic data structures to test whether an item is in a set. False positives are possible, but false negatives are not. Meaning, if a bit in the filter is not set, you can be sure the item is not in the set. If it is in the set, the mapped item may be in the set.
This is a hugely important technique if you need to process and track massive amounts of unique data units, as it is very space-efficient. From Dynamo and Postgresql, to HBase and Bitcoin, Bloom Filters are used in almost all modern distributed systems. This weekend I will be reading the original paper by Bloom from 1970, and another more recent survey paper that describes several variants and applications that have been developed over the years.
"Space/Time Trade-offs in Hash Coding with Allowable Errors", Bloom, Burton H., in Communications of the ACM, 13 (7): 422–426
"Cache-, Hash- and Space-Efficient Bloom Filters", Putze, F.; Sanders, P.; Singler, J., in Demetrescu, Camil, Experimental Algorithms, 6th International Workshop, WEA 200
This article titled "Überlebensstrategie für die digitale Transformation" appeared in German last week in the "Die Zukunft beginnt heute (the future starts today)" section of Wirtschaftwoche.
Smaller companies have a lot to gain in the digital era – provided they adopt the right mindset. The winners will be those that view their business from the eyes of their customers and understand that fast-paced innovation is the key to long-term growth. With this mindset they can take on even the largest enterprises who are slow to adapt to the fast moving digital reality.
The digital era is here. Companies that haven't realized that by now will fall behind. In many industry segments and markets, for example, platform services, we've already witnessed how start-ups and niche providers have unleashed a revolution. Companies that used to be dominant but stare at all the changes around them for too long in a state of paralysis can quickly end up in a struggle to survive – look no further than the entertainment and music industry where streaming services have eaten a significant piece of the cake of the hard-copy providers. The better you understand why and how small and medium-sized players can conquer global markets, you'll be better positioned to come out as a winner.
Digitalization allows even the smallest companies to think big because it puts technology into their hands that was previously hard to access and too costly to acquire. But adopting modern technologies alone is not enough to win the market battle. However, when new technologies are combined with a passion for putting the interests of the customer at the heart of everything you do, they can give agile companies that decisive push to the front of the pack. And Mittelstand companies have fantastic opportunities, provided they digitize more of their existing business models. Especially in manufacturing-based industries, introducing more software that complements hardware can eliminate fixed costs and allow you to quickly scale up to a global level. Companies that embrace this can rise to become leading players, taking the place previously reserved for the 'big shots' in their industries.
Digitalization starts with having the right mindset: namely one aimed at creating innovative digital experiences. Continuous customer-centric experimentation has been the leading principle at Amazon from the start, in both our e-commerce activities as well as in Amazon Web Services. We found out that by organizing our innovation efforts around customers' needs, we could innovate very fast. Since 2006, Amazon Web Services has introduced far more than 2,500 new services and features, and around 90% of them were the result of wishes articulated directly by customers.
The first requirement for developing an innovation mindset is to adapt your offerings fast to changing customer behavior. There are great examples of German companies that already do this. One is Vorwerk and its all-in-one cooking machine, called Thermomix. This premium product has been on the market for more than 50 years. But the way people cook today is different than in the 1960s. Today, cooking must be convenient, fast, and healthy. People want to prepare meals without too much effort, yet some appreciate a bit of guidance during the entire cooking process, from picking a new recipe from Vorwerk's cloud-based database to putting the finished meal on the table.
Companies that want to adopt a digital innovation mindset should start leaving their comfort zone – even if they don't (yet) feel any pressure to change. Or put another way: they have to develop an inner drive to not just deliver on their customers' changing needs, but rather anticipate them.
A company that does this very well is SKF, the global market leader in ball bearings and a supplier to many industries. SKF keeps the strategy of its clients in mind at all times and tries to think along with them. For example, wind turbine operators. SKF will ask itself: What are the critical, vulnerable issues in their business model? Doing maintenance on wind turbines is a complicated affair because they are located so far apart from each other. At the same time, the wind turbines have to work reliably whenever the wind conditions are ideal. So SKF takes the initiative to develop services outside its own core business. In this case, it offers ways for wind farm operators to operate and maintain their sites from any location using cloud solutions.
Digitization opens up new opportunities for companies to create value. So adopting a digital innovation mindset will automatically lead you to start thinking what kind of value you would like to add to a market in the future. Beckhoff, a leading supplier of automation technology devices, is a great example in this context. The firm developed a solution that sends data about mission-critical systems from the shop floor of their customers into the cloud. The cloud connection opens a two-way street, a bi-directional connection: All of a sudden Beckhoff's customers can receive and analyze data from their machines via the cloud, which enables them to operate their manufacturing or fulfill maintenance tasks from anywhere in the world. By offering a solution like this, Beckhoff is not only transforming itself from a hardware producer to a software provider, but also serving as the catalyst for the business transformation of its customers. This shift will allow Beckhoff to claim a new position in the value chain.
Entrenching a digital innovation mindset in your company won't happen overnight. However, as the growing number of firms that have been successful in the digital era by adopting this mindset proves, it's worth the effort. In the end it means you won't just merely survive all the changes happening around you. Instead you will lay the foundation for a bright future.
In November 2015, Amazon Web Services announced that it would launch a new AWS infrastructure region in the United Kingdom. Today, I'm happy to announce that the AWS Europe (London) Region, our 16th technology infrastructure region globally, is now generally available for use by customers worldwide.
UK companies are using AWS to innovate across diverse industries, such as energy, manufacturing, medicaments, retail, media, and financial services and the UK is home to some of the world's most forward-thinking businesses. These include startups like Fanduel, JustEat, and Monzo to enterprises such as British Gas, Trainline, Travis Perkins, News UK, the Financial Times.
The British Government is also helping to drive innovation and has embraced a cloud-first policy for technology adoption. Take Peterborough City Council as an example. The council has deployed IoT Weather Stations in Schools across the City and is using the sensor information collated in a Data Lake to gain insights on whether the weather or pollution plays a part in learning outcomes.
London has also established itself as a critical center for the financial services sector and a significant hub for venture capital activity across all Europe. The City's thriving venture capital and start-up accelerator communities are fueling growth and innovation, making it one of the most important locations in the world to do business. AWS is working with incubators and accelerators such as SeedCamp and Techstars, in London; Ignite100 in Newcastle; and DotForge in Sheffield and Manchester to help startups make the most of the cloud.
We believe in our customers and are investing for the long term. With the AWS Europe (London) Region, we look to better serve end users in the UK. With the launch of the AWS Europe (London) Region, AWS can enable many more UK enterprise, public sector and startup customers to reduce IT costs, address data locality needs, and embark on rapid transformations in critical new areas, such as big data analysis and Internet of Things.
All around us we see that the AWS capabilities foster a culture of experimentation with businesses of all sizes. AWS is not only affordable but it is secure and scales reliably to drive efficiencies into business transformations. I have been humbled by just how much our UK customers have been able to achieve using AWS technology so far. In just this past month we've had HSBC, ARM, Missguided, and most recently at re:Invent 2016, Trainline, talking with us about how they are using AWS to transform and scale their businesses.
Following are just a few of the reasons that customers have given us for building their business on the AWS Cloud:
Blend seamlessly into the digital world: With the rising importance of technology-driven business transformation, an emphasis on certain enterprise and consumer-based opportunities emerges.
To take advantage of the game-changing opportunities, businesses are looking to blend into the digital world. Take GoSquared, a UK startup that runs all its development and production processes on AWS, as an example. GoSquared provides various analytics services that web and mobile companies can use to understand their customers' behaviors. With AWS, GoSquared can process tens of billions of data points every day from four continents to provide customers with a single view.
Use catalysts for real-time business models: The Internet of Things (IoT) is undoubtedly driving a philosophy of interconnecting people, process, and machines to create massive volumes of data that has potential for disruptive change.
The BMW Group is using AWS for its new connected-car application that collects sensor data from BMW 7 Series cars to give drivers dynamically updated map information. BMW built its new car-as-a-sensor (CARASSO) service in only six months leveraging several AWS services. By running on AWS, CARASSO can adapt to rapidly changing load requirements that can scale up and down by two orders of magnitude within 24 hours. By 2018, CARASSO is expected to process data collected by a fleet of 100,000 vehicles traveling more than eight billion kilometers.
Mobilize business operations by computing everywhere: Computing and data processing within the confines of a data center or office is easy. There is much interesting and potentially valuable data out in the field, if only it could be collected, processed, and turned into actionable intelligence.
This data could be located miles below the surface of the Earth in a mine or an oil well, in a sensitive and safety-critical location like a hospital or a factory, or even on another planet. At re:Invent 2016, AWS announced Greengrass (in limited preview), a new service designed to extend the AWS programming model to small, simple, field-based devices.
Parse real time information to generate visibility : Big data tools have enabled organizations to manage resources, anticipate activity relevant to their business, and make informed decisions faster. Real-time monitoring and evaluation of events have led to a positive impact on performance or operations.
Channel 4 (in the UK) chose AWS to help monetize volumes of platform data. By running on AWS and using Amazon EMR, the broadcast company can collect and analyze vast amounts of data in real time to deliver highly targeted ads to viewers during a 60-minute program.
Extract valuable insights with machine learning: The world is witnessing the emergence of a broad and powerful range of new systems—computer programs that can teach themselves to grow and change when exposed to new data.
Fraud.net is a good example of this. Fraud.net use AWS to support highly scalable, big data applications that run machine learning processes for real-time analytics. Fraud.net uses AWS to build and train machine learning models in detecting online payment fraud. Fraud.net uses Amazon Machine Learning to provide more than 20 machine learning models and relies on Amazon DynamoDB and AWS Lambda to run code without provisioning or managing servers.
Develop the next-generation software application that is capable of action: Chances are that you may already be using artificial intelligence as you interact with applications that not only sense and comprehend but are capable of action, especially when one views solutions through the lens of automation.
The Airbnb application, for instance, uses Aerosolve to deliver its dynamic pricing feature. Unbabel uses a combination of artificial intelligence and human translation to deliver fast, cost-effective, high-quality translation services globally. VizSense is yet another paragon that develops advanced visual search and image recognition solutions to serve companies in e-commerce, mobile commerce, and online advertising.
These short sketches illustrate the power of the cloud for customers, but it is still early days. With the launch of the AWS Europe (London) Region, I look forward to seeing many more innovative use cases enabled by AWS.
Our AWS Europe (London) Region is open for business now and we are excited to offer a complete portfolio of services—from our foundational service stack for compute, storage, and networking to our more advanced solutions and applications. We look forward to broadening this portfolio to include more services over the next several quarters.
For more information about the services we offer in our London region, see the Europe/Middle East/Af rica tab of our Region Table.
For more information about how customers are innovating using AWS, see All AWS Customer Stories.
You can learn more about how we help connect the world to the cloud at AWS Global Infrastructure.
Earlier this year, Amazon Web Services (AWS) announced it would launch a new AWS infrastructure region in Montreal, Quebec. Today, I'm happy to share that the Canada (Central) Region is available for use by customers worldwide. The AWS Cloud now operates in 40 Availability Zones within 15 geographic regions around the world, with seven more Availability Zones and three more regions coming online in China, France, and the U.K. in the coming year.
The Canadian opportunity
Canada has set forth a bold innovation agenda grounded in entrepreneurship, scientific research, growing small and medium-sized businesses with a focus on environmentally friendly technologies, and the transition to a digital economy. This agenda leverages the transformative aspects of technology and encourages Canadian companies, universities, governments, not-for-profits, and entrepreneurs to contribute to building a durable innovation economy.
Given this, enterprises, public sector bodies, startups, and small businesses are looking to adopt agile, scalable, and secure public cloud solutions. The new Canada (Central) Region offers a robust suite of infrastructure, management, and developer services that can enable innovators to deploy market-leading applications. Access to secure, scalable, low-cost AWS infrastructure in Canada allows customers to innovate and provide tools to meet privacy, sovereignty, and compliance requirements.
The new AWS Canada (Central) Region also continues the company's focus on delivering cloud technologies to customers in an environmentally friendly way. AWS data centers in Canada will draw from a regional electricity grid that is 99 percent powered by hydropower. For more information about AWS efforts, see AWS & Sustainability.
Some examples of how current customers use AWS are:
Kik Interactive is a Canadian chat platform with hundreds of millions of users around the globe. It adopted Amazon Redshift, Amazon EMR and AWS Lambda to power its data warehouse, big data, and data science applications, supporting the development of product features at a fraction of the cost of competing solutions.
Rapid time to market
The Globe and Mail (Globe) is one of Canada's most read newspapers, with a national weekly circulation of 4.7 million. To increase online readership, it worked with AWS Partner Network (APN) Partner ClearScale to develop a personal recommendation capability. The solution, which leverages Amazon Kinesis, Amazon DynamoDB, and Amazon EMR to collect, store, and process the data, as well as AWS CloudFormation and AWS OpsWorks to support the Globe's DevOps environment, was deployed in three months—less than half the time it would have taken had the newspaper built it on-premises.
Enterprise-class services available from Canada
Box is an enterprise content management and collaboration platform used by more than 41 million users and 59,000 businesses—including 59% of the Fortune 500. It relies on the scale and power of Amazon Simple Storage Service (Amazon S3) to deliver in-region storage options to businesses and organizations across the world in Canada, Japan, Singapore, Australia, Ireland, Germany, and the U.S., as part of its Box Zones ecosystem. Having the ability to provide these services locally enables Box to better serve Canadian enterprises looking for cloud solutions while ensuring their data is stored inside Canada.
Lululemon Athletica is a Canadian athletic apparel company that is using AWS Lambda, AWS CodePipeline, and AWS CodeDeploy to rapidly build and deploy their digital marketing and e-commerce solutions for the upcoming 2016 holiday season. By using AWS to manage the continuous deployment and delivery of their applications, Lululemon personnel can focus on market differentiation instead of maintaining custom infrastructure solutions.
The Municipal Property Assessment Corporation, a public sector organization responsible for providing valuations for more than 5 million properties in Ontario, runs its core property valuation engine 5,000 percent faster at one-tenth the cost by using AWS versus legacy IT architecture.
SmartSimple, an AWS Canadian APN Partner, provides cloud-based grants and case and research management solutions to Fortune 500 companies, government agencies, universities, and research organizations around the world. With AWS' security, business continuity, scaling and agile capabilities, SmartSimple can architect solutions on AWS so that their customers can comply with their privacy requirements.
ScribbleLive provides a cloud platform that media companies use to cover breaking news on their websites and engage with their audiences. Using AWS, ScribbleLive can scale to support news websites and blogs in 14 languages worldwide and maintain high availability at peak capacity, all the while decreasing operational costs by 35 percent.
Vidyard is a revolutionary platform that provides business users with the ability to personalize and strategically target messages through online video. The fast-growing, Canadian startup uses Amazon EC2, Amazon S3, Amazon Elasticsearch Service, Amazon RDS, and AWS OpsWorks to scale quickly while securely delivering seamless video upload and playback services for its customers.
AWS as your strategic cloud provider
As these customer stories show, we believe that the AWS Cloud can and will serve as the foundation for Canada's innovation agenda. For more customer case studies, see All AWS Customer Stories.
We are excited to offer a robust portfolio of services from our foundational service stack for compute, storage, and networking to our more advanced solutions and applications. We look forward to broadening this portfolio to include additional services over the next several quarters. For more information about the services we offer in our Canada (Central) Region, see the Region Table.
With the launch of the Canada (Central) Region, AWS will enable enterprise customers and startups, as well as government, education, and nonprofit organizations in Canada to deploy innovative and cost-effective IT solutions in areas such as big data, machine learning, serverless computing, mobile, and more. Millions of AWS customers have recognized the value of the cloud. The new Canada (Central) Region provides a high-performance, enterprise-grade, scalable, and secure way for Canadian organizations to do the same. You can learn more about our growing footprint at AWS Global Infrastructure.
AWS lance sa nouvelle Région du Canada (Centre) à Montréal, Québec
Par Werner Vogels, le 8 décembre 2016
Plus tôt cette année, Amazon Web Services (AWS) a annoncé avoir pour projet de lancer une nouvelle infrastructure régionale AWS à Montréal, Québec. Aujourd'hui, j'ai le plaisir de vous annoncer que la Région du Canada (Centre) est maintenant opérationnelle pour ses clients du monde entier. Le Cloud AWS est maintenant disponible sur 40 zones de disponibilité réparties dans 15 régions du monde. Sept autres zones de disponibilité et trois autres régions seront mises en ligne en Chine, en France et au Royaume-Uni au cours de la prochaine année.
Une occasion en or au Canada
Le Canada a lancé un programme audacieux d'innovation qui vise les écotechnologies et la transition à une économie numérique et repose sur l'entreprenariat, la recherche scientifique et la croissance des PME. Ce programme vise l'exploitation de la puissance transformatrice des technologies et encourage les entreprises, les universités, les gouvernements, les organismes sans but lucratif et les entrepreneurs du Canada à contribuer au développement d'une économie durable fondée sur l'innovation.
C'est pour cette raison que les entreprises, les entités du secteur public, les startups et les PME cherchent à adopter des solutions de nuage public qui sont flexibles, évolutives et sûres. La nouvelle Région du Canada (Centre) offre une gamme solide de services d'infrastructure, de gestion et de développement qui permet aux innovateurs de déployer des applications de premier plan sur le marché. Les clients, ayant accès à une infrastructure AWS sécuritaire et évolutive à faible coût au Canada, peuvent innover et utiliser des outils leur permettant de respecter les exigences en matière de protection de la vie privée, de souveraineté et de conformité.
La nouvelle Région AWS du Canada (Centre) soutient aussi l'objectif de la compagnie de fournir aux clients les technologies infonuagiques d'une façon qui respecte l'environnement. Les centres de données d'AWS au Canada seront alimentés par un réseau électrique régional à 99 pour cent hydroélectrique. Pour en savoir plus sur les moyens déployés par AWS, voir AWS & Sustainability.
Voici quelques exemples d'utilisation des services d'AWS par les clients actuels :
Kik Interactive est une plateforme de clavardage canadienne qui a des centaines de millions d'utilisateurs partout dans le monde. Elle a adopté Amazon Redshift, Amazon EMR et AWS Lambda pour faire fonctionner ses applications d'entrepôt de données, de mégadonnées et de science des données, ce qui lui permet de mettre au point des fonctionnalités de produit à la fraction du coût des solutions de la concurrence.
Mise en marché rapide
The Globe and Mail (Globe) est l'un des quotidiens les plus lus au Canada, tiré à 4,7 millions d'exemplaires chaque semaine au pays. Pour augmenter son lectorat en ligne, il a utilisé les services de Clearscale, un partenaire du AWS Partner Network (APN), pour développer sa capacité à recevoir des recommandations personnelles. La solution, qui fait usage de Amazon Kinesis, Amazon DynamoDB et Amazon EMR pour recueillir, stocker et traiter les données et AWS CloudFormation et AWS OpsWorks pour soutenir l'environnement DevOps du Globe, a été déployée en trois mois seulement—la moitié du temps que cela aurait pris pour installer une telle solution locale pour le quotidien.
Des services de classe entreprise offerts à partir du Canada
Box est une plateforme de gestion de contenu et de collaboration pour les entreprises utilisée par plus de 41 millions d'utilisateurs et 59 000 entreprises, y compris 59 % de celles figurant sur la liste de Fortune 500. Elle dépend de l'évolutivité et de la puissance d'Amazon Simple Storage Service (Amazon S3) pour offrir des options de stockage régionales aux entreprises et aux organisations du monde entier au Canada, au Japon, à Singapour, en Australie, en Irlande, en Allemagne et aux États-Unis, dans le cadre de son écosystème Box Zones. Avoir la capacité de fournir ces services localement permet à Box de mieux servir les entreprises canadiennes qui cherchent non seulement des solutions infonuagiques, mais à ce que leurs données soient stockées au Canada.
Lululemon Athletica est une compagnie de vêtements de sport canadienne qui utilise AWS Lambda, AWS CodePipeline et AWS CodeDeploy pour rapidement élaborer et déployer des solutions de cybermarketing et de commerce électronique pour le temps des Fêtes 2016. En utilisant AWS pour gérer le déploiement continu et la livraison de leurs applications, les membres du personnel de Lululemon peuvent se concentrer à se distinguer sur le marché plutôt que d'être préoccupés par la gestion de solutions d'infrastructure ponctuelles.
La Société d'évaluation foncière des municipalités (MPAC), un organisme du secteur public responsable de l'évaluation de plus de cinq millions de propriétés en Ontario, exploite son principal moteur d'évaluation foncière 5 000 pour cent plus vite et à un dixième du coût avec AWS par rapport à leur architecture informatique préexistante.
SmartSimple, un partenaire canadien du réseau APN d'AWS, offre des bourses infonuagiques et des solutions de gestion de dossiers et de recherche à des compagnies figurant sur la liste de Fortune 500, des organismes gouvernementaux, des universités et des organismes de recherche du monde entier. Grâce aux capacités offertes par AWS en matière de sécurité, de continuité des affaires, d'évolutivité et d'agilité, SmartSimple peut mettre au point des solutions AWS qui permettent à ses clients de se conformer à leurs exigences de protection de la vie privée.
Expansion du marché
ScribbleLive offre une plateforme infonuagique utilisée par les sociétés de médias pour couvrir les nouvelles de dernière heure sur leurs sites Web et pour communiquer avec leurs publics. À l'aide d'AWS, ScribbleLive peut faire accroître ses capacités pour soutenir les sites Web et les blogues de nouvelles dans 14 langues à travers le monde, ainsi que conserver une haute disponibilité à pleine capacité tout en diminuant les coûts opérationnels de 35 pour cent.
Vidyard est une plateforme révolutionnaire qui donne aux entreprises la capacité de personnaliser et de cibler de manière stratégique ses messages en utilisant la vidéo en ligne. Cette startup canadienne en plein croissance utilise Amazon EC2, Amazon S3, Amazon Elasticsearch Service, Amazon RDS et AWS OpsWorks pour faire accroître ses capacités rapidement tout en offrant des services sécuritaires de téléversement vidéo en continu et de lecture vidéo pour ses clients.
AWS est votre fournisseur de services infonuagiques stratégique
Comme le démontrent ces témoignages clients, nous croyons que le nuage AWS peut et va servir de fondement au programme d'innovation du Canada. Pour lire d'autres études de cas de clients, allez à Tous les témoignages de clients AWS.
Nous sommes emballés de vous offrir un solide portefeuille de services, allant de notre gamme essentielle pour le calcul, le stockage et la mise en réseau à nos solutions et nos applications plus avancées. Nous avons hâte d'élargir ce portefeuille et d'y ajouter d'autres services au fil des prochains trimestres. Pour en savoir plus sur les services que nous offrons dans notre Région du Canada (Centre), voir le Tableau des régions.
Grâce à la création de la Région du Canada (Centre), AWS permettra aux entreprises, aux startups ainsi qu'aux organismes gouvernementaux, aux organismes du milieu de l'éducation et aux organismes sans but lucratif du Canada de déployer des solutions informatiques innovantes et rentables dans des domaines comme les mégadonnées, l'apprentissage machine, l'informatique sans serveur, la téléphonie mobile, et bien plus encore. Des millions de clients d'AWS ont reconnu la valeur du cloud. La nouvelle Région du Canada (Centre) offre un moyen hautement performant, professionnel, évolutif et sûr qui permettra aux organisations canadiennes de faire de même. Vous pouvez en apprendre plus sur notre empreinte croissante à Infrastructure mondiale AWS.
In my keynote at AWS re:Invent today, I announced 13 new features and services (in addition to the 15 we announced yesterday).
My favorite parts of James Bond movies is are where 007 gets to visit Q to pick up and learn about new tools of the trade: super-powered tools with special features which that he can use to complete his missions, and, in some cases, get out of some nasty scrapes. Bond always seems to have the perfect tool for every situation that he finds himself in. *
At AWS, we want to be the Q for developers, giving them the super-powered tools and services with deep features in the Cloud. In the hands of builders, the impact of these services has been to completely transform the way applications are developed, debugged, and delivered to customers.
I was joined by 32,000 James Bonds at the conference today from all around the world, and we introduced new services focused on accelerating this transformation across development, testing and operations, data and analytics, and computation itself.
Transformation in Development, Testing, & Operations
Although development and operations are often overlooked, they are the engines of agility for most organizations. Today, cCompanies cannot afford to wait two or three years between releases, and; customers have found that continually releasing incremental functionality to customer frequently reduces risk and improves quality.
Today, we're making available broad new services which that let builders prepare and operate their applications more quickly and efficiently, and respond to changes in both their business and their operating environment, swiftly. We launched the following new services and features today to help.
AWS OpsWorks for Chef : a fully managed Chef Automate environment, available through AWS OpsWorks to fuel even more automation and reduce the heavy lifting associated with continuous deployment.
Amazon EC2 Systems Manager : A collection of tools for package installation, patching, resource configuration, and task automation on Amazon EC2.
AWS Codebuild: A new, fully managed, extensible service for compiling source code and running unit tests, which is integrated with other application lifecycle management services— such as AWS CodeDeploy, AWS CodeCommit, and AWS CodePipeline— for dramatically decreasing the time between iterations of software.
Amazon X-Ray: A new service to analyze, visualize, and debug distributed applications, allowing builders to identify performance bottlenecks and errors.
Personal Health Dashboard: A new personalized view of AWS service health for all customers, allowing developers to gain visibility into service health issues which that may be affecting their application.
AWS Shield : protective Protective armor against distributed denial of service (DDoS) attacks, available as Shield Standard and Shield Advanced. Shield Standard gives DDoS protection to all customers using API Gateway, Elastic Load Balancing, Route 53, CloudFront, and EC2. Shield Advanced protects against more sophisticated DDoS attacks, with access to help through a 24x7 AWS DDoS response team.
Transformation in Data
In the old world, access to infrastructure resources was a big differentiator for big, wealthy companies. No more. Today, any developer can have access to a wealth of infrastructure technology services which that bring advanced technology to their fingertips times in the Cloud. The days of differentiation through infrastructure are behind us; the technology is now evenly distributed.
Instead, most companies today and in the future will differentiate themselves through the data that they collect and have access to, and the way in which they can put that data to work for the benefit of their customers. We rolled out three new services today to make that easier.:
Amazon Pinpoint : A data-driven engagement service for mobile apps. Define which segment of customers to engage with, schedule a push notification engagement campaign, and track the results in real-time.
AWS Batch: Fully- managed batch processing at any scale, with no batch processing software to install or servers to manage.
Dynamically provision compute resources and optimize task distribution based on volume and resource requirements
AWS Glue : A fully- managed data catalog and ETL service, which that makes it easy to move data between data stores, while also simplifying and automating time time-consuming data discovery, conversion, mapping, and job scheduling tasks.
Transformation in Compute
Amazon EC2 made it possible to build application architectures in a way we had always wanted to; and, over the past decade, gave us the opportunity to build secure, resilient, available applications with decoupled application components which that can be scaled independently, and updated more frequently. When I talk to our customers, I hear time and again how they are taking these transformative principles, and taking them to the next level, by building smaller, more discrete, distributed components using containers and AWS Lambda.
Today, we're accelerating this transformation with a new distributed application coordination service, new Lambda functionality, and an open source container framework.:
AWS Step Functions: Coordinate the components of distributed applications using visual workflows. Step through functions at scale.
Blox: A collection of open source projects for container management and orchestration.
Thirteen new services and major features focused on developers. We're excited to see how customers you will put these new features to work.
*: Sean Connery is the definitive Bond.
From the early days of Amazon, Machine learning (ML) has played a critical role in the value we bring to our customers. Around 20 years ago, we used machine learning in our recommendation engine to generate personalized recommendations for our customers. Today, there are thousands of machine learning scientists and developers applying machine learning in various places, from recommendations to fraud detection, from inventory levels to book classification to abusive review detection. There are many more application areas where we use ML extensively: search, autonomous drones, robotics in fulfillment centers, text processing and speech recognition (such as in Alexa) etc.
Among machine learning algorithms, a class of algorithms called deep learning has come to represent those algorithms that can absorb huge volumes of data and learn elegant and useful patterns within that data: faces inside photos, the meaning of a text, or the intent of a spoken word.After over 20 years of developing these machine learning and deep learning algorithms and end user services listed above, we understand the needs of both the machine learning scientist community that builds these machine learning algorithms as well as app developers who use them. We also have a great deal of machine learning technology that can benefit machine scientists and developers working outside Amazon. Last week, I wrote a blog about helping the machine learning scientist community select the right deep learning framework from among many we support on AWS such as MxNet, TensorFlow, Caffe, etc.
Today, I want to focus on helping app developers who have chosen to develop their apps on AWS and have in the past developed some of the seminal apps of our times on AWS, such as Netflix, AirBnB, or Pinterest or created internet connected devices powered by AWS such as Alexa and Dropcam. Many app developers have been intrigued by the magic of Alexa and other AI powered products they see being offered or used by Amazon and want our help in developing their own magical apps that can hear, see, speak, and understand the world around them.
For example, they want us to help them develop chatbots that understand natural language, build Alexa-style conversational experiences for mobile apps, dynamically generate speech without using expensive voice actors, and recognize concepts and faces in images without requiring human annotators. However, until now, very few developers have been able to build, deploy, and broadly scale applications with AI capabilities because doing so required specialized expertise (with Ph.D.s in ML and neural networks) and access to vast amounts of data. Effectively applying AI involves extensive manual effort to develop and tune many different types of machine learning and deep learning algorithms (e.g. automatic speech recognition, natural language understanding, image classification), collect and clean the training data, and train and tune the machine learning models. And this process must be repeated for every object, face, voice, and language feature in an application.
Today, I am excited to announce that we are launching three new Amazon AI services that eliminate all of this heavy lifting, making AI broadly accessible to all app developers by offering Amazon's powerful and proven deep learning algorithms and technologies as fully managed services that any developer can access through an API call or a few clicks in the AWS Management Console. These services are Amazon Lex, Amazon Polly, and Amazon Rekognition that will help AWS app developers build these next generation of magical, intelligent apps. Amazon AI services make the full power of Amazon's natural language understanding, speech recognition, text-to-speech, and image analysis technologies available at any scale, for any app, on any device, anywhere.
After the launch of the Alexa Skill Kit (ASK), customers loved the ability to build voice bots or skills for Alexa. They also started asking us to give them access to the technology that powers Alexa, so that they can add a conversational interface (using voice or text) to their mobile apps. They also wanted the capability to publish their bots on chat services like Facebook Messenger and Slack.
Amazon Lex is a new service for building conversational interfaces using voice and text. The same conversational engine that powers Alexa is now available to any developer, making it easy to bring sophisticated, natural language 'chatbots' to new and existing applications. The power of Alexa in the hands of every developer, without having to know deep learning technologies like speech recognition, has the potential of sparking innovation in entirely new categories of products and services. Developers can now build powerful conversational interfaces quickly and easily, that operate at any scale, on any device.
The speech recognition and natural language understanding technology behind Amazon Lex and Alexa is powered by deep learning models that have been trained on massive amounts of data. Developers can simply specify a few sample phrases and the information required to complete a user's task, and Lex builds the deep learning based intent model, guides the conversation, and executes the business logic using AWS Lambda. Developers can build, test, and deploy chatbots directly from the AWS Management Console. These chatbots can be accessed anywhere: from web applications, chat and messenger apps such as Facebook Messenger (with support for exporting to Alexa Skills Kit and Slack support coming soon), or connected devices. Developers can also effortlessly include their Amazon Lex bots in their own iOS and Android mobile apps using the new Conversational Bots feature in AWS Mobile Hub.
Recently, a few selected customers participated in a private beta of Amazon Lex. They provided us with valuable feedback as we rounded off Amazon Lex for a preview launch. I am excited to share some of the feedback from our beta customers HubSpot and Capital One.
HubSpot, a marketing and sales software leader, uses a chatbot called GrowthBot to help marketers and sales personnel be more productive by providing access to relevant data and services. Dharmesh Shah, HubSpot CTO and Founder, tells us that Amazon Lex enabled sophisticated natural language processing capabilities on GrowthBot to provide a more intuitive UI for customers. Hubspot could take advantage of advanced AI and ML capabilities provided by Amazon Lex, without having to code the algorithms.
Capital One offers a broad spectrum of financial products and services to consumers, small businesses, and commercial clients through a variety of channels. Firoze Lafeer, CTO Capital One Labs, tells us that Amazon Lex enables customers to query for information through voice or text in natural language and derive key insights into their accounts. Because Amazon Lex is powered by Alexa's technology, it provides Capital One with a high level of confidence that customer interactions are accurate, allowing easy deployment and scaling of bots.
The concept of a computer being able to speak with a human-like voice goes back almost as long as ENIAC (the first electronic programmable computer). The concept has been explored by many popular science fiction movies and TV shows, such as "2001: A Space Odyssey" with HAL-9000 or the Star Trek computer and Commander Data, which defined the perception of computer-generated speech.
Text-to-speech (TTS) systems have been largely adopted in a variety of real-life scenarios such as telephony systems with automated speech responses or help for visually or speech-impaired people. Prof. Stephen Hawking's voice is probably the most famous example of synthetic speech used to help the disabled.
TTS systems have continuously evolved through the last few decades and are nowadays capable of delivering a fairly natural-sounding speech. Today, TTS is used in a large variety of use cases and is turning into a ubiquitous element of user interfaces. Alexa and its TTS voice is yet another step towards building an intuitive and natural language interface that follows the pattern of human communication.
With Amazon Polly, we are making the same TTS technology used to build Alexa's voice to AWS customers. It is now available to any developer aiming to power their apps with high-quality spoken output.
In order to mimic human speech, we needed to address a variety of challenges. We needed to learn how to interpret various text structures such as acronyms, abbreviations, numbers, or homographs (words spelled the same but pronounced differently and having different meanings). For example:
I heard that Outlander is a good read, though I haven't read it yet, or
St. Mary's Church is at 226 St. Mary's St.
Last but not least, as the quality of TTS gets better and better, we expect a natural intonation matching the semantics of synthesized texts. Traditional rule-based models and ML techniques, such as classification and regression trees (CART) and hidden Markov models (HMM) present limitations to model the complexity of this process. Deep learning has shown its capacity in representing complex and nonlinear relationships at different levels of speech synthesis process. The TTS technology behind Amazon Polly takes advantage of bidirectional long short-term memory (LSTM) networks using a massive amount of data to train models that convert letters to sounds and predict the intonation contour. This technology enables high naturalness, consistent intonation, and accurate processing of texts.
Amazon Polly customers have confirmed the high quality of generated speech for their use cases. Duolingo uses Amazon Polly voices for language learning applications, where quality is critical. Severin Hacker, the CTO of Duolingo, acknowledged that Amazon Polly voices are not just high in quality, but are as good as natural human speech for teaching a language.
The Royal National Institute of Blind People uses the Amazon TTS technology to support the visually impaired through their largest library of books in the UK. John Worsfold, Solutions Implementation Manager at RNIB, confirmed that Amazon Polly's incredibly lifelike voices captivate and engage RNIB readers.
We live in a world that is undergoing digital transformation at a rapid rate. One key outcome of this is the explosive growth of images generated and consumed by applications and services across different segments and industries. Whether it is a consumer app for photo sharing or printing, or the organization of images in the archives of media and news organizations, or filtering images for public safety and security, the need to derive insight from the visual content of the images continues to grow rapidly.
There is an inherent gap between the number of images created and stored, and the ability to capture the insight that can be derived from these images. Put simply, most image stores are not searchable, organized, or actionable. While a few solutions exist, customers have told us that they don't scale well, are not reliable, are too expensive, rely on complex pipelines to annotate, verify, and process massive amount of data for training and testing algorithms, need a team of highly specialized and skilled data scientists, and require costly and highly specialized hardware. For companies that have successfully built a pipeline for image analysis, the processes of maintaining, improving, and keeping up with the research in this space proves to be high friction. Amazon Rekognition solves these problems.
Amazon Rekognition is a fully managed, deep-learning–based image analysis service, built by our computer vision scientists with the same proven technology that has already analyzed billions of images daily on Amazon Prime Photos. Amazon Rekognition democratizes the application of deep learning technique for detecting objects, scenes, concepts, and faces in your images, comparing faces between two images, and performing search functionality across millions of facial feature vectors that your business can store with Amazon Rekognition. Amazon Rekognition's easy-to-use API, which is integrated with Amazon S3 and AWS Lambda, brings deep learning to your object store.
Getting started with Rekognition is simple. Let's walk through some of the core features of Rekognition that help you build powerful search, filter, organization, and verification applications for images.
Object and scene detection
Given an image, Amazon Rekognition detects objects, scenes, and concepts and then generates labels, each with a confidence score. Businesses can use this metadata to create searchable indexes for social sharing and printing apps, categorization for news and media image archives, or filters for targeted advertisement. If you are uploading your images to Amazon S3, it is easy to invoke an AWS Lambda function that passes the image to Amazon Rekognition and persist the labels with confidence scores into an Elasticsearch index.
With any given image, you can now detect faces present, and derive face attributes like demographic information, sentiment, and key landmarks from the face. With this fast and accurate API, retail businesses can respond to their customers online or in store immediately by delivering targeted ads. Also, these attributes can be stored in Amazon Redshift to generate deeper insights of their customers.
Amazon Rekognition's face comparison and face search features can provide businesses with face-based authentication, verification of identity, and the ability to detect the presence of a specific person in a collection of images. Whether simply comparing faces present in two images using CompareFaces API, or creating a collection of faces by invoking Amazon Rekognition's IndexFace API, businesses can rely on our focus on security and privacy, as no images are stored by Rekognition. Each detected face is transformed into an irreversible vector representation, and this feature vector (and not the underlying image itself) is used for comparison and search.
I am pleased to share some of the positive feedbacks from our beta customers.
Redfin is a full-service brokerage that uses modern technology to help people buy and sell houses. Yong Huang, Director of Big Data & Analytics, Redfin, tell us that Redfin users love to browse images of properties on their site and mobile apps and they want to make it easier for their users to sift through hundreds of millions of listing and images. He also added that Amazon Rekognition generates a rich set of tags directly from images of properties. This makes it relatively simple for them to build a smart search feature that helps customers discover houses based on their specific needs. And, because Amazon Rekognition accepts Amazon S3 URLs, it is a huge time-saver for them to detect objects, scenes, and faces without having to move images around.
Summing it all up
We are in the early days of machine learning and artificial intelligence. As we say in Amazon, we are still in Day 1. Yet, we are already seeing the tremendous value and magical experience Amazon AI can bring to everyday apps. We want to enable all types of developers to build intelligence in to their applications. For data scientists, they can use our P2 instances, Amazon EMR Spark MLLib, deep learning AMIs, MxNet and Amazon ML to build their own ML models. For app developers, we believe that these three Amazon AI services enable them to build next-generation apps to hear, see, and speak with humans and the world around us.
We'll also be hosting a Machine Learning " State of the Union" that covers all the three new AmazonAI services announced today along with demos from Motorola Solutions and Ohio Health – head over to Mirage (as we added more seating!). Also, we have a series of breakout sessions on using MXNet at AWS re:Invent on November 30th at the Mirage Hotel in Las Vegas.
Machine learning is playing an increasingly important role in many areas of our businesses and our lives and is being employed in a range of computing tasks where programming explicit algorithms is infeasible.
At Amazon, machine learning has been key to many of our business processes, from recommendations to fraud detection, from inventory levels to book classification to abusive review detection. And there are many more application areas where we use machine learning extensively: search, autonomous drones, robotics in fulfillment centers, text and speech recognitions, etc.
Among machine learning algorithms, a class of algorithms called deep learning hascome to represent those algorithms that can absorb huge volumes of data and learn elegant and useful patterns within that data: faces inside photos, the meaning of a text, or the intent of a spoken word. A set of programming models has emerged to help developers define and train AI models with deep learning; along with open source frameworks that put deep learning in the hands of mere mortals. Some examples of popular deep learning frameworks that we support on AWS include Caffe, CNTK, MXNet, TensorFlow, Theano, and Torch.
Among all these popular frameworks, we have concluded that MXNet is the most scalable framework. We believe that the AI community would benefit from putting more effort behind MXNet. Today, we are announcing that MXNet will be our deep learning framework of choice. AWS will contribute code and improved documentation as well as invest in the ecosystem around MXNet. We will partner with other organizations to further advance MXNet.
AWS and Support for Deep Learning Frameworks
At AWS, we believe in giving choice to our customers. Our goal is to support our customers with tools, systems, and software of their choice by providing the right set of instances, software (AMIs), and managed services. Just like in Amazon RDS―where we support multiple open source engines like MySQL, PostgreSQL, and MariaDB, in the area of deep learning frameworks, we will support all popular deep learning frameworks by providing the best set of EC2 instances and appropriate software tools for them.
Amazon EC2, with its broad set of instance types and GPUs with large amounts of memory, has become the center of gravity for deep learning training. To that end, we recently made a set of tools available to make it as easy as possible to get started: a Deep Learning AMI, which comes pre-installed with the popular open source deep learning frameworks mentioned earlier; GPU-acceleration through CUDA drivers which are already installed, pre-configured, and ready to rock; and supporting tools such as Anaconda and Jupyter. Developers can also use the distributed Deep Learning CloudFormation template to spin up a scale-out, elastic cluster of P2 instances using this AMI for even larger training runs.
As Amazon and AWS continue to invest in several technologies powered by deep learning, we will continue to improve all of these frameworks in terms of usability, scalability, and features. However, we plan to contribute significantly to one in particular, MXNet.
Choosing a Deep Learning Framework
Developers, data scientists, and researchers consider three major factors when selecting a deep learning framework:
- The ability to scale to multiple GPUs (across multiple hosts) to train larger, more sophisticated models with larger, more sophisticated datasets. Deep learning models can take days or weeks to train, so even modest improvements here make a huge difference in the speed at which new models can be developed and evaluated.
- Development speed and programmability, especially the opportunity to use languages they are already familiar with, so that they can quickly build new models and update existing ones.
- Portability to run on a broad range of devices and platforms, because deep learning models have to run in many, many different places: from laptops and server farms with great networking and tons of computing power to mobiles and connected devices which are often in remote locations, with less reliable networking and considerably less computing power.
The same three things are important to developers at AWS and many of our customers. After a thorough evaluation, we have selected MXNet as our deep learning framework of choice , where we plan to use it broadly in existing and upcoming new services.
As part of that commitment, we will be actively promoting and supporting open source development through code contributions (we've made quite a few already), improving the developer experience and documentation online and on AWS, and investing in supporting tools for visualization, development, and migration from other frameworks.
Background on MXNet
MXNet is a fully featured, flexibly programmable, and ultra-scalable deep learning framework supporting state of the art in deep learning models, including convolutional neural networks (CNNs) and long short-term memory networks (LSTMs). MXNet has its roots in academia and came about through the collaboration and contributions of researchers at several top universities. Founding institutions include the University of Washington and Carnegie Mellon University.
"MXNet, born and bred here at CMU, is the most scalable framework for deep learning I have seen, and is a great example of what makes this area of computer science so beautiful - that you have different disciplines which all work so well together: imaginative linear algebra working in a novel way with massive distributed computation leading to a whole new ball game for deep learning. We're excited about Amazon's investment in MXNet, and can't wait to see MXNet go from strength to strength" Andrew Moore – Dean of Computer Science at Carnegie Mellon University.
The efficiency by which a deep learning framework scales out across multiple cores is one of its defining features. More efficient scaling allows you to significantly increase the rate at which you can train new models, or dramatically increase the sophistication of your model for the same amount of training time.
This is an area where MXNet shines: we trained a popular image analysis algorithm, Inception v3 (implemented in MXNet and running on P2 instances), using an increasing number of GPUs. Not only did MXNet have the fastest throughput of any library we evaluated (as measured by the number of images trained per second), but the throughput rose by almost the same rate as the number of GPUs used for training (with a scaling efficiency of 85%).
Developing With MXNet
Efficient Models & Portability In MXNet
Learn more about MXNet
We're excited about MXNet. If you would like to learn more, you can check out the MXNet home page, or GitHub repository for more information, and can get started right now, using the Deep Learning AMI, or on your own machine. We'll also be hosting a Machine Learning "State of the Union" and a series of breakout sessions and workshops on using MXNet at AWS re:Invent on November 30th at the Mirage Hotel in Las Vegas.
It's still day one for this new era of machine intelligence; in fact, we probably haven't even woken up and had our first cup of coffee yet. With tools like MXNet (and the other deep learning frameworks), and services such as EC2, it's going to be an exciting time.
Previously, I wrote about Amazon QuickSight, a new service targeted at business users that aims to simplify the process of deriving insights from a wide variety of data sources quickly, easily, and at a low cost. QuickSight is a very fast, cloud-powered, business intelligence service for the 1/10th the cost of old-guard BI solutions. Today, I am very happy to announce that QuickSight is now generally available in the N. Virginia, Oregon, and Ireland regions.
When we announced QuickSight last year, we set out to help all customers—regardless of their technical skills—make sense out of their ever-growing data. As I mentioned, we live in a world where massive volumes of data are being generated, every day, from connected devices, websites, mobile apps, and customer applications running on top of AWS infrastructure. This data is collected and streamed using services like Amazon Kinesis and stored in AWS relational data sources such as Amazon RDS, Amazon Aurora, and Amazon Redshift; NoSQL data sources such as Amazon DynamoDB; and file-based data sources such as Amazon S3. Along with data generated in the cloud, customers also have legacy data sitting in on-premises datacenters, scattered on user desktops, or stored in SAS applications.
There’s an inherent gap between the data that is collected, stored, and processed and the key decisions that business users make on a daily basis. Put simply, data is not always readily available and accessible to organizational end users. The data infrastructure to collect, store, and process data is geared primarily towards developers and IT professionals whereas insights need to be derived by not just technical professionals but also non-technical business users. Most business users continue to struggle to answer key business questions such as, “Who are my top customers and what are they buying?”, “How is my marketing campaign performing?”, and “Why is my most profitable region not growing?” While BI solutions have existed for decades, customers have told us that it takes an enormous amount of time, IT effort, and money to bridge this gap.
The reality is that many traditional BI solutions are built on top of legacy desktop and on-premises architectures that are decades old. They require companies to provision and maintain complex hardware infrastructure and invest in expensive software licenses, maintenance fees, and support fees that cost upwards of thousands of dollars per user per year. They require teams of data engineers to spend months building complex data models and synthesizing the data before they can generate their first report. To scale to a larger number of users and support the growth in data volume spurred by social media, web, mobile, IoT, ad-tech, and ecommerce workloads, these tools require customers to invest in even more infrastructure to maintain performance. Finally, their complex user experiences are designed for power users and not suitable for the fast-growing segment of business users. The cost and complexity to implement, scale, and use BI makes it difficult for most companies to make data analysis ubiquitous across their organizations.
Enter Amazon QuickSight
QuickSight is a cloud-powered BI service built from the ground up to address the big data challenges around speed, complexity, and cost. QuickSight puts data at the fingertips of your business users in an easy-to-use user interface and at one-tenth the cost of traditional BI solutions, even if that data is scattered across various sources such as Amazon Redshift, Amazon RDS, Amazon S3, or Salesforce.com; legacy databases running on-premises; or even user desktops in Microsoft Excel or CSV file formats.
Getting started with QuickSight is simple. Let’s walk through some of the core experiences of QuickSight that make it so easy to set up, connect to your data sources, and build visualizations in minutes.
Powered by Innovation
QuickSight is built on a large number of innovative technologies to get a business user their first insights fast. Here are a few of the key innovations that power QuickSight:
SPICE: One of the key ingredients that makes QuickSight so powerful is the Super-fast, Parallel, In-memory Calculation Engine (SPICE). SPICE is a new technology built by the same team that created technologies such as DynamoDB, Amazon Redshift, and Amazon Aurora. It is the underlying engine that allows QuickSight to deliver blazing fast response times on large data sets. SPICE sits between the user interface and the data source and can rapidly ingest all or part of the data into its fast, in-memory, columnar-based data store that’s optimized for analytical queries. SPICE is cloud-native, which means that customers don’t need to provision, manage, or scale infrastructure manually. Data is automatically replicated across multiple Availability Zones for redundancy and also backed up to S3 for durability. This allows us to enable organizations to reliably and securely scale to support thousands of users who can all perform fast, interactive analysis across a wide variety of AWS data sources.
Auto-discovery: One of the challenges with BI is discovering and accessing the data. As a native offering from AWS, QuickSight comes deeply integrated with AWS data sources such as Amazon Redshift, RDS, and S3. For instance, QuickSight auto-discovers all RDS instances and Amazon Redshift clusters to which any logged-in user has access. Customers can visualize their data by picking a table and then getting to a visualization in just a few clicks. In addition to AWS data sources, QuickSight also lets customers connect to third-party databases running on Amazon EC2 or on-premises and popular business applications like Excel and Salesforce.
AutoGraph: Picking the right visualization is not easy, and there is lot of science behind it. For instance, optimal visualization depends on various factors: the type of data field selected (“Is it time, number, or string?”), cardinality of the data (“Does this field have only 4 unique values or 1 million values?”), and number of data fields that you are trying to visualize. While QuickSight supports multiple graph types (e.g., bar charts, line graphs, scatter plots, box plots, pie charts, and so on), one of the things we have tried to simplify is a capability that automatically picks the right visualization for selected data using AutoGraph. Users pick the data fields to visualize and QuickSight automatically selects the most optimal visual type.
Collaboration and sharing of live analytics: Users often want to slice and dice their data and share it in various ways. With QuickSight, you can collaborate on analyses, which are visual explorations of your data, and allow others to modify the analyses in any way. You can also share your analyses as read-only dashboards and allow your viewers to interact and filter the visualizations without modification. QuickSight lets you combine visualizations into guided tours, or stories, that you can share with other users to tell the story of your data.
What our customers are saying about QuickSight
In the past months, thousands of AWS customers participated in the preview of QuickSight, including global enterprises and startups from a range of industries. Many worked closely with the team to provide early feedback and helped us rapidly iterate on the product. I am pleased to share some of the positive feedback from our preview customers like MLB Advanced Media, Infor, and Hotelbeds.com.
MLB Advanced Media (MLBAM) is a digital media and content infrastructure provider that powers an ever-growing number of massively popular media and entertainment properties. Brandon SanGiovanni, who is a traffic manager at MLBAM, tells us that QuickSight made it easy for them to explore and analyze their data in a fraction of time expected and provided them with a comprehensive view of their business without being constrained by pre-built dashboards and metrics.
Infor is a business application provider with more than 90,000 customers and 58 million cloud users. Steve Stahl, who is a Senior BI Development Manager at Infor, tells us that they found QuickSight’s SPICE engine to be fast and let them easily and quickly process and visualize datasets on RDS and Amazon Redshift. They’ve been using QuickSight during preview to analyze their customer data and look forward to continuing to use it after launch.
Miguel Iza is the Head of Data and Analytics at Hotelbeds, a global distributor of accommodations serving more than 185 destination countries worldwide and more than 25 million room nights annually. He tells us that QuickSight simplifies the way their users access data to perform self-service analysis and share insights with other. They plan to adopt QuickSight for their new data solution and looks forward to QuickSight democratizing business analytics in their company.
How you can get started
I’m excited that QuickSight is now generally available in N. Virginia, Oregon, and Ireland, with other regions coming soon. Get started by signing up for free at Amazon QuickSight, with 1 user and 1 GB of SPICE capacity. For more details, see Amazon QuickSight Now Generally Available: Fast, Easy to Use Business Analytics for Big Data on the AWS Blog.
On September 29, 2016, Amazon announced the Alexa Prize, a $2.5 million university competition to advance conversational AI through voice. We received applications from leading universities across 22 countries. Each application was carefully reviewed by senior Amazon personnel against a rigorous set of criteria covering scientific contribution, technical merit, novelty, and ability to execute. Teams of scientists, engineers, user experience designers, and product managers read, evaluated, discussed, argued, and finally selected the twelve teams who would be invited to participate in the competition.
Today, we’re excited to announce the 12 teams selected to compete with an Amazon sponsorship. In alphabetical order, they are:
- Carnegie-Mellon University: CMU Magnus
- Carnegie-Mellon University: TBD
- Czech Technical University, Prague: eClub Prague
- Heriot-Watt University, UK: WattSocialBot
- Princeton University: Princeton Alexa
- Rensselaer Polytechnic Institute: BAKAbot
- University of California, Berkeley: Machine Learning @ Berkeley
- University of California, Santa Cruz: SlugBots
- University of Edinburgh, UK: Edina
- University of Montreal, Canada: MILA Team
- University of Trento, Italy: Roving Minds
- University of Washington, Seattle: HuskyBot
These teams will each receive a $100,000 research grant as a stipend, Alexa-enabled devices, free Amazon Web Services (AWS) services to support their development efforts, access to new Alexa Skills Kit (ASK) APIs, and support from the Alexa team. Teams invited to participate without sponsorship will be announced on December 12, 2016.
We have challenged these teams to create a socialbot, a conversational AI skill for Alexa that converses engagingly and coherently with humans for 20 minutes on popular topics and news events such as Entertainment, Fashion, Politics, Sports, and Technology. This seemingly intuitive task continues to be one of the ultimate challenges for AI.
Teams will need to advance several areas of conversational AI including knowledge acquisition, natural language understanding, natural language generation, context modeling, common sense reasoning, and dialog planning. We will provide students with data and technical support to help them tackle these problems at scale, and live interactions and feedback from Alexa’s large user base to help them test ideas and iterate their algorithms much faster than previously possible.
As teams gear up for the challenge, we invite all of you to think about what you’d like to chat with Alexa about. In April, you and millions of other Alexa customers will be able to test the socialbots and provide feedback to the teams to help them create a socialbot you’ll want to chat with every day. Your feedback will also help select the finalists. In the meantime, follow the #AlexaPrize hashtag and bookmark the Alexa Prize site for updates.