Running a business at the scale of Amazon, we often have to solve problems that no other company has faced before. The disadvantage of this is that there is no “how to” guide for us—a lot is unknown. However, the advantage is that when we solve a new problem, it’s an opportunity to reinvent our services and create new benefits for our customers. Indeed, we have created some of our most innovative and successful ideas when we have entered unchartered territory.
When you’re a customer-centric company, you often find yourself in the great unknown because customers will always want more and better. You will need to invent on their behalf. A great example of this approach to innovation and problem solving is the creation of the AWS Nitro System, the underlying platform for our EC2 instances.
After years of optimizing traditional virtualization systems to the limit, we knew we had to make a dramatic change in the architecture if we were going to continue to increase performance and security for our customers. This realization forced us to rethink everything and became the spark for our creating the Nitro System, the first infrastructure platform to offload virtualization functions to dedicated hardware and software. Now, with the Nitro System, we can offer the best price performance in the cloud, the most secure environment, and a faster pace of innovation.
I have always been very fortunate to meet our AWS customers where they have most impact, at their customers. Many of these AWS customers are solving really hard human problems, in ways that is extremely inspiring for any builder, like me. That became the inspiration for the Now Go Build series that chronicles my conversations with these innovators and their customers.
In the first season, we had a wonderful diverse group of young businesses showing their impact on the world. From providing identity to smallholder farms in Indonesia to healthcare in Brazil and conservation in South Africa.
The second season that we are launching today will again have four wonderful stories. In the coming weeks I will go into more detail, but these are the first three:
A few days ago I was fortunate to pick up a copy of a book that had a major impact on my early career as kernel engineer;
The Design and Implementation of the 4.3 BSD UNIX Operating System by Samuel J. Leffler, Marshall Kirk McKusick, Michael J. Karels and John S. Quarterman.
It was the first authoritative description of Berkeley UNIX, its design and implementation. The book covers the internal structure of the 4.3 BSD systems and the concepts, data structures and algorithms used in implementing the system facilities. But most importantly it was written by practitioners and builders and as such gave insights that academic text book would never give you.
In those days I was doing an internship at NIKHEF who were still using a collection of PDP 11s and one of my tasks was to get BSD2.9 to run on them. Lots of late nights and head scratching, but got it done eventually. I did learn how to boot from tape, over and over again (Zen!!). When I returned to school, they were about to decommission a PDP 11. I convinced them to put it in a old (big) cleaning closet, upgrade the power to the room, and I went right back to building out my BSD kernel expertise. I started late at Computer Science (28) but worked hard to catch up by getting my hands dirty.
When I posted on twitter I found of the book, many of our peers came up with a list of other books I had also read from that era.
One of the most rewarding parts of my job is getting to watch different industries implement new technologies that improve and transform business operations. Manufacturing, in particular, has always captivated my attention in this respect. When I think about how Amazon’s globally connected distribution network has changed in the last decade alone, it’s incredible. From the Internet of Things (IoT) to Artificial Intelligence (AI) and task automation to predictive maintenance technology, the advancements in this space are creating a world of new opportunity.
But this is complicated by that fact that many manufacturers have been around for decades or longer. Some of their equipment was designed before the internet even existed. If replacing this equipment isn’t an option, how do these manufacturers begin their journey to modern manufacturing? The choice of what to embrace and where to start can be daunting.
Ultimately, the reason for adopting any new technology in manufacturing is usually to achieve one or more of the following objectives: produce more, increase safety, or increase quality—and all at a lower cost. The good news is that the most important thing a manufacturer needs to accomplish with any of these objectives is something they already have. It’s something they’ve had since the moment they opened their doors, whether that was yesterday or 100 years ago: data.
The global healthcare pandemic has been like nothing many of us in Europe have ever known. During this time, many organizations have been contemplating their role in the COVID-19 crisis, and how they can best serve their communities. I can tell you it has been no different for us at Amazon Web Services (AWS). We are focused on where we can make the biggest difference, to help the global communities in which we all live and work. This is why today we are announcing that the AWS Europe (Milan) Region is now open. The opening of the AWS (Milan) Region demonstrates our ongoing commitment to the people of Italy and the long-term potential we believe there is in the country.
La maggior parte di noi, in Europa, non aveva mai conosciuto prima una pandemia globale come quella in corso. Durante questo periodo, molte organizzazioni stanno riflettendo sul proprio ruolo nella crisi COVID-19 e su quale può essere il modo migliore per supportare la propria comunità. Posso dirvi che per noi di Amazon Web Services (AWS) non è stato diverso. Ci siamo concentrati su come e dove avremmo potuto fare la differenza più grande aiutando le comunità globali in cui viviamo e lavoriamo. Con questo obiettivo in mente, oggi annunciamo l'apertura della Regione AWS Europe (Milano). Il lancio della Regione AWS in Italia conferma il nostro costante impegno per gli italiani e rafforza ulteriormente il nostro sostegno al grande potenziale del paese.
As COVID-19 has disrupted life as we know it, I have been inspired by the stories of organizations around the world using AWS in very important ways to help combat the virus and its impact. Whether it is supporting the medical relief effort, advancing scientific research, spinning up remote learning programs, or standing-up remote working platforms, we have seen how providing access to scalable, dependable, and highly secure computing power is vital to keep organizations moving forward. This is why, today, we are announcing the AWS Africa (Cape Town) Region is now open.
On March 16, 2020, at 9:26 PM, I received an urgent email from my friend DJ Patil, former White House Chief Data Scientist, Head of Technology for Devoted Health, a Senior Fellow at the Belfer Center at the Harvard Kennedy School, and Advisor to Venrock Partners. You don’t get that many titles after your name unless you’re pretty good at something. For DJ, that “something” is math and computer science.
DJ was writing to me from the California crisis command center. He explained that he was working with governors from across the country to model the potential impact of COVID-19 for scenario planning. He wanted to help them answer critical questions, like “How many hospital beds will we need?” and “Can we reduce the spread if we temporarily close places where people gather?” and “Should we issue a shelter-in-place order and for how long?” While nobody can predict the future, modeling the virus with all the factors they did know was their best shot at helping leaders make informed decisions, which would impact hundreds of thousands of lives.
Back when Jeff Bezos filled orders in his garage and drove packages to the post office himself, crunching the numbers on costs, tracking inventory, and forecasting future demand was relatively simple. Fast-forward 25 years, Amazon's retail business has more than 175 fulfillment centers (FC) worldwide with over 250,000 full-time associates shipping millions of items per day.
Amazon's worldwide financial operations team has the incredible task of tracking all of that data (think petabytes). At Amazon's scale, a miscalculated metric, like cost per unit, or delayed data can have a huge impact (think millions of dollars). The team is constantly looking for ways to get more accurate data, faster.
That's why, in 2019, they had an idea: Build a data lake that can support one of the largest logistics networks on the planet. It would later become known internally as the Galaxy data lake. The Galaxy data lake was built in 2019 and now all the various teams are working on moving their data into it.
A data lake is a centralized secure repository that allows you to store, govern, discover, and share all of your structured and unstructured data at any scale. Data lakes don't require a pre-defined schema, so you can process raw data without having to know what insights you might want to explore in the future. The following figure shows the key components of a data lake.
Have you ever received a call from your bank because they suspected fraudulent activity? Most banks can automatically identify when spending patterns or locations have deviated from the norm and then act immediately. Many times, this happens before victims even noticed that something was off. As a result, the impact of identity theft on a person's bank account and life can be managed before it's even an issue.
Having a deep understanding of the relationships in your data is powerful like that.
Consider the relationships between diseases and gene interactions. By understanding these connections, you can search for patterns within protein pathways to find other genes that may be associated with a disease. This kind of information could help advance disease research.
The deeper the understanding of the relationships, the more powerful the insights. With enough relationship data points, you can even make predictions about the future (like with a recommendation engine). But as more data is connected, and the size and complexity of the connected data increases, the relationships become more complicated to store and query.
Subscribe to this weblog's