Thursday, April 2, 2009

With Hadoop, Amazon Adds A Web-Scale Data Processing Engine To Its Cloud Computer

With Hadoop, Amazon Adds A Web-Scale Data Processing Engine To Its Cloud Computer: "This is actually a big deal because it allows developers to better take advantage of the massive computing power Amazon has to offer and create applications which process huge reservoirs of data (conveniently stored in Amazon S3) in parallel. MapReduce is the name of the data processing framework Google created to index and search the Web. It literally breaks up huge computational tasks and spreads them to different servers. This is called mapping the data. Once each processor is done with its portion of the math problem, it sends the result back so that all the different partial answers can be combined and then “reduced” into one final answer."

No comments:

Post a Comment