back

by mark_l_watson·16y ago·view on hn ↗
I use Hadoop with custom map/reduce apps, but I am just experimenting a little with Pig, and playing with Hive is on my to-do list. I may change my mind on this, but I find the map/reduce model to be easy to work with, and development/debugging is fairly quick and easy running a single development Hadoop node.

BTW, I don't view Hadoop as only being useful for very large datasets. It seems reasonable to build automated Hadoop processing into a new application that has smaller data size requirements. You don't give up much in performance, and the extra development time is reasonable. Then you have lots of flexibility for scaling up. Also, if you only have sporadic needs to process large data sets, using Elastic MapReduce is very cost effective.