Over the past few years there have been numerous eulogies given for Hadoop – the powerful, open-source framework for storing and processing data named after a toy elephant. Of course, one could argue that Hadoop isn’t all dead – just mostly dead. Still, many organizations who invested heavily in the Hadoop ecosystem have found themselves at a crossroads, wondering what life after Hadoop is like and what lies ahead. This article addresses life after Hadoop and lays out a strategy for organizations entering the post-Hadoop era.
Remembering the elephant
For many organizations, life with Hadoop was pretty good and it provided some real muscle for handling large amounts of unstructured data. For some, SQL-on-Hadoop solutions even helped offload work from even more complex (and expensive) data warehouses. That said, the care and feeding of Hadoop, like other elephants, was, well… not trivial – especially when it came time for baths and cleaning out the pen. I’ll not digress any further with my personal pet trauma but suffice to say that living with large animals had its downsides.