Are Companies Really Using Apache Hadoop?
These days, it seems impossible to have a discussion about Big Data without mentioning Apache Hadoop, the open-source framework that allows firms to run data applications on large hardware clusters. Despite the chatter, though, a handful of recent surveys suggest companies are slow to actually adopt the platform. For example, in a new Big Data Analytics Market Study from Dresner Advisory Services, only 17 percent of responding companies said they stored and managed data within a Hadoop ecosystem; another 50 percent suggested they would adopt a Big Data platform at some point in the future. "Big Data is a very popular topic, in part due to the high volume of discussion and debate within the supplier, analyst, and media communities,” Howard Dresner, founder and chief research officer at Dresner Advisory Services, wrote in a statement accompanying the survey. “While awareness is high across the board, more importance is assigned from suppliers, and we see a fairly substantial lag in current use and near term adoption plans within organizations today." Dresner’s survey isn’t the first to question the true rate of Hadoop adoption. Earlier this year, a similar survey from research firm Gartner found that only 26 percent of corporate respondents had deployed, piloted, or experimented with Hadoop, while a mere 18 percent said they would invest in the technology at some point over the next two years. “Early Hadoop projects typically involve a small number of users and this no doubt keeps user populations down at this stage of the market,” Nick Heudecker, research director at Gartner, wrote in a brief research note at the time. “Moreover, the Hadoop stack remains unsuitable for simultaneous use by multiple users, also keeping numbers down. Another factor, and possibly the best explanation for the small number of Hadoop users, is the skills shortage.” Whether or not Hadoop justifies its hype, Big Data platforms remain an important technology segment for companies seeking to wrangle an ever-increasing amount of in-house data. As a Dice analysis found earlier this year, specializing in the following Big Data technologies can potentially earn you the following annual salaries: Cassandra: $128,646 MapReduce: $127,315 Cloudera: $126,816 HBase: $126,369 Pig: $124,563 Flume: $123,186 Hadoop: $121,313 Although Hadoop appeared at the bottom of that list, the platform can still earn qualified tech pros a hefty amount. And as Gartner mentioned, a possible industry-wide skills shortage could lead to a number of open jobs and contracting opportunities.