Big data integration tool targets Hadoop skills gap

Channel partners can get a free taste of a big data integration tool from Information Builders that could help get them over the Hadoop talent gap.

The software company last week debuted iWay Big Data Integrator, Developer's Edition, which it is offering at no cost for six months. The tool could boost the productivity of channel partners[4] working on big data projects and could help other partners launch businesses[5] in the growing market, industry executives said. The limited edition for developers is available here[6]. Big Data Integrator, an evolution of the iWay Hadoop Data Manager, initially launched in March 2017 on the Microsoft Azure Marketplace Cloud.

According to Information Builders, Big Data Integrator simplifies Hadoop[7], providing a graphical interface for Hadoop and Apache Spark[8] data ingestion and transformation that reduces the need for coding. Hadoop, a software framework for storing and processing large data sets, provides a foundation for developing big data applications. Apache Spark is a data processing engine that supports in-memory computing[9] and can run on Hadoop.

Jake Freivald[10], vice president of product marketing at Information Builders, said harnessing the power of Hadoop is difficult and involves complicated programming, adding the task of managing Hadoop environments is somewhat arduous as well.

Potential channel impact

A stumbling block for organizations, including channel players such as value-added resellers (VARs)[11], seeking to create big data applications via Hadoop has been the need to acquire specialized programming talent, Freivald noted. A VAR selling big data software, for example, might need to hire people to customize the application for individual customers.

"But the people they hire are very technical and don't necessarily understand the business requirements for customization," Freivald said. Such companies find themselves "spending more on highly paid developers who are relatively narrow in what they can do."

Big Data Integrator, however, targets more mainstream developers who understand data and business requirements, but may not be experts in Hadoop or related technologies such as Sqoop and Flume[12]. Information Builders' big data integration tool incorporates Sqoop for data replication, import and export and Flume for gathering and ingesting unstructured data. Big Data Integrator also uses Information Builders' iWay Service Manager for ingesting non-Hadoop data from sources such as internet of things sensors.

Freivald said Big Data Integrator's graphical interface displays available data sources and reduces coding and debugging time. He said developers "can do what they need to do, instead of spending a lot of time on technical details."

Big data integration: Tool addresses skills shortage

David Menninger[13], an analyst at Ventana Research who focuses on data, information and analytics technologies, suggested Information Builders' integration platform addresses the shortfall in Hadoop expertise.

"Our research shows that there is a real lack of Hadoop skills in the market still," he said, noting that one in six organizations reports having sufficient Hadoop skills.

Menninger said Big Data Integrator can improve the efficiency of consulting firms focusing on Hadoop and big data. In addition, it can also help those service providers transfer big data projects to their customers. He said enterprise customers usually wish to assume responsibility for a project once the consultant[14] has completed the job.

"They typically don't want to bring on a consulting firm forever," Menninger said. "These tools make it a whole lot easier to hand [a project] off."

That's because the customer organization can more readily maintain a big data application's code on its own, since Information Builders' big data integration tool doesn't require Sqoop expertise or other Hadoop-related talent, he said.

Freivald, meanwhile, said the data integration platform can also help channel partners move a big data application from one environment to another. For example, a VAR that built an application that uses Cloudera's Hadoop distribution[15] may want to replicate that app for a customer that uses the MapR Technologies Inc.[16] Hadoop distribution. "The Big Data Integrator-based processes would remain essentially the same," Freivald said.

In addition, Big Data Integrator can potentially lower the Hadoop skill barrier that prevents consulting firms and other channel companies from entering the big data sector, according to Menninger.

"This is a way for them to get involved in the market and participate," he said.

Channel companies and other organizations may consider Big Data Integrator a useful tool for simplifying data ingestion in Hadoop[17] environments. But Menninger said an exclusive focus on an integration platform misses the bigger picture, citing Information Builders' portfolio of data management and analytics products.

"It is really one tool in an arsenal that is much broader than the big data ingestion process," he said of Big Data Integrator. "Hopefully, [customers] recognize more issues than just ingesting the data. If all you are doing is ingesting the data you are probably setting yourself up for failure."

 

References

  1. ^ partners (www.techtarget.com)
  2. ^ Terms of Use (searchitchannel.techtarget.com)
  3. ^ Privacy Policy (www.techtarget.com)
  4. ^ channel partners (searchitchannel.techtarget.com)
  5. ^ partners launch businesses (searchitchannel.techtarget.com)
  6. ^ available here (www.informationbuilders.com)
  7. ^ Hadoop (searchcloudcomputing.techtarget.com)
  8. ^ Apache Spark (searchitoperations.techtarget.com)
  9. ^ in-memory computing (searchcloudapplications.techtarget.com)
  10. ^ Jake Freivald (www.informationbuilders.com)
  11. ^ value-added resellers (VARs) (searchitchannel.techtarget.com)
  12. ^ technologies such as Sqoop and Flume (searchcloudcomputing.techtarget.com)
  13. ^ David Menninger (davidmenninger.ventanaresearch.com)
  14. ^ consultant (searchitchannel.techtarget.com)
  15. ^ Cloudera's Hadoop distribution (searchdatamanagement.techtarget.com)
  16. ^ MapR Technologies Inc. (searchstorage.techtarget.com)
  17. ^ data ingestion in Hadoop (searchdatamanagement.techtarget.com)

Comments

Post a Comment

Popular posts from this blog

New Article Posted: Seven Things You Should Do in Your AMS

Given how advanced and thorough the modern association management software (AMS) system has become, there are many opportunities for your organization to leverage it. Here are seven ways I believe every association should be using their AMS. Click here to read the rest of the article . Of course, if you’re already on my  announcements list , you would have received notice of this new article right in your email box. Not signed up yet?  Click here to sign up . Go to the article
Read more

10 Common Mistakes Biblical Counselors Sometimes Make, Part 8

Image
A Word from Bob: You’re reading Part 8 of a 10-part blog series on 10 Common Mistakes Biblical Counselors Sometimes Make . Mistake #1: We Elevate Data Collection Above Soul Connection . Mistake #2: We Share God’s Eternal Story Before We Listen Well and Wisely to Our Friend’s Earthly Story . Mistake #3: We Talk at Counselees Rather Than Exploring Scriptures with Counselees . Mistake #4: We Practice Half-Biblical Counseling If We Address Sin but Neglect Suffering . Mistake #5: We Fail to Follow the Trinity’s Model of Comforting Care . Mistake #6: We Tend to View People One-Dimensionally . Mistake #7: We Devalue Emotions Rather Than Seeing Emotions as God’s Idea . Mistake #8: We Minimize the Complexity of the Soul-Body Interconnection Because we are soul physicians and soul care-givers, our tendency is to focus on matters of the soul. Most biblical counselors are not trained in the medical field, so we wisely avoid offering medical guidance. Yet, that appropr...
Read more