Robert Mitchell of IT World recently reported on the top trends for big data. A significant problem for big data and analytics is the storage of data. Bill Loconzolo, Intuit’s vice president of data engineering, and Dean Abbott, chief data scientist at Smarter Remarketer both reference the cloud and data lakes. Storage technology is lagging behind and Hadoop, which had originally shown promise as a storage system, is not yet a reliable system for businesses. Companies must use immature, new technology or risk being left behind in the big data race.
Technologies can now process cloud data, such as Reshift by Amazon and its BI data warehouse, BigQuery data analytics by Google, Bluemix by IBM, and Kinesis data processing by Amazon. Brian Hopkins, analyst with Forrester Research, states that big data will increasingly be processed by companies both on-site and in the cloud, simultaneously.
- Hadoop is expected to emerge as a data operating system that will manipulate, operate, analyze, and file data. Hadoop will function as a hub that is able to run different queries and operations, so it will be convenient to store data for analysis within a Hadoop system that can manage SQL, MapReduce, stream processing, and graph analytics.
- Data lakes or enterprise data hubs will hold data in a Hadoop Tools are being designed to analyze the data, but highly skilled individuals will be required to build the view of the data that is required with no pre-designed search or analytic record model. Companies are still working on capabilities such as monitoring access, securing data, and tracing the data from source to destination.
- Big data analytical accuracy is improving. Big data combined with computers can predict future patterns and behaviors valuable for marketers and manufacturers. Intuit is using Hadoop as a core processor, but is testing query tools such as Apache Spark and Spark SQL to obtain query data and graphs.
Mark Beyer, an analyst at Gartner, emphasizes the need for IT managers to allow skilled analysts to experiment with the new technologies to create new data resources and analysis tools. Once perfected, IT leaders should then decide when to roll out new systems to users.
It is exciting times for all IT professionals. Stay ahead and find the talent that can provide new IT resources to your organization. Contact an A&A Search professional for your IT staffing needs.