With the prevalence of Facebook, Twitter, instant messages and other user-generated content, today’s databases are growing at an exponential pace. In fact, according to Google CEO Eric Schmidt, every two days we create the same amount of information that we did from the dawn of civilization until 2003! How are organizations expected to handle all of this data?
Jesse Davis, Director of Research and Development here at DataDirect, recently participated in a webinar with Database Trends and Applications on Big Data, Analytics and Hadoop in the enterprise. During the webinar, he addressed the challenges of managing Hadoop clusters, such as:
- Tuning cluster performance
- Application development
- Managing and updating software
- Set-up / configuration
- Systems support
- Security
The main problem with Big Data is that there aren’t enough experts to analyze data in such a large scale. The volume is growing so quickly that data scientists have had to develop a new name for storage sizes over a zettabyte, which they now call a yobibyte.
So, how can you get nuggets of useful data out of vast volumes of data? The answer is the data refinery! It can collect and store raw data and then transform it into something that is easy to navigate.
To learn more about the Big Data, Hadoop and the future of analytics in the enterprise, tune into the webinar below.