Every company strives to make its business processes as lean and efficient as possible. Likewise, every company today is used to dealing with large amounts of data. And everything and everyone is now networked anyway. All you have to do is take your cell phone out of your pocket and check the last 50 e-mails.
So it's no wonder that buzzwords like the Internet of Things or Business process optimization are on everyone's lips. But what is really behind them? Where does Big Data actually? And more importantly, what do all these topics have to do with each other? This blog series aims to address these questions. The focus of this article is on a Introduction to Big Data. Part two then explains how the maturity of IoT business processes is related to the topic of big data. The third part of this blog series concludes by explaining this connection using a very specific use case.
The origin of big data
Depending on the source, different statements are made about what big data is. Essentially, this term describes Mass data, in an extremely high volume occur. At the same time, they are very Complex and inconsistent and must also be processed in the shortest possible time. An illustrative example of this would be a system that processes audio, video and text files in real time from a must process and analyze a large number of different sources.
Some of you may be thinking: "Yes, but processing large amounts of data efficiently is not a new topic. This problem has been around for as long as there have been computers." That is correct. Nevertheless, big data is different. To understand this better, you need to take a brief look at the History of the origin of this term throw.
At the very beginning of this development, until around the 1970s, were Flat Files: Data files without a standardized description or structure. A very important Further development then took place through the emergence of relational databases which still plays an important role in many areas today. de facto standard represent. With SQL and the Entity Relationship Model these were able to modeled and processed become. Other database models also developed in this environment, such as object or graph-oriented databases. With the Boom in PCs and the Internet and with the Distribution of content management or ERP systems also took the data volumes are increasing rapidly. In order to better control these, for example Data warehouses and Data marts around manage data efficiently. The Emergence of Web 2.0 also brought with it a growing volume of videos, music and images and presented a further challenge, making issues such as Virtualization and cloud computing came more and more into focus.
So is big data really new or just a further development of existing technologies?
The answer is: It is both. This is because many approaches from the big data environment, such as non-relational databases, have existed for some time. However, the existing approaches alone are not effective when it comes to, for example evaluate several petabytes of data for a specific problem (leaving aside the expensive use of mainframe systems).
What is new is that, for example Memory and CPU have become extremely affordable are. The Maturity of many technologies - virtualization and cloud computing - has changed significantly. further developed. As a result, it is now easier than ever to combine infrastructures(aaS), platforms(aaS) and software(aaS) in almost any way. can be scaled as required. And this is precisely where new frameworks such as Hadoop come in. Through the targeted use of existing and new technologies/algorithms, they make it possible to efficiently process and analyze huge amounts of data on computer clusters.
As a consequence, this has of course Different effects to the Business models of companies. All the more so when a company with networking and IoT. The second part of this blog series will therefore go into more detail and explain how exactly the connection between the Maturity level of IoT business processes and big data is.
In the second part of this blog series the connection between big data and the maturity level of IoT business processes will be explained.
doubleSlash offers a very exciting and interesting Big Data workshop for all those who would like to delve further into the topic of Big Data and deepen their knowledge of certain technologies in this field.
Find out more about IoT services and connected products here
