IBM Big Data Platform

IBM is unique in having developed an enterprise class big data platform that allows you to address the full spectrum of big data business challenges.

What is Big Data?

We live in the information age. Each digital process to produce or exchange some data. All the time the data is being generated all around us. Systems, sensors, mobile device generates and transmits data. Data obtained at an alarming rate generated. "Every day, we create 2.5 quintillion bytes of data - so much that 90% of the data in the world today has been created in the last two years of this data comes from everywhere: sensors used. gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and the phone's GPS signal, to name a few, while the data is "big data."

Handling Huge Amount of Data

There are huge volumes of data in the world:

• From the beginning of recorded time until 2003, We created 5 billion gigabytes of data.
• In 2011, the same amount was created every two days
• In 2013, the same amount of data is created every 10 minutes.

Big data spans three dimensions: Volume, Velocity and Variety

Volume: Enterprises are awash with ever-growing data of all types, easily amassing terabytes—even petabytes—of information.

• Turn 12 terabytes of Tweets created each day into improved product sentiment analysis
• Convert 350 billion annual meter readings to better predict power consumption

Velocity: Sometimes 2 minutes is too late. For time-sensitive processes such as catching fraud, big data must be used as it streams into your enterprise in order to maximize its value.

• Scrutinize 5 million trade events created each day to identify potential fraud
• Analyze 500 million daily call detail records in real-time to predict customer churn faster
• The latest I have heard is 10 nano seconds delay is too much.

Variety: Big data is any type of data - structured and unstructured data such as text, sensor data, audio, video, click streams, log files and more. New insights are found when analyzing these data types together.

• Monitor 100’s of live video feeds from surveillance cameras to target points of interest
• Exploit the 80% data growth in images, video and documents to improve customer satisfaction

The data generated via these digital system have certain patterns that can benefit decision making. To extract value from big data, we need optimal processing power and great analytical skills and abilities.

Advantages over other Database Systems

• Traditional RDBMS systems cannot handle the data.
• Since the data is not fully structured. (Variety)
• Volume is quite large (Volume)
• Scalability and performance is not achieved.

Finally the Big Data is

• `Big- Data’ is similar to ‘Small-data’ but bigger but having data bigger it requires different approaches:
• Techniques, tools, architecture With an aim to solve new problems Or old problems in a better way

Whom Does Big Data matter

• Research Community

With Big data Research community can track of latest researches and database of researcher and fields of research.

• Business Community

New tools, new capabilities, new infrastructure, new business models etc., With Big Data Business communities Perform Data Analysis on annual reports , on their business activities to grow business.

• Health Care

With Big Data Medical professionals can improve care and reduce costs by extracting relevant clinical information from vast amounts of data to better understand the past and predict future outcomes.

• Customer Service

With Big Data Customer representatives can use data to get a more holistic view of their customers. Understanding their likes and dislikes in real-time in order to resolve a problem of capitalize on happy clients faster.

• Insurance

With Big Data an insurance or citizen service provider can apply advanced analytics to data and detect fraud quickly , before funds are paid out.

Revenues Generation using Big Data

How Revenues grow exponentially every Year. Below is the statistics from US Market

The Social Layer in an Instrumented Interconnected World

The social layer is showing the use of big data by all these social networks and large companies on daily basis. lets have a look on below given analysis

Types of tools typically used in Big Data Scenario

Where the processing is hosted?

• Distributed server/cloud

Where data is stored?

• Distributed Storage (eg: Amazon s3)

Where is the programming model?

• Distributed processing (Map Reduce)

How data is stored and indexed?

• High performance schema free database

What operations are performed on the data?

• Analytic/Semantic Processing (Eg. RDF/OWL)

Dealing With big Data When the operations on data are complex:

• Eg. Simple counting is not a complex problem.
• Modeling and reasoning with data of different kinds can get extremely complex

Good news with big-data:

Often, because of the vast amount of data, modeling techniques can get simpler (e.g., smart counting can replace complex model-based analytics)……as long as we deal with the scale.

Time For Thinking About Data

What do you do with the data?

Lets take an example: “From application developers to video streamers, organizations of all sizes face the challenge of capturing, searching, analyzing, and leveraging as much as terabytes of data per second—too much for the constraints of traditional system capabilities and database management tools.”

Why Big Data?

Key enablers for the appearance and growth of ‘Big-Data’ are:

• Increase in storage capabilities
• Increase in processing power
• Availability of data