If you know anything about Hadoop architecture - the task seemed daunting to
us and it proved to be one of the most challenging engineering feat that we
have accomplished so far.
After almost 24 months of development, tens of thousands of lines of Java,
Scala and C++ code, multiple design iterations, several releases and dozens
of benchmarks later we have the product that can deliver real-time
performance to Hadoop with only minimal integration and no ETL required.
Backed-up by customer deployments that prove our performance claims and
validate our architecture.
Here's how we did it.
The Idea - In-Memory Hadoop Accelerator
Hadoop is based on two key technologies: HDFS for storing data, and MapReduce
for processing that data in parallel. Everything else in Hadoop itself and
the entire ecosystem coalesce around these two technologies.
Both - HDFS and MapReduce - were ... (more)
A few months ago, I spoke at the conference where I explained the difference
between caching and an in-memory data grid. Today, having realized that many
people are also looking to better understand the difference between two major
categories in in-memory computing: In-Memory Database and In-Memory Data
Grid, I am sharing the succinct version of my thinking on this topic - thanks
to a recent analyst call that helped to put everything in place
TL;DR
Skip to conclusion to get the bottom line.
Nomenclature
Let's clarify the naming and buzzwords first. In-Memory Database (IMDB) is a ... (more)
So You Want to Use JSON with the 'Internet of Things'
Here is a comprehensive article on the basics of using JSON, the Javascript
Object Notation. JSON packs lot more power into just information in a piece
of text.
JSON has become a nice connector to bridge the physical (IoT) and the Digital
worlds (Web Services).
Get this still in development plug-in (pURL) if you want to read URLs in JSON
format to send out a listenable event. This is just the beginning to create
automatons based on URL content, be it web services or something else.
http://freedomotic.com/content/plugins/purl
... (more)
The JVM issues vary from Java OutOfMemory Error to JVM Crash. Application
developers might be not completely equipped to determine the root cause of
the issue, hence DevOps can play a vital role in narrowing down the issue and
connecting the right people/team to rectify the problem.
Application developers can deploy their applications with success in a
staging or QA environment and broadcast that the application is working fine.
However, the staging or QA environment is not completely identical with
production at - least with respect to load. Due to huge load, the
application ca... (more)
It seems that about every 10 years the Internet takes on a new popular name.
In the late 90′s the term World Wide Web (www) set the standard but as a
new generation of SaaS and more connected applications came online the
Internet needed a rebranding and the Web 2.0 revolution came into favor. By
2010 the concept of Web 2.0 was almost long forgotten and the Cloud started
to dominate the popular terminology. While Cloud technically refers to an
infrastructure model for elastic scalability most people use the term to
refer to applications supported by the Internet in general.
If y... (more)
(Please refer to the following article: Oracle 12c In-Memory Database is Out
- Hardly Anybody Notices for update on Oracle 12c databases)
Contemporary large servers are routinely configured with 2TB of RAM. It is
thus possible to fit an entire average size OLTP database in memory directly
accessible by CPU. There is a long history of academic research on how to
best utilize relatively abundant computer memory. This research is becoming
increasingly relevant as databases serving business applications are heading
towards memory centric design and implementation.
If you simply place... (more)
Subscribe to Java Developer Magazine Email News Alerts
Subscribe via RSS
