RDD : Resilient Distributed Dataset

Apache Spark has a key feature known as Resilient distributed datasets(RDD’s). The data structure available in Spark.

RDD’s are fault tolerant and can be operated in parallel.

RDD’s can be created by:-

  • Parallel execution of collection available in driver program.
  • By referencing dataset in an external storage system such as HDFS, HBase or data source supporting Hadoop input format.

RDD’s support two types of operations:

  1. Transformations
  2. Actions

Transformations generate a new dataset from already available dataset.

Actions return values to driver after working on dataset.

Unlike Map-Reduce, Spark does not carry out the complete life-cycle of data processing for task completion.

Spark is efficient and operates on datasets only when results are required by driver program.

CAP Theorem and Distributed System

CAP Theorem is also known as Brewer’s Theorem.

CAP refers to Consistency, Availability and Partition tolerance.

  • Consistency – All nodes in a cluster have same data at any particular time.
  • Availability – It is the end result of request i.e. success or failure.
  • Partition tolerance – As per this feature, the system continues to operate despite any hardware or network failures.

According to CAP Theorem any distributed system can exhibit only 2 out of the above 3 features.

Most distributed systems experience network failures, hence partition tolerance has to be met by distributed systems.

For the remaining one condition to be met:

Databases designed based on ACID properties choose consistency over availability.

Note: Consistency here not to be confused with the one in ACID.

On the other hand, databases designed based on BASE properties choose availability over consistency.

HBase

In case of HBase, consistency and partition tolerance are met. But HBase does not offer availability. HBase is a NoSQL database.

Law of Action : Economics and Startups

Newton’s 3rd law states :-

“For every action there is an equal and opposite reaction.”

A simple action such as walking proves the validity of the above statement, when we push our feet backwards on the ground, our body moves forward.

Does this mean all actions are bound to this law ?

Yes, if not all actions, most of them are bound to this law.

A ball pushed downwards on water surface bounces back due to the buoyancy exhibited by water.

A ball thrown up must come back because of gravity.

This law holds good even in the world of Economics and Startups, wonder how ?

Trade markets, Financial results, traction all these are driven by a law of action.

Cause and effect is a normalizing feature associated in Economics.

But is the world ready for a level playing field ?

No it isn’t and it will never be.

Every product created which exists is bound by this law of action.

Action and Reaction constitute a single entity. A pure play source of energy being transferred between two states.

A Startup in order to succeed must have necessary traction, if traction is the result, there needs to be a action in-order to achieve the desired result.

A chain of individual components and the synergy generated by this chain is what makes a startup succeed.

Everything that matters must be active in order to succeed. Being active at the right moment results in stupendous success.

To get the timing right is one thing and to get the time going with a thing is another. The former is the action and the latter is its reaction.

Mysuru Dasara : State Festival of Karnataka

The legacy of Mysuru Dasara festival dates back to four centuries ago. The 10 days of celebration in Mysuru city represents the tradition and rich cultural heritage of Karnataka. During these 10 days Mysuru city and Mysuru palace is decorated and grandeur of the place is reclaimed.

The story of Mysuru getting it’s name gives a picture of the place and festival.

Mysuru gets it’s name from a asura(demon) known as Mahishasura.

Folktales of this place or the history here says that long ago in Mysuru lived Mahishasura who performed deep penance and requested a boon from lord Brahma.

The boon was that ‘Mahishasura should not have death’. Brahma to which replied all beings that are born must die. In turn Mahishasura asked for a boon that if he must die, let it be at the hands of a woman. To which Brahma agreed and granted the boon.

When Mahishasura became invincible, goddess Chamundi(a form of goddess Durga) was born in order to kill Mahishasura. A fierce battle ensued between Mahishasura and Chamundi before Mahishasura  was killed.

Hence, the name Mysuru for the place.

Untill 1973, the present state of Karnataka was known as Mysore state.

Chamundi hills located close to the city of mysuru is the abode of goddess Chamundi.

The state goddess Chamundi is carried on a procession by a group of elephants with Arjuna(chief elephant 2018) carrying 750 Kg golden howdah having the idol of goddess chamundi.

People from various countries and states arrive at Mysuru to experience the beauty of this occasion.

On this auspicious day, wishing everyone a happy Dasara.

Technology and Elections: Democracy taken for a ride

We live in a tech savvy world where political polarization is widely done than ever before. Technology which was meant to bring transparency and clarity is used to clog and obscure people of reality.

Elections which are a quintessential part of democracy has been diluted of it’s attempt to create true change and nation building process.

There is a reason to discuss about current trending technologies as the following avenues can create a biased opinion in the minds of voters.

Social Media

  • WhatsApp – Probably one of the most vulnerable platform to spread fake news, easy to opinionate, ability to reach out quickly to masses through groups and the power it gives for sharing content on it’s own platform and posts from other platforms. Political parties have been using this platform excessively to propitiate people by their propaganda.
  • Facebook – Biggest social media platform which literally captures every single action of it’s users on it’s platform. It can influence all it’s members through groups, friend lists, pages, Facebook Ads. Given their ability to target ads to a particular group of people, this platform has the best route to impact people.

Further, the comments on posts and the likes/reactions for a post have multiple fake profiles used by political parties to create a opinion.

  • YouTube – The leading video platform too is not spared from fake news, fake videos in recent years. Simple way of misusing the platform can be through it’s likes and dislikes button, again comments section here too can be misused. Each and every political video has 80 : 1 like, dislike ratio. Edited videos of speeches pose a threat to easier understanding for the people.
  • Twitter – Twitter on the other hand is more for niche users which can be used like a cyber war platform for different political parties. Hashtags, retweets from socially influential people and parody accounts can bemuse people. Also, the follower count of various political leaders is questionable.

The above mentioned points address the user aspects only, if dug deep dark data in back-end higher insights shall be extracted through data dumps and data generated through users. And this data is available with the platform plus the data provider for the user device. If there is a need the government in concerned jurisdiction can also access user data.

IT cell of major political parties today is equivalent to any medium size software consulting firm. It’s almost near to impossible for independent and lesser known political parties to make their presence felt on these above mentioned platforms.

The recent findings of data misuse from Cambridge Analytica from Facebook data is a single case acknowledged properly so far, there are various loopholes on all these platforms which needs to be fixed.

Television

With television channels available on mobile devices the line is thinning between conventional tv’s and internet tv’s. Here consider only conventional tv, there are several channels which operate 24X7 reporting about elections, lobbying for their desired political parties. There is little consensus expressed by these news channels to report unbiased news.

Moreover the surveys and opinion polls conducted by most news channels bring out results in favor of political parties which they approve and validate.

Newspaper / Print Media

Once a heavily dependent mode of news consumption, now reduced in it’s numbers on advent of internet and television. Yet, they are making their presence felt through online news portals these days. All newspapers in circulation today have political affiliation to some or the other political party, this mode is comparatively trustable given there is an organization to look through articles printed on the newspaper.

Unlike the other modes newspapers are on record, from ages.

Thus, with these pointers on current technologies we head towards the Karnataka assembly elections on 12th May 2018. Vote for a candidate without prejudice but with reasoning, if you don’t like any of the candidates, press NOTA.

Exercise your power to vote.

Arrays in JAVA

An array is a collection of variables which are of similar type. Arrays can be accessed with their indexes.

Arrays can be used to store numbers, list of values and other variables.

Consider the following example below :

int[ ] Bitcoin = new int[5];

Here, we are defining an array Bitcoin of integer datatype consisting of 5 Bitcoins.

We can reference each element in the array with an index, say the elements of the array are {5,50,85,22,78}. We can access the 2nd element by using Bitcoin[1]. Note indexing in Java starts at 0.

Let us look into a program to access arrays by referencing:

public class Cryptocurrency {

    public static void main(String[] args) {

        String[ ] Blockchain = { “Amsterdam”, “Berlin”, “Capetown”, “Delhi”};

        System.out.println(Blockchain[2]);

    }

}

Program to calculate length of an array.

public class Coins {

    public static void main(String[] args) {

        int[ ] Bitcoin = new int[5];

        System.out.println(Bitcoin.length);

    }

}

Program to find sum of an array.

public class Coins {

    public static void main(String[] args) {

        int [ ] Bitcoin = {5,50,85,22,78};

        int sum=0;

        for(int k=0; k<Bitcoin.length; k++) {

            sum += Bitcoin[k];

        }

        System.out.println(sum);

    }

}

In this code, the variable sum stores all the variables by adding all the elements of array. Initially the sum is assigned a value of zero. A for loop traverses through the array list because of the expression k<Bitcoin.length, that is the number of elements will be equivalent to a value less than length of the array. Remember, the indexing starts at [0] for arrays.

Printing values of an Array.

public class Coins {

    public static void main(String[] args) {

        int[ ] values = {5,50,85,22,78};

            for (int P: values) {

            System.out.println(P);

        }

    }

}

2D and 3D Arrays

Accessing 2D Array

public class Coins {

    public static void main(String[] args) {

        int[ ][ ] Bitpart = { {77, 55, 34}, {41, 54, 63} };

        int j = Bitpart[1][0];

        System.out.println(j);

    }

}

Accessing 3D Array

public class Coins {

    public static void main(String[] args) {

        int[ ][ ] Bitpart = { {17, 21, 32}, {41,70,44}, {51, 62, 75} };

        Bitpart[0][2] = 32;

        int m = Bitpart[1][0];

        System.out.println(m);

    }

}

Similarly, N dimensional arrays can also be accessed.

Source:1)THE Java™ Programming Language, Fourth Edition

       By Ken Arnold, James Gosling, David Holmes

       2) SoloLearn Online Platform.

Functions and Built-in Functions in Python

Function

A function is the simplest callable object in Python.

NOTE: For Indentation either use TAB space or 4 space characters, never both.

def tax(bill):
“””Adds 25% tax to a restaurant bill.”””
bill *= 1.25
print “With tax: %f” % bill
return bill

def tip(bill):
“””Adds 35% tip to a restaurant bill.”””
bill *= 1.35
print “With tip: %f” % bill
return bill

meal_cost = 100
meal_with_tax = tax(meal_cost)
meal_with_tip = tip(meal_with_tax)

Function Syntax

def spam():                                             /*Header*/
“”” This is spam function”””            /* Comment*/
print “Spaceship!”                            /*body*/

spam()

Call and Response

def square(n):
“””Returns the square of a number.”””
squared = n**2
print “%d squared is %d.” % (n, squared)
return squared

square(5)

Parameters and Arguments

A function can require as many parameters as you’d like, but when called the function, it should generally pass in a matching number of arguments.

def power(base, exponent):  # Add your parameters here!
result = base**exponent
print “%d to the power of %d is %d.” % (base, exponent, result)

power(20, 6)  # Add your arguments here!

Functions Calling Functions

A function can call another function.

def n_plus_one(n):
return n + 1

def couple_add(n):
return n_plus_one(n) + 2

Importing Module

A module is a file that contains definitions, including variables and functions, that we can use once it is imported.

import math
print math.sqrt(25)

math includes a number of useful variables and functions, and sqrt() is one of those functions. In order to access math, all you need is the import keyword. When you simply import a module this way, it’s called a generic import.

It’s possible to import only specific variables or functions from a given module. Pulling in just a single function from a module is called a function import.

# Import *just* the sqrt function from math

from math import sqrt

if we want all the variables and functions in a module but don’t want to have to constantly type math then we can use

# Import *everything* from the math module

from math import *

Built-in Functions

max()

The max() function takes any number of arguments and returns the largest one.

>> maximum = max(10,35,64)
>> print maximum

min()

min() function returns the smallest of a given series of arguments.

>> minimum = (10,35,64)
>> print minimum

abs()

The abs() function returns the absolute value of the number it takes as an argument, it gives the number’s distance from 0 on an imagined number line.
>> absolute = abs(-42)
>> print absolute

type()

The type() function returns the type of the data it receives as an argument.

>> print type(42)
>> print type(4.2)
>> print type(‘spam’)

Now let us see the code which returns biggest, smallest and absolute values.

def biggest_number(*args):
print max(args)
return max(args)

def smallest_number(*args):
print min(args)
return min(args)

def distance_from_zero(arg):
print abs(arg)
return abs(arg)

biggest_number(-105, -53, 51, 100)
smallest_number(-106, -55, 54, 105)
distance_from_zero(-104)

These are some of the functions and built-in functions in python.

Google Analytics for Jan-Feb-March 2018(Q4 )

The statistics for MOHANMA.COM for Jan-Feb-March(starting Jan 26th to 31st March) are as shown above.

There have been a total of 737 sessions with 1673 pageviews.

The Bounce Rate of the website is 66.76%, which is pretty good as lower the bounce rate better the performance of website.

New visitors account for 76.5% of total visitors and returning visitors account for 23.5%.

India accounts for the highest traffic, with Karnataka being the leading region to in-flowing traffic with 576 sessions so far.

Among devices, mobile phones lead the traffic generated followed by desktop and lastly by tablets.

Apple iphones are the leading mobile phones used to access the website followed by Xiaomi and Motorola phones.

Thanks to everyone who have been visiting the website.

Love,

Mohan M A