Unit 4

Functional programming: Functional programming is a programming paradigm in which code is

structured primarily in the form of functions. The origins of this programming style arise from a branch of
mathematics known as lambda calculus, which is the study of functions and their mathematical properties.
In contrast to the popular object-oriented and procedural approaches, functional programming offers a
different way of thinking when solving a problem.

Concepts of Functional Programming: Any Functional programming language is expected to follow

these concepts.

• Pure Functions: These functions have two main properties. First, they always produce the same
output for the same arguments irrespective of anything else. Secondly, they have no side-effects
i.e. they do modify any argument or global variables or output something.
• Recursion: There are no “for” or “while” loop in functional languages. Iteration in functional
languages is implemented through recursion.
• Functions are First-Class and can be Higher-Order: First-class functions are treated as first-class
variable. The first-class variables can be passed to functions as a parameter, can be returned from
functions or stored in data structures.
• Variables are Immutable: In functional programming, we can’t modify a variable after it’s been
initialized. We can create new variables – but we can’t modify existing variables.

Pure Functions: Pure functions have two properties.

• It always produces the same output for the same arguments. For example, 3+7 will always be 10
no matter what.
• It does not change or modifies the input variable.
The second property is also known as immutability. The only result of the Pure Function is the value it
returns. They are deterministic. Programs done using functional programming are easy to debug because
pure functions have no side effects or hidden I/O. Pure functions also make it easier to write
parallel/concurrent applications. When the code is written in this style, a smart compiler can do many things
– it can parallelize the instructions, wait to evaluate results when needing them, and memorize the results
since the results never change as long as the input doesn’t change.
# writing a function that multiplies numbers by 10
def pure_func(numbers: [int]) -> [int]:
# creating a new list to store the results
new_nums = []
# looping over the original input list
for num in numbers:
# appending newly calculated numbers into the new list
new_nums.append(num * 10)
return new_nums

original_nums = [1, 2, 3, 4]
changed_nums = pure_func(original_nums)
print(orginal_nums)
print(changed_nums)

Functions are First-Class and can be Higher-Order:

First-class objects are handled uniformly throughout. They may be stored in data structures, passed as
arguments, or used in control structures. A programming language is said to support first-class functions if
it treats functions as first-class objects.
Properties of first class functions:
• A function is an instance of the Object type.
• You can store the function in a variable.
• You can pass the function as a parameter to another function.
• You can return the function from a function.
• You can store them in data structures such as hash tables, lists, …

# Python program to demonstrate

# higher order functions
def shout(text):
return text.upper()
def whisper(text):
return text.lower()
def greet(func):
# storing the function in a variable
greeting = func("Hi, I am created by a function passed as an argument.")
print(greeting)

greet(shout)
greet(whisper)

Higher Order Functions

Before moving on to read about higher-order functions, let's understand how functions can be used as first-
class objects.
In languages that do not support functional programming, the functions can either be defined or called.
However, in languages that support functional programming, there are functions that are treated as first-
class objects or first-class citizens.
Here, we can use functions as any other data, i.e. we can store them in variables, we can also pass them to
other functions and return them from other functions just like we do with other variables. Let's look at some
code that demonstrates these.
# a simple function that prints text
def function():
print("I'm just a function!")
# function call
function()
# assigning the function to a variable
another_name_for_function = function()
# using the newly assigned variable to call the function
another_name_for_function()
Output:
I'm just a function!
I'm just a function!
In line number 6, we're just creating a new reference to the pure function function() that is
named another_name_for_function. Now you can call the function() using the variable as well, as we did
in line 7. Take a look at a few more examples.
def another_func():
print("Just another function!")
# printing a few objects, along with a function as an object
# printing the function will return a function object
print("dog", another_func, 31)
Output:
dog <function another_func at 0x7f81b4d29bf8> 31
You can display a function using the print() method as above.
# creating a list containing a function
some_objects = ['dog', another_func, 31]
print(some_objects[1])
# calling the function from the list showing object behavior of functions
print(some_objects[1]())
Output:
<function another_func at 0x7f735844b280>
Just another function!
It is also possible to store a function in a list much like any other object and call it.
Now, coming to higher-order functions. We know that functions can be passed to other functions are
parameters and that is exactly what we do in higher-order functions. Higher-order functions essentially
either accept other functions as parameters or return other functions. Let's look at some code to demonstrate
higher-order functions.
# a non-generic function that only writes the input text
def write_multiple(text: str, n: int):
for i in range(n):
print(text)
# this function call will print the input text 6 times
write_multiple("word", 6)
Output:
word
word
word
word
word
word
This function simply takes some text as input and prints it a certain number of times, which is also taken in
input. What if we wanted to perform some other function other than printing, and generalize our
function write_multiple to performing other functions than just printing? We could simply leave this choice
to the user, and pass the print function whenever we wanted to print the text and other methods for
performing other functions.
# writing a higher-order generic function that takes an action (to be performed on the input text) and a
'number of times' as input
def higher_order_write(text: str, n: int, action):
# for 'n' number of times
for i in range(n):
# perform the action on text
action(text)
# this function call will perform the 'print' action 4 times on the input text
higher_order_write("words", 4, print)
Output:
word
word
word
word
Let's say we have a task, wherein we have to create a function that adds the numbers 2, 5, and 10 to certain
numbers in a list. We can surely create functions such as increment2 and increment5 and increment10 for
this task the following way.
def increment2(nums: [int]) -> [int]:
new_nums = []
# looping over the original list
for num in nums:
# appending new values in the new list
new_nums.append(num + 2)
# returning a new list
return new_nums
print(increment2([12, 13]))
Output:
[14, 15]

Built-in Higher-order Functions: The following built-in higher-order functions are some of the
commonly used functions that return an iterator instead of a list or other data types for memory/space
efficiency reasons.
The map function:
This function gives us the ability to apply any function to every element on an object that is iterable (an
object over which we can iterate, or use a loop). For example, if we wanted to append a greeting to multiple
names in a list, we could do the following:
names = ['Bhavya', 'Abhijeet', 'Alkesh', 'Muskaan']
def add_greeting(name: str):
return "Hello " + name
names_and_greetings = map(add_greeting, names)
# this map function print will return an iterator
print(names_and_greetings)
# since it is an iterator, we can iterate over it and print the names separately
for the name in names_and_greetings:
print(name)
Output:
<map object at 0x7f87d27c9340>
Hello Bhavya
Hello Abhijeet
Hello Alkesh
Hello Muskaan
The filter Function
This function, as the name suggests, filters out some values from an iterable according to a specified
condition. The filter function returns either True or False.
Let's say we want to filter out the numbers that are even from a list of numbers, we can do it the following
way.
# a function that returns true for an even number given as input and false for an odd number
def is_even(num: int) -> bool:
return num % 2 == 0
nums = [1, 2, 3, 4, 5, 6, 7, 8]
# the filter function will extract values from the list that give 'True'
evens = filter(is_even, nums)
print(list(evens)) # since the filter will return an iterable object we print it as a type list
Output:
[2, 4, 6, 8]
You can even combine the two functions - map and filter and use them for expensive data manipulations!
Lambda Expressions OR Anonymous function: Lambda expressions are another concept of functional
programming. They are essentially anonymous functions. While creating functions in python, we usually
use the def keyword and give the function a unique name. However, lambda expressions allow us to skip
that process and write small functions much more quickly.
The syntax of a lambda expression is as follows:
lambda [arguments] : expression
For example, we can write a higher-order function that returns the nthsup> power of a number using a
lambda expression.
# creating a higher-order function that returns a lambda expression
def higher_order_power(n: int) -> int:
# this lambda expression returns the nth power of x
return lambda x: x ** n
power3 = higher_order_power(3)
print(power3(4)) # prints the cube of 4
Output:64
The higher-order function returned to us the 3rd power of 4 which is 64.
x = lambda a, b : a * b
print(x(5, 6))
def myfunc(n):
return lambda a : a * n
mydoubler = myfunc(2)
print(mydoubler(11))

Let's look at an example of a lambda function to see how it works. We'll compare it to a regular user-defined
function.
Assume I want to write a function that returns twice the number I pass it. We can define a user-defined
function as follows:
def f(x):
return x * 2

f(3)
>> 6
Now for a lambda function. We'll create it like this:
lambda x: x * 3
As I explained above, the lambda function does not have a return keyword. As a result, it will return the
result of the expression on its own. The x in it also serves as a placeholder for the value to be passed into
the expression. You can change it to whatever you want.
Now if you want to call a lambda function, you will use an approach known as immediately invoking the
function. That looks like this:
(lambda x : x * 2)(3)

>> 6
The reason for this is that since the lambda function does not have a name you can invoke (it's anonymous),
you need to enclose the entire statement when you want to call it.
When Should You Use a Lambda Function?
You should use the lambda function to create simple expressions. For example, expressions that do not
include complex structures such as if-else, for-loops, and so on.
So, for example, if you want to create a function with a for-loop, you should use a user-defined function.

reduce() function: Unlike map and filter functions, reduce() is not a built-in function, but is defined in
built-in functools module. It also needs two arguments, a function and an iterable. However it returns a
single value. The argument function is applied to two successive items in the list from left to right. Result
of the function in first call becomes first argument and third item in list becomes second. Cumulative result
is the return value of reduce() function.
In the example below, add() function is defined to return addition of two numbers. This function is used in
reduce() function along with a range of numbers between 0 to 100. Output is sum of first 100 numbers.
import functools
def add(x,y):
return x+y
num=functools.reduce(add, range(101))
print ('sum of first 100 numbers ',num)
Output:
sum of first 100 numbers 5050
We can use a lambda function instead of user-defined add() function for the same output.
num=functools.reduce(lambda x,y:x+y, range(101))

Lambda function can also be used as a filter. Following program use lambda function that filters all
vowels from given string.
string='''
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
'''
consonants=list(filter(lambda x: x not in ['a','e','i','o','u'],string))
print (''.join(consonants))

Lambda function as argument to map() function. The lambda function itself takes two arguments taken
from two lists and returns first number raised to second.The resulting mapped object is then parsed to output
list.
powersmap=map(lambda x,y: x**y, [10,20,30], [4,3,2])
print (list(powersmap))

Using map with filter:

list(map(square, filter(is_odd, numbers)))
[1, 9, 49, 121]

Advantages of Functional Programming in Python

• Predictability: It’s crucial to have predictable code since it makes testing and debugging easier.
When a function includes side effects, it can be more challenging to predict how it would act in
certain circumstances. Pure functions are essential to functional programming because they make
it simple to predict a function’s result given a specific input. Proving advantage to Developer.
• Modularity: Functional programming facilitates the simpler avoidance of errors brought on by
mutable states and side effects through the use of pure functions and immutability.
• Concurrency & Parallelism: Due to the simplicity of creating multi-threaded or multi-process
applications the functions can run concurrently without running the danger of colliding. Making no
issue for race situations and synchronization problems are not a concern because functions do not
depend on any external state. Additionally, Functional programming promotes the use of
immutable data structures and encourages writing pure functions, which makes it easier to write
parallel and concurrent code.
 • Code optimization & Refactoring: Since functions have predictable outputs, the compiler or
interpreter can make assumptions about how the function will behave and optimize accordingly.
This makes functional programming a good choice for applications that need to be highly
performant.
• Readability, Reusability & Maintainability: Small, reusable functions can be used by developers to
build more complex programs thanks to functional programming. Additionally, it promotes the use
of functions and discourages the use of side effects, making the code easier to read and understand.
This reduces the complexity of the code and makes it simpler to maintain.
• Testability: With the use of pure functions, functional programming makes it easier to test code, as
it ensures that the output of a function only depends on its input.
• Avoidance of Bugs: By Using the above mention methods when combined helps to avoid bugs easy
for the developer to detect any bugs and resolve them.

Logic programming: Writing code in Python, C, C++, Java, etc. we have observed paradigms like object-
oriented programming (OOPS), abstraction, looping constructs and numerous other states of programming.
Logic Programming is just another programming paradigm that works on relationships. These relationships
are built using facts and rules and are stored as a database of relations. It is a programming methodology
that works on formal and explicit logic of events.
Relation:Relations are the basis of logic programming. A relation can be defined as a fact that follows a
certain rule. For example, a relation given by [ A -> B ] is read as “if A is true, then B occurs”. In language
terms, this can be read as, “If you are an Engineer, then you are a Graduate” and infers that, “Engineers are
Graduates”. In programming languages, the semantics of writing relations changes based on the language’s
syntax, but this is the overall rationality behind what relations mean.
Facts: Every program that is built on logic needs facts. To achieve a defined goal, facts need to be provided
to the program. As the name suggests in general English, facts are merely the truth. True statements that
represent the program and the data. For instance, Washington is the capital of the USA.
Rules: Rules, like programming syntax are the constraints that help in drawing conclusions from a domain.
These are logical clauses that the program or the fact needs to follow to build a relation.

USE CASES OF LOGIC PROGRAMMING:

1. Logic Programming is extensively used in Natural Language Processing (NLP) since

understanding languages is about recognition of patterns that numbers cannot represent.

2. It is also used in prototyping models. Since expressions and patterns can be replicated using logic,
prototyping is made easy.

3. Pattern matching algorithms within image processing, speech recognition and various other
cognitive services also use logic programming for pattern recognition.

4. Scheduling and Resource Allocation are major operations tasks that logic programming can help
solve efficiently and completely.

5. Mathematical proofs are also easy to decode using logic programming.

Structure of Python Logic Programming: Let’s talk about facts and rules. Facts are true statements- say,
Bucharest is the capital of Romania. Rules are constraints that lead us to conclusions about the problem
domain. These are logical clauses that express facts. We use the following syntax to write a rule (as a
clause):
H :- B1, …, Bn.
We can read this as:
H if B1 and … and Bn.
Here, H is the head of the rule and B1, …, Bn is the body. A fact is a rule with no body:

An example would be:

fallible(X) :- human(X)
Every logic program needs facts based on which to achieve the given goal. Rules are constraints that
get us to conclusions.

How to Solve Problems with Logic Programming : Logic Programming uses facts and rules for solving
the problem. That is why they are called the building blocks of Logic Programming. A goal needs to be
specified for every program in logic programming. To understand how a problem can be solved in logic
programming, we need to know about the building blocks − Facts and Rules –

Facts: Actually, every logic program needs facts to work with so that it can achieve the given goal. Facts
basically are true statements about the program and data. For example, Delhi is the capital of India.

Rules: Actually, rules are the constraints which allow us to make conclusions about the problem domain.
Rules basically written as logical clauses to express various facts. For example, if we are building any game
then all the rules must be defined.
Rules are very important to solve any problem in Logic Programming. Rules are basically logical
conclusion which can express the facts. Following is the syntax of rule −
A∶− B1,B2,...,Bn.
Here, A is the head and B1, B2, ... Bn is the body.
For example − ancestor(X,Y) :- father(X,Y).
ancestor(X,Z) :- father(X,Y), ancestor(Y,Z).
This can be read as, for every X and Y, if X is the father of Y and Y is an ancestor of Z, X is the ancestor
of Z. For every X and Y, X is the ancestor of Z, if X is the father of Y and Y is an ancestor of Z.

Kanren: Kanren is a library within PyPi that simplifies ways of making business logic out of code. The
logic, rules, and facts we discussed previously can be turned into code using ‘kanren’. It uses advanced
forms of pattern matching to understand the input expressions and build its own logic from the given input.

Examples: kanren enables the expression of relations and the search for values which satisfy them. The
following code is the "Hello, world!" of logic programming. It asks for `1` number, `x`, such that `x == 5`
>>> from kanren import run, eq, membero, var, conde
>>> x = var()
>>> run(1, x, eq(x, 5))
(5,)
Multiple variables and multiple goals can be used simultaneously. The following code asks for a number
x such that `x == z` and `z == 3`
>>> z = var()
>>> run(1, x, eq(x, z),
eq(z, 3))
(3,)
kanren uses [unification], an advanced form of pattern matching, to match within expression trees.
The following code asks for a number, x, such that `(1, 2) == (1, x)` holds.
>>> run(1, x, eq((1, 2), (1, x)))
(2,)
The above examples use `eq`, a *goal constructor* to state that two expressions are equal. Other goal
constructors exist such as `membero(item, coll)` which states that `item` is a member of `coll`, a
collection.
The following example uses `membero` twice to ask for 2 values of x, such that x is a member of `(1, 2,
3)` and that x is a member of `(2, 3, 4)`.
>>> run(2, x, membero(x, (1, 2, 3)), # x is a member of (1, 2, 3)
membero(x, (2, 3, 4))) # x is a member of (2, 3, 4)
(2, 3)
### Logic Variables
As in the above examples, `z = var()` creates a logic variable. You may also, optionally, pass a token
name for a variable to aid in debugging:
>>> z = var('test')
>>> z
~test
Lastly, you may also use `vars()` with an integer parameter to create multiple logic variables at once:
>>> a, b, c = vars(3)
>>> a
~_1
>>> b
~_2
>>> c
~_3

### Representing Knowledge

kanren stores data as facts that state relationships between terms.
The following code creates a parent relationship and uses it to state facts about who is a parent of whom
within the Simpsons family.
>>> from kanren import Relation, facts
>>> parent = Relation()
>>> facts(parent, ("Homer", "Bart"),
... ("Homer", "Lisa"),
... ("Abe", "Homer"))
>>> run(1, x, parent(x, "Bart"))
('Homer',)
>>> run(2, x, parent("Homer", x))
('Lisa', 'Bart')

We can use intermediate variables for more complex queries. Who is Bart's grandfather?
>>> y = var()
>>> run(1, x, parent(x, y),
parent(y, 'Bart'))
('Abe',)
We can express the grandfather relationship separately. In this example we use `conde`, a goal constructor
for logical *and* and *or*.
>>> def grandparent(x, z):
... y = var()
... return conde((parent(x, y), parent(y, z)))
>>> run(1, x, grandparent(x, 'Bart'))
('Abe,')

QUESTION:

• Define a relation food and program these facts: avocado, carrot, tomato and broccoli are food.

• Define a relation color and program these facts: carrot is orange, avocado is green, broccoli is
green and tomato is red.

• Define a relation likes and program these facts: Jeff likes carrot, avocado and baseball, Bill likes
avocado and baseball, Steve likes tomato and baseball, Mary likes broccoli and Peter likes
baseball.

from kanren import Relation, fact, run, var

food = Relation()
color = Relation()
likes = Relation()

fact(food, "avocado")
fact(food, "carrot")
fact(food, "tomato")
fact(food, "broccoli")

fact(color, "avocado", "green")

fact(color, "carrot", "orange")
fact(color, "broccoli", "green")
fact(color, "tomato", "red")

fact(likes, "Jeff", "avocado")

fact(likes, "Jeff", "carrot")
fact(likes, "Jeff", "baseball")
fact(likes, "Bill", "avocado")
fact(likes, "Bill", "baseball")
fact(likes, "Steve", "tomato")
 fact(likes, "Steve", "baseball")
fact(likes, "Mary", "broccoli")
fact(likes, "Peter", "baseball")
An example query:
Jeff likes some food and its color is green.
>>> x = var()
>>> run(2, x, likes("Jeff", x), color(x, "green"))
('avocado',)

PyDatalog: pyDatalog adds the logic programming paradigm to Python's toolbox, in a pythonic way. You
can now run logic queries on databases or Python objects, and use logic clauses to define python classes.
In particular, pyDatalog can be used as a query language:
▪ it can perform multi-database queries (from memory datastore, 11 relational databases, and noSQL
database with appropriate connectors)
▪ it is more expressive than SQL, with a cleaner syntax;
▪ it facilitates re-use of SQL code snippet (e.g. for frequent joins or formula);
Datalog = SQL + recursivity
Datalog is a truly declarative language derived from Prolog, with strong academic foundations. It
complements Python very well for:
▪ managing complex sets of related information (e.g. in data integration or the semantic web).
▪ simulating intelligent behavior (e.g. in games),
▪ performing recursive algorithms (e.g. in network protocol, code and graph analysis, parsing)
▪ solving discrete constraint problems.
▪

from pyDatalog import pyDatalog

pyDatalog.create_terms('factorial, N')
factorial[N] = N*factorial[N-1]
factorial[1] = 1
print(factorial[3]==N) # prints N=6

from pyDatalog import pyDatalog

# Creating new pyDatalog variables

pyDatalog.create_terms("A, Pow2, Pow3")

def square(n):
return n*n

# Creating square function (predefined logic) as pyDatalog varible {Optional}

pyDatalog.create_terms("square")

input_values = range(10)[::-1] # Reverse order as desired

# Displaying the output (querying with & operator)

print ( A.in_(input_values) & (Pow2 == square(A)) & (Pow3 == A**3) )

pyDatalog module deals with data using its terms. That means, we need to create pyDatalog terms using
create_terms() method of pyDatalog class in pyDatalog module. This method globally declares the datalog
constants, variables, and unprefixed predicates. Then, based on several operators, we can query and filter
the results based on the specified conditions. Of course, the relational operators have been overridden (i.e.,
modified) to work as desired with pyDatalog variables.
Also, you may note that input_values has been reversed, as in_() method loop parses values in reverse
order as mentioned, so this pre-reversal will maintain the ascending order of output.

Variables and expressions: The next step is to declare the variables we'll use. They must start with an
upper-case letter:
pyDatalog.create_terms('X,Y')
Variables appear in logic queries, which return a printable result
# give me all the X so that X is 1
print(X==1)
X
-
1
Queries can contain several variables and several criteria ('&' is read 'and'):
# give me all the X and Y so that X is True and Y is False
print((X==True) & (Y==False))
X|Y
-----|------
True | False
Note the parenthesis around each equality: they are required to avoid confusion with (X==(True &
Y)==False).
Some queries return an empty result :
# give me all the X that are both True and False
print((X==True) & (X==False))
[]
Besides numbers and booleans, variables can represent strings. Furthermore, queries can contain python
expressions:
# give me all the X and Y so that X is a name and Y is 'Hello ' followed by the first letter of X
print((X==raw_input('Please enter your name : ')) & (Y=='Hello ' + X[0]))
Please enter your name : World
X|Y
------|--------
World | Hello W
In the second equality, X is said to be bound by the first equality, i.e. the first equality gives it a value,
making it possible to evaluate the expression in the second equality.
pyDatalog has no symbolic resolver (yet) ! If a variable in an expression is not bound, the query returns
an empty solution :
# give me all the X and Y so that Y is 1 and Y is X+1
print((Y==1) & (Y==X+1))
[]
To use your own functions in logic expressions, define them in Python, then ask pyDatalog to create
logical terms for them:
def twice(a):
return a+a
pyDatalog.create_terms('twice')
print((X==1) & (Y==twice(X)))
X|Y
--|--
1|2
Note that X must be bound before calling the function.
Similarly, pyDatalog variables can be passed to functions in the Python standard library:
# give me all the X and Y so that X is 2 and Y is the square root of X
import math
pyDatalog.create_terms('math')
print(X==2) & (Y==math.sqrt(X))
X|Y
--|--------------
2 | 1.41421356237
Loops: Let's first declare the Variables we'll need:
A loop can be created by using the .in() method (we'll see that there are other ways to create loops later):
from pyDatalog import pyDatalog
pyDatalog.create_terms('X,Y,Z')
# give me all the X so that X is in the range 0..4
print(X.in_((0,1,2,3,4)))
X
-
0
1
3
2
4
Here is the procedural equivalent
for x in range(5):
print x
0
1
2
3
4

Parallel Programming: The more complex a program gets the more often it is handy to divide it into
smaller pieces. This does not refer to source code, only, but also to code that is executed on your machine.
One solution for this is the usage of subprocesses in combination with parallel execution. Thoughts behind
this are:
• A single process covers a piece of code that can be run separately
• Certain sections of code can be run simultaneously, and allow parallelization in principle
• Using the features of modern processors, and operating systems, for example every core of a
processor we have available to reduce the total execution time of a program
• To reduce the complexity of your program/code, and outsource pieces of work to specialized agents
acting as subprocesses
Using sub-processes requires you to rethink the way your program is executed, from linear to parallel. It is
similar to changing your work perspective in a company from an ordinary worker to a manager - you will
have to keep an eye on who is doing what, how long does a single step take, and what are the dependencies
between the intermediate results.
A possible use case is a main process, and a daemon running in the background (master/slave) waiting to
be activated. Also, this can be a main process that starts worker processes running on demand. In practice,
the main process is a feeder process that controls two or more agents that are fed portions of the data, and
do calculations on the given portion.
Keep in mind that parallelization is both costly, and time-consuming due to the overhead of the
subprocesses that is needed by your operating system. Compared to running two or more tasks in a linear
way, doing this in parallel you may save between 25 and 30 percent of time per subprocess, depending on
your use-case. For example, two tasks that consume 5 seconds each need 10 seconds in total if executed in
series, and may need about 8 seconds on average on a multi-core machine when parallelized. 3 of those 8
seconds may be lost to overhead, limiting your speed improvements.

How many maximum parallel processes can you run?

The maximum number of processes you can run at a time is limited by the number of processors in your
computer. If you don’t know how many processors are present in the machine, the cpu_count() function
in multiprocessing will show it.
import multiprocessing as mp
print("Number of processors: ", mp.cpu_count())

What is Synchronous and Asynchronous execution?

In parallel processing, there are two types of execution: Synchronous and Asynchronous.
A synchronous execution is one the processes are completed in the same order in which it was started. This
is achieved by locking the main program until the respective processes are finished.
Asynchronous, on the other hand, doesn’t involve locking. As a result, the order of results can get mixed
up but usually gets done quicker.
There are 2 main objects in multiprocessing to implement parallel execution of a function: The Pool Class
and the Process Class.
1. Pool Class
1. Synchronous execution
• Pool.map() and Pool.starmap()
• Pool.apply()
2. Asynchronous execution
• Pool.map_async() and Pool.starmap_async()
• Pool.apply_async())
2. Process Class

There are two main ways to handle parallel programs:

• Shared Memory
In shared memory, the sub-units can communicate with each other through the same memory space. The
advantage is that you don’t need to handle the communication explicitly because this approach is sufficient
to read or write from the shared memory. But the problem arises when multiple process access and change
the same memory location at the same time. This conflict can be avoided using synchronization techniques.
• Distributed memory
In distributed memory, each process is totally separated and has its own memory space. In this scenario,
communication is handled explicitly between the processes. Since the communication happens through a
network interface, it is costlier compared to shared memory.

Multi−Processing in Python : Multi−processing, as opposed to multi−threading, offers genuine

parallelism by using several processes, each with their own memory space. A high−level interface for
implementing multi−processing is provided by the multiprocessing module of Python. Multiprocessing is
appropriate for CPU−bound activities since each process runs in a distinct Python interpreter, avoiding the
GIL multi−threading restriction.
Using pool class:
The Pool class in multiprocessing can handle an enormous number of processes. It allows you to run
multiple jobs per process (due to its ability to queue the jobs). The memory is allocated only to the executing
processes, unlike the Process class, which allocates memory to all the processes. The Pool class takes the
number of worker processes to be present in the pool and spawns the processes.
Launching many processes using the Process class is practically infeasible as it could break the OS. Hence
comes Pool which shall handle the distribution of jobs to and collection of results from all the spawned
processes in the presence of a minimal number of worker processes (most preferably, the number of worker
processes is equal to CPU cores).
Pool doesn’t work in interactive interpreter and Python classes. It requires __main__ module to be
importable by the children.
Multiprocessing is used in the code below.The burden is divided among the available processes by the
map() method once the pool class generates a pool of worker processes. The results list is a collection of
the results.
import multiprocessing
def square(number):
return number ** 2
numbers = [1, 2, 3, 4, 5]
with multiprocessing.Pool() as pool:
results = pool.map(square, numbers)
print(results)

Using Process
The Process class in multiprocessing allocates all the tasks in the memory in one go. Every task created
using the Process class has to have a separate memory allocated.
Imagine a scenario wherein ten parallel processes are to be created where every process has to be a separate
system process.
Here’s an example (order of the output is non-deterministic):
The Process class initiated a process for numbers ranging from 0 to 10. target specifies the function to be
called, and args determines the argument(s) to be passed. start() method commences the process. All the
processes have been looped over to wait until every process execution is complete, which is detected using
the join() method. join() helps in making sure that the rest of the program runs only after the
multiprocessing is complete.
sleep() method helps in understanding how concurrent the processes are!

from multiprocessing import Process

def numbers(start_num):
for i in range(5):
print(start_num+i, end=' ')
if __name__ == '__main__':
p1 = Process(target=numbers, args=(1,))
 p2 = Process(target=numbers, args=(10,))
p1.start()
p2.start()
# wait for the processes to finish
p1.join()
p2.join()

# output:
# 1 2 3 4 5 10 11 12 13 14
In this program, two separate processes run the numbers function at the same time with different parameters.
First, we create two Process objects and assign them the function they will execute when they start running,
also known as the target function. Second, we tell the processes to go ahead and run their tasks. And third,
we wait for the processes to finish running, then continue with our program.
While the output looks the same as if we ran the numbers function twice sequentially, we know that the
numbers in the output were printed by independent processes.

Benefits of Using Multiprocessing :Here are a few benefits of multiprocessing:

• better usage of the CPU when dealing with high CPU-intensive tasks
• more control over a child compared with threads
• easy to code
The first advantage is related to performance. Since multiprocessing creates new processes, you can make
much better use of the computational power of your CPU by dividing your tasks among the other cores.
Most processors are multi-core processors nowadays, and if you optimize your code you can save time by
solving calculations in parallel.
The second advantage looks at an alternative to multiprocessing, which is multithreading. Threads are not
processes though, and this has its consequences. If you create a thread, it’s dangerous to kill it or even
interrupt it as you would do with a normal process. Since the comparison between multiprocessing and
multithreading isn’t in the scope of this article, I encourage you to do some further reading on it.
The third advantage of multiprocessing is that it’s quite easy to implement, given that the task you’re trying
to handle is suited for parallel programming.

Exchanging objects between processes

Multiprocessing supports two types of communication channel between processes:
Queues:The Queue class is a near clone of queue.Queue. For example:
from multiprocessing import Process, Queue
def f(q):
q.put([42, None, 'hello'])
if __name__ == '__main__':
q = Queue()
p = Process(target=f, args=(q,))
p.start()
print(q.get()) # prints "[42, None, 'hello']"
p.join()
Queues are thread and process safe.
Pipes: The Pipe() function returns a pair of connection objects connected by a pipe which by default is
duplex (two-way). For example:
from multiprocessing import Process, Pipe
def f(conn):
conn.send([42, None, 'hello'])
conn.close()
if __name__ == '__main__':
parent_conn, child_conn = Pipe()
p = Process(target=f, args=(child_conn,))
p.start()
print(parent_conn.recv()) # prints "[42, None, 'hello']"
p.join()
The two connection objects returned by Pipe() represent the two ends of the pipe. Each connection object
has send() and recv() methods (among others). Note that data in a pipe may become corrupted if two
processes (or threads) try to read from or write to the same end of the pipe at the same time. Of course there
is no risk of corruption from processes using different ends of the pipe at the same time.

Synchronization between processes:

Multiprocessing contains equivalents of all the synchronization primitives from threading. For instance one
can use a lock to ensure that only one process prints to standard output at a time:
from multiprocessing import Process, Lock
def f(l, i):
l.acquire()
try:
print('hello world', i)
finally:
l.release()
if __name__ == '__main__':
lock = Lock()
for num in range(10):
Process(target=f, args=(lock, num)).start()
Without using the lock output from the different processes is liable to get all mixed up.

Key differences between Data Parallelisms and Task Parallelisms :

Data Parallelisms Task Parallelisms

1. Same task are performed on different 1. Different task are performed on the same or different
subsets of same data. data.

2. Synchronous computation is performed. 2. Asynchronous computation is performed.

Data Parallelisms Task Parallelisms

3. As there is only one execution thread 3. As each processor will execute a different thread or
operating on all sets of data, so the speedup process on the same or different set of data, so speedup is
is more. less.

4. Amount of parallelization is proportional 4. Amount of parallelization is proportional to the number

to the input size. of independent tasks is performed.

5. It is designed for optimum load balance on 5. Here, load balancing depends upon on the e availability
multiprocessor system. of the hardware and scheduling algorithms like static and
dynamic scheduling.

Network programming: Python plays an essential role in network programming. The standard library of
Python has full support for network protocols, encoding, and decoding of data and other networking
concepts, and it is simpler to write network programs in Python than that of C++.

There are two levels of network service access in Python. These are:
• Low-Level Access
• High-Level Access
In the first case, programmers can use and access the basic socket support for the operating system using
Python's libraries, and programmers can implement both connection-less and connection-oriented protocols
for programming.
Application-level network protocols can also be accessed using high-level access provided by Python
libraries. These protocols are HTTP, FTP, etc

Why Python for Network Programming?

There are several reasons for using Python for this course. The simplicity of python makes it the most
powerful language. Python is syntactically simplest to implement amongst it's counterparts.
Also, you can do almost everything with python. 'Ohh.. Almost everything, Can we make a website using
python? Can we make face detection application using python? can we make our own personal assistant
using python? Can we make a penetration testing tool using python?
The answer to all of the above questions is a big YES!
The third party libraries support provided by python makes it limitless. There is a proper documentation for
the third party libraries as well, hence using them in your application becomes easier.
Lastly, python is powerful enough to make websites like Quora and provide the backbone for the Google
search engine, so yes, python is the perfect choice for network programming.

Sockets:

To understand what sockets are, let's start with the Internet Connection. The Internet Connection basically
connects two points across the internet for data sharing and other stuff. One process from computer C1 can
communicate to a process from computer C2, over an internet connection. It has following properties:
 • Reliable: It means until the cables connecting two computers are safe, data will be transferred
safely.
• Point-to-Point: Connection is established between 2 points.
• Full-Duplex: It means transfer of information can occur in both ways i.e. from client to server as
well as server to client simultaneously(at the same time).
Sockets are the endpoints of a bidirectional, point-to-point communication channel. Given an internet
connection, say between client(a browser) and the serverwe will have two sockets. A Client Socket and
a Server Socket.
Socket acts on two parts: IP Address + Port Number
For now, just focus on Connect and Bind methods.
Connect is used by the client socket to start a connection with the server. This request is fulfilled
by bind method of the server socket. If you are having problem with the code, don't worry. Every bit of it
will be explained separately with examples.
Before concluding, let's see some differences between the client and the server sockets:
• Unlike client sockets, server sockets are not short lived. For example: you might need youtube.com
for a single request but youtube.com has to be up 24*7 for any request which it might receive from
users across the globe.
• Unlike client socket which uses Ephermal Port for connection, server socket requires a standard or
well defined port for connection like: Port 80 for Normal HTTP Connection, Port 23 for Telnet etc.
There are two type of sockets: SOCK_STREAM and SOCK_DGRAM. Below we have a comparison of
both types of sockets.

SOCK_STREAM SOCK_DGRAM

For TCP protocols For UDP protocols

Reliable delivery Unrelible delivery

Guaranteed correct ordering of packets No order guaranteed

Connection-oriented No notion of connection(UDP)

Bidirectional Not Bidirectional

Socket Module in Python :To create a socket, we must use socket.socket() function available in the Python
socket module, which has the general syntax as follows:
S = socket.socket(socket_family, socket_type, protocol=0)

• socket_family: This is either AF_UNIX or AF_INET. We are only going to talk about INET
sockets in this tutorial, as they account for at least 99% of the sockets in use.

• socket_type: This is either SOCK_STREAM or SOCK_DGRAM.

• Protocol: This is usually left out, defaulting to 0.

Client Socket Methods: Following are some client socket methods:

connect( ): To connect to a remote socket at an address. An address format(host, port) pair is used
for AF_INET address family.

Server Socket Methods : Following are some server socket methods:

• bind( ):This method binds the socket to an address. The format of address depends on socket
family mentioned above(AF_INET).
• listen(backlog):This method listens for the connection made to the socket. The backlog is the
maximum number of queued connections that must be listened before rejecting the connection.
• accept( ): This method is used to accept a connection. The socket must be bound to an address
and listening for connections. The return value is a pair(conn, address) where conn is a new
socket object which can be used to send and receive data on that connection, and address is the
address bound to the socket on the other end of the connection.

Few General Socket Methods:For the below defined socket object,

s = socket.socket(socket_family, socket_type, protocol=0)

TCP Socket Methods UDP Socket Methods

s.recv()→ Receives TCP messages s.recvfrom()→ Receives UDP messages

s.send()→ Transmits TCP messages s.sendto()→ Transmits UDP messages

Some Basic Socket Methods

• close() This method is used to close the socket connection.

• gethostname() This method returns a string containing the hostname of the machine where the
python interpreter is currently executing. For example: localhost.

• gethostbyname() If you want to know the current machine's IP address, you may
use gethostbyname(gethostname())

Working with TCP Sockets:

In TCP there is a concept of handshake. So, What is a handshake? It's a way to ensure that the connection
has been established between interested hosts and therefore data transfer can be initiated.
In simple terms, when you make a phone call to someone you first say "Hello", and in return the person
replies with a "Hello". This ensures that the connection has been established between both the parties and
data(voice in this case)transfer can begin now. This is, for sure, the simplest example of handshake.
Unlike UDP, TCP protocol requires that you establish a connection first. So if A is to send a letter using
TCP protocol to B, he has to first establish a connection with B. Once a connection is established, A can
then send the first envelope and wait for B to acknowledge that he has received it. Once A gets the
acknowledgment from B that the envelope 1 is safely received, A can send envelope 2. The process repeats
until A tells B that he has sent all the envelopes.
On the basis of the above example we can device the properties of TCP:
• Reliable: TCP manages message acknowledgment, retransmission and timeout. Multiple attempts
to deliver the message are made. If it gets lost along the way, the server will re-request the lost part.
• Ordered: The messages are delivered in a particular order in which they were meant to be.

Simple Server Program

#!/usr/bin/python
#This is tcp_server.py script
import socket #line 1: Import socket module
s = socket.socket() #line 2: create a socket object
host = socket.gethostname() #line 3: Get current machine name
port = 9999 #line 4: Get port number for connection
s.bind((host,port)) #line 5: bind with the address
print "Waiting for connection..."
s.listen(5) #line 6: listen for connections

while True:
conn,addr = s.accept() #line 7: connect and accept from client
print 'Got Connection from', addr
conn.send('Server Saying Hi')
conn.close() #line 8: Close the connection
This script will do nothing as of now. It waits for a client to connect at the port specified. If we run this
script now, without having a client, it will wait for the connection,
Similarly, every website you visit has a server on which it is hosted, which is always waiting for clients to
connect. Now let's create a client.py program and try to connect with our server.py.

Simple Client Program

Below is the client.py program. The client tries to connect to server's port, 9999(well defined port). The
code line, s.connect((host, port)) opens up a TCP connection to the hostname on the port 9999.
#!/usr/bin/python
#This is tcp_client.py script
import socket
s = socket.socket()
host = socket.gethostname() # Get current machine name
port = 9999 # Client wants to connect to server's
# port number 9999
s.connect((host,port))
print s.recv(1024) # 1024 is bufsize or max amount of data to be received at once
s.close()
Now, run the server.py script first(if you haven’t yet) and then run the client.py script.

Flow Diagram of the Program

Working with UDP Sockets:

UDP is a connectionless protocol. In this protocol data is sent over the internet as datagrams. In the previous
example, 20 pages may refer to 20 datagrams. Let's have a look at some properties of the UDP protocols.

• Unreliable: When a UDP message is sent, there is no way to know if it will reach its destination or
not; it could get lost along the way. In UDP, there is no concept of acknowledgment, retransmission,
or timeout (as in TCP).

• Not ordered: If two messages are sent to the same recipient, the order in which they arrive cannot
be predicted.

• Lightweight: There is no ordering of messages, no tracking connections, etc. Hence UDP messages
are used when the rate of data transmission required is more and relibility is not important.

• Datagrams: Packets are sent individually and are checked for integrity only if they arrive.

SERVER PROGRAM:
#!usr/bin/python
import socket
sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM) # For UDP
udp_host = socket.gethostname() # Host IP
udp_port = 12345 # specified port to connect
#print type(sock) ============> 'type' can be used to see type
# of any variable ('sock' here)
sock.bind((udp_host,udp_port))
while True:
print "Waiting for client..."
data,addr = sock.recvfrom(1024) #receive data from client
print "Received Messages:",data," from",addr
CLIENT PROGRAM:
#!usr/bin/python
import socket
sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM) # For UDP
udp_host = socket.gethostname() # Host IP
udp_port = 12345 # specified port to connect
msg = "Hello Python!"
print "UDP target IP:", udp_host
print "UDP target Port:", udp_port
sock.sendto(msg,(udp_host,udp_port)) # Sending message to UDP server

Flow Diagram of the Program