Data Structures and

Algorithms
What Are Data
Structures?
What Are Data Structures?
• A data structure is a way of organizing and storing
data to perform operations efficiently.

• It defines a specific way of organizing and storing

data in a computer so that it can be used efficiently.

• The choice of a data structure depends on the

nature of the data, the operations to be performed,
and the constraints of the application.
What Are Data Structures?
• The following are some key concepts related to
data structures:

• Data: This refers to the raw facts and figures that are
processed by a computer. It can be numbers,
characters, images, or any other form of information.

• Structure: This refers to the way data is organized. A

data structure defines the relationship between the data
elements and the operations that can be performed on
them.
What Are Data Structures?
• The following are some key concepts related
to data structures:

• Operations: These are the actions that can be

performed on the data, such as insertion, deletion,
searching, and sorting.

• Memory Allocation: Data structures also involve

the allocation and management of memory to store
and retrieve data efficiently.
The two main types of data structures are:

• Primitive data types: These are the basic data types, such
as integers, floats, characters, etc.

• Composite data structures: These are structures that can

hold multiple elements, such as arrays, structs, linked lists,
trees, and graphs.
Data Structures Architecture
The following are some common types of data
structures:
Why are data structures important
in programming?
• Data structures are essential in programming because
they enable efficient organization and manipulation of
data.

• Choosing the right data structure for a specific problem

can significantly impact the performance of the code.
Why are data structures important
in programming?
• The choice of data structure significantly influences
algorithm efficiency.

• The use of an appropriate data structure can lead to

faster search, insertion, and deletion operations.

• It's crucial to analyze the requirements of the problem to

determine the most suitable data structure.
Linear Data Structure vs Nonlinear Data Structure

• In a linear data structure, elements are arranged in a

sequential manner, and each element has a unique
predecessor and successor, except for the first and last
elements.

• Linear structures facilitate the traversal of elements in a single,

sequential order.
Linear Data Structure vs Nonlinear Data Structure

• Examples:
• Arrays, Linked Lists, Stacks, and Queues are common linear data
structures.
• In an array, elements are stored in contiguous memory locations,
and their indices define the order.
• In linked lists, each element points to the next one, forming a
chain.

• Traversal:
• Elements are accessed and processed in a linear fashion,
typically through iteration.
• The order of elements is predetermined.
Linear Data Structure vs Nonlinear Data Structure

• In a nonlinear data structure, elements are not arranged in a

sequential order.

• Elements can have multiple predecessors and successors,

and relationships among elements are not strictly linear.
Linear Data Structure vs Nonlinear Data Structure

• Examples:
• Trees and Graphs are common nonlinear data structures.
• In trees, each element (node) can have multiple child nodes,
forming a hierarchical structure.
• Graphs allow arbitrary connections between nodes, creating
complex relationships.

• Traversal:
• Traversal in nonlinear structures may involve more complex
algorithms.
•
Algorithm
What is an Algorithm?
• An algorithm is any well-defined computational procedure
that takes some value, or set of values, as input and produces
some value, or set of values, as output.

• An algorithm is thus a sequence of computational steps

that transform the input into the output.
What is an Algorithm?
• For example, we might need to sort a sequence of numbers
into nondecreasing order (We say the list is to be sorted in
“nondecreasing order” instead of increasing order to allow for
the possibility that the same number may appear more than
once in the list.)

• Here is how we formally define the sorting problem:

What is an Algorithm?
• For example, given the input sequence (31; 41; 59; 26; 41; 58),
a sorting algorithm returns as output the sequence (26; 31; 41;
41; 58; 59).

• Such an input sequence is called an instance of the sorting

problem.

• In general, an instance of a problem consists of the input

needed to compute a solution to the problem.
What is an Algorithm?
• Sorting is a fundamental operation in computer science.

• As a result, we have a large number of good sorting algorithms at our

disposal.

• Which algorithm is best for a given application depends on among other

factors:
 the number of items to be sorted
 the extent to which the items are already somewhat sorted (degree of pre-
existing ordered items)
 possible restrictions on the item values (e.g. if an algorithm is designed for
floating-point numbers, using integer values might require additional
considerations.)
 the architecture of the computer
 the kind of storage devices to be used: main memory, disks, etc.
What is an Algorithm?
• An algorithm is said to be correct if, for every input instance, it
halts (ends) with the correct output.

• Incorrect algorithms can sometimes be useful, if we can control

their error rate.
Some of the key characteristics of an algorithm:
• Finiteness: An algorithm must have a finite (predetermined
steps or instruction) number of steps.
• Definiteness: Each step in the algorithm must be clearly
defined and unambiguous (clear).
• Input: An algorithm must take one or more inputs.
• Output: An algorithm must produce one or more outputs.
• Effectiveness: An algorithm must always produce the desired
output for any valid input.
• Effi ciency: An algorithm should use resources (such as time
and space) efficiently.
Usage of Algorithms
Algorithms are used in a wide variety of applications,
including:
• Searching: Finding a specific element in a list or database.
• Sorting: Arranging a list of elements in order.
• Optimization: Finding the best solution to a problem.
• Machine learning: Training a computer to learn from data.
• Cryptography: Encrypting and decrypting data.
Algorithm Efficiency
• Effi ciency
• The effi ciency of an algorithm refers to its ability to use resources (such as time
and space) effectively in solving a particular computational problem.

• In simpler terms, it measures how well an algorithm performs in terms of speed

and memory usage.

• An algorithm is said to be effi cient and fast, if it takes less time to execute and
consumes less memory space.

• Different algorithms devised to solve the same problem often differ dramatically
in their effi ciency.

• These differences can be much more significant than differences due to hardware
and software.
Algorithm Efficiency
• Effi ciency
• Let's consider a simple example of searching for a specific
element in an array.

• We'll compare two different algorithms: linear search and

binary search.

• Linear search goes through each element in the array one by

one until it finds the target or finishes the array.

• Binary search repeatedly divides the search space in half,

making it more efficient than linear search for a sorted array.
Algorithm Efficiency
• Effi ciency
• In this example, if the array is small or the element is likely to
be near the beginning, the linear search might perform
reasonably well.
•

• However, for larger sorted arrays, binary search is more

efficient due to its time complexity.
Algorithm analysis
What is Algorithm analysis?
• Algorithm analysis is the process of evaluating the
effi ciency and performance characteristics of algorithms.

• It involves studying how an algorithm behaves in terms of its

time complexity, space complexity, and other resource
usage.

• The goal of algorithm analysis is to understand and

compare algorithms based on their efficiency and to make
informed decisions about which algorithm is more suitable for
a particular problem.
What is Algorithm analysis?
• By understanding the characteristics of different
algorithms through analysis, developers and engineers
can make informed decisions about which algorithm
to use based on the specific requirements and
constraints of a given problem or application.
Some key aspects of algorithm analysis:
1. Time Complexity:
• Measures the amount of time an algorithm takes to complete as
a function of the input size.

• Time complexity of an algorithm signifies the total time required

by the program to run to completion.

• Often expressed using big-O notation to describe the upper

bound on the growth rate.
Some key aspects of algorithm analysis:
2. Space Complexity:
• Measures the amount of memory space an algorithm requires
as a function of the input size.
• Similar to time complexity, it is expressed using big-O
notation.

• Space complexity must be taken seriously for multi-user

systems and in situations where limited memory is available.
Some key aspects of algorithm analysis:
3. Best, Worst, and Average Case Analysis:
• Best Case: Analyzing the performance of an algorithm when it
operates on the input that leads to the best possible running
time.

• Define the input for which algorithm takes less time or

minimum time.

• In the best case calculate the lower bound of an algorithm.

• Example: In the linear search when search data is present at

the first location of large data then the best case occurs.
Some key aspects of algorithm analysis:

3. Best, Worst, and Average Case Analysis:

What are lower bound and upper bound?
The lower bound of an algorithm represents the minimum
amount of resources (usually time) the algorithm must consume
to solve a specific problem for any possible input of a given size.

• The lower bound represents an optimistic perspective,

indicating the best possible performance, while the upper
bound represents a pessimistic perspective, indicating the
worst-case performance.
Some key aspects of algorithm analysis:
3. Best, Worst, and Average Case Analysis:
• Worst Case: Analyzing the performance of an algorithm when
it operates on the input that leads to the maximum running
time.

• Define the input for which algorithm takes a long time or

maximum time.
• In the worst calculate the upper bound of an algorithm.

Example: In the linear search when search data is not present at

all then the worst case occurs. (searching all the elements)
Some key aspects of algorithm analysis:
3. Best, Worst, and Average Case Analysis:
• Worst Case Analysis is important because it provides an upper
bound on the running time of an algorithm.

• It ensures that the algorithm will not exceed a certain level of

inefficiency, regardless of the input.

• This is essential for guaranteeing performance in real-world

scenarios.
Some key aspects of algorithm analysis:

3. Best, Worst, and Average Case Analysis:

• Average Case: Analyzing the expected performance of an
algorithm when it operates on inputs randomly drawn from all
possible inputs.

• Average case = all random case time / total no of case

 Some key aspects of algorithm analysis:
• Let us consider an algorithm of sequential searching in an
array of size n.

• Its worst-case runtime complexity is O(n)

• Its best-case runtime complexity is O(1)

• Its average case runtime complexity is O(n/2)

Some key aspects of algorithm analysis:
4. Recursion and Recurrence Relations:
• Algorithms that call themselves are recursive.

• Analyzing their time complexity often involves solving

recurrence relations.
Some key aspects of algorithm analysis:
5. NP-completeness:
• NP-completeness is a concept in computational complexity theory.
• It is a classification of decision problems, which are problems that
have a yes/no answer.
• It provides a framework for classifying problems based on their
diffi culty.
• Knowing a problem is NP-complete saves researchers time from
searching for an efficient algorithm that likely doesn't exist.
• It pushes the boundaries of computational research and drives the
development of new algorithms and approaches.
NP-completeness example: The Knapsack Problem
• Given the set of items each with a weight and a value.
• Is there a subset of items that can be packed into the knapsack
- the capacity of a knapsack such that the total weight is
less than or equal to 6 and the total value is greater
than or equal to 8?
Some key aspects of algorithm analysis:
6. Sorting Algorithms:
• Different sorting algorithms
have different time
complexities (e.g., Bubble
Sort, Insertion Sort, Selection
Sort, QuickSort, Merge Sort).

• Understanding their time

complexity is crucial for
selecting the appropriate
algorithm for a specific task.
Some key aspects of algorithm analysis:
7. Greedy Algorithms: make locally optimal choices at each
stage with the hope of finding a global optimum.

They follow a "greedy" strategy by selecting the best available

option without considering the overall consequences or future
steps..

However, it's important to note that greedy algorithms don't

always guarantee finding the globally optimal solution for every
problem.
Some key aspects of algorithm analysis:
Greedy Algorithms example: Greedy Coin Change
Given a set of coin denominations and a target amount, find
the minimum number of coins needed to make up that amount.
Some key aspects of algorithm analysis:
Generic outline of a greedy algorithm:
1. Make a greedy choice at each step by selecting the best
available option without considering the consequences of
the choice on future steps.
2. Ensure that the chosen solution at each step is feasible and
doesn't violate any constraints.
3. Iterate the process by making the greedy choice at each
step until a solution to the entire problem is obtained.
4. In the end, the set of selected local solutions at each step
forms a solution that might be an optimal one to the
entire problem.
 Some key aspects of algorithm analysis:
8. Dynamic Programming:

• Dynamic programming breaks a problem into sub problems and

solves each sub problem only once, storing the solutions to avoid
redundant computations.

• The solution to the overall problem can be constructed from the

solutions of its sub problems in an optimal way.

• The term "programming" in dynamic programming does not refer

to writing code but rather to planning and organizing a series of
steps or decisions to achieve an optimal solution.
Some key aspects of algorithm analysis:
Dynamic Programming example:
• One classic example of a problem solved
using dynamic programming is the
computation of Fibonacci numbers.

• The Fibonacci sequence is a series of

numbers in which each number is the sum of
the two preceding ones, usually starting with
0 and 1. Mathematically, it is defined by the
recurrence relation:

• F(n)=F(n−1)+F(n−2)
• with initial conditions F(0)=0 and F(1)=1.
Some key aspects of algorithm analysis:
Dynamic Programming:
• The recursive solution to solve the Fibonacci leads to an
exponential number of recursive calls (exponential time
complexity) due to redundant computations, making it highly
ineffi cient for larger values of n.

• Dynamic programming, can be employed to optimize the

Fibonacci solution, specifically using memoization or
bottom-up tabulation techniques.
Some key aspects of algorithm analysis:
Dynamic Programming:
• When calculating Fibonacci numbers recursively, we often
compute the same values multiple times, resulting in
wasted time.

• To address this issue, we employ Memoization to check and

ensures that the same method or input is not executed
multiple times, optimizing the overall efficiency of the
process."
Analyzing algorithmic
complexity Big O
notation
Big-O Notation:
• Big-O Notation is commonly used in computer science to
analyze the efficiency of algorithms.

• The letter O in Big O notation stands for "order," and it is used

to represent the upper bound or worst case of the growth
rate of a function.

• The upper bound of an algorithm represents an upper limit on

the amount of resources (usually time) the algorithm will
consume to solve a specific problem for any possible input of a
given size.
Big-O Notation:
• In simpler terms, Big O notation provides a way to express
how long it takes an algorithm to run as the input size
increases.

• Big O notation provides an asymptotic analysis that helps in

comparing algorithms and understanding their scalability as the
input size increases.

• Asymptotic analysis is a method used to analyze the

efficiency and performance of algorithms. It focuses on how the
running time or space requirements of an algorithm grow as the
input size increases towards infinity.
Big-O Notation:
Here are some common Big O notations:
• O(1): Constant time complexity.
• O(n): Linear time complexity.
• O(log n): Logarithmic time complexity.
• O(n log n): Linearithmic time complexity.
• O(n^2): Quadratic time complexity.
• O(2^n): Exponential time complexity.
Growth-Rate Functions
O(1): Constant time complexity. Regardless of the input
size, the algorithm's running time remains constant. The
algorithm's running time does not grow with the input size.
It's the most efficient in terms of time complexity, but it's not
always achievable for more complex problems (input).
Examples include simple arithmetic operations (x=x+1),
accessing any element in an array.

O(n): Linear time complexity. Time requirement for a linear

algorithm increases directly with the size of the input. If the
input size doubles, the running time also roughly doubles.
Example include iterating through an array or list once.
Growth-Rate Functions
O(logn): Logarithmic time complexity. The algorithm's
running time grows logarithmically with the input size.
This means that the running time increases slowly as the
input size increases.
O(log n) is typically more effi cient than linear O(n)
growth. This makes algorithms with logarithmic time
complexity more effi cient for larger datasets.
Common in algorithms that repeatedly divide the problem space
in half. (e.g. divide-and-conquer algorithms)
Example as binary search, divide the problem into smaller
parts at each step.
Growth-Rate Functions
O(n*logn): Linearithmic time complexity. Common for many efficient
sorting algorithms.
O(n*logn) is more effi cient than quadratic O(n2) time complexity
(because the growth rate of nlogn⁡is slower than O(n2)).
O(n*logn) is less effi cient than linear O(n) (because as n increases,
the growth rate of nlogn⁡ is faster than linear (O(n)) or
logarithmic O(logn) time complexity.

O(n2): Quadratic time complexity. Time requirement for a quadratic

algorithm increases rapidly with the size of the input. The running time
of the algorithm is proportional to the square of the input size.
Common in nested loops where each iteration involves linear operations.
O(n2) is less efficient than linear or logarithmic time complexity
Growth-Rate Functions
O(n3): Cubic Time Complexity. The runtime of the algorithm
is proportional to the cube of the input size.
Time requirement for a cubic algorithm increases more rapidly
with the size of the input than the time requirement for a
quadratic algorithm.
This often occurs in algorithms with three nested loops or
algorithms that involve solving subproblems, each taking linear
time.
Growth-Rate Functions
O(2n): Exponential time complexity. The runtime grows
exponentially with the size of the input.
As the size of the input increases, the time requirement for an
exponential algorithm increases too rapidly to be practical.
Often found in brute-force algorithms that try all possible
solutions.
A brute force algorithm is an exhaustive search method that
systematically evaluates all possible solutions to a problem
and selects the best one. The term "brute force" implies that
the algorithm doesn't use any clever optimization
techniques and simply tries all possibilities.
Constant Time: O(1)
• An algorithm is said to run in constant time O(1) if it requires
the same amount of time regardless of the input size.
Examples: x=x + 1;
• array: accessing any element
• fixed-size stack: push and pop methods
• fixed-size queue: enqueue and dequeue methods
 Linear Time: O(n)
• An algorithm is said to run in linear time O(n) if its time
execution is directly proportional to the input size n, i.e. time
grows linearly as input size increases.

Examples:
• array: linear search, traversing, find minimum
• ArrayList: contains method // find an element if it exists
• queue: contains method
Calculating Time Complexity
• If there is a single iteration, and the iterative variable is
incrementing linearly (size increases +1) then it's O(n) e.g.

for(i=0; i<n; i++) //O(n)

for(i=0; i<n; i = i + 4) // still O(n)

• If the iterative variable is incremented
geometrically(multiplying the previous one by a fixed
number), then it's O(log n) e.g.
for(i=1;i<n;i = i * 2) //O(log n)
Logarithmic Time: O(log n)

• An algorithm is said to run in logarithmic time O(log n) if its

time execution is proportional to the logarithm of the input size.

Example:
Binary search
Quadratic Time: O(n2)

• An algorithm is said to run in logarithmic time O(n2) if

its time execution is proportional to the square of the
input size.
• Examples:
• bubble sort

• selection sort

• insertion sort
Nested loop
If there is nested loop, where one has a complexity of
O(n) and the other O(logn), then overall complexity is
O(nlogn).

for(i=0; i<n; i ++) // O(n)

{
for(j=1; j<n; j=j*3) // O(log n)
}

//Overall O(nlogn)
 The Execution Time of Algorithms

• Each operation in an algorithm (or a program) has a cost.

 Each operation takes a certain of time.
count = count + 1;  take a certain amount of time, but it
is constant

A sequence of operations:

count = count + 1; Cost: c1

sum = sum + count; Cost: c2

 Total Cost = c1 + c2
The Execution Time of Algorithms (cont.)
Example: Simple If-Statement
Cost Times
if (n < 0) c1 1
absVal = -n c2 1
else
absVal = n; c3 1

Total Cost <= c1 + max(c2,c3)

63
The Execution Time of Algorithms (cont.)
Example: Simple Loop
Cost Times
i = 1; c1
sum = 0; c2
while (i <= n) { c3
i = i + 1; c4
sum = sum + i; c5
}

 The time required for this algorithm is proportional to n

 The Execution Time of Algorithms (cont.)
Example: Simple Loop
Cost Times
i = 1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
i = i + 1; c4 n if n=5 so
sum = sum + i; c5 n n+1=6
because when
} reaching 5 the
Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*c5 while should
stop but it
 The time required for this algorithm is proportional to n tests i=6 and
this has cost.
The Execution Time of Algorithms (cont.)
Cost Times
i=1; c1
sum = 0; c2
while (i <= n) { c3
j=1; c4
Example: Nested Loop
while (j <= n) { c5
sum = sum + i; c6
j = j + 1; c7
}
i = i +1; c8
}

 The time required for this algorithm is proportional to n2

 The Execution Time of Algorithms (cont.)
Cost Times
i=1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
Example: Nested Loop
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}
Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8

 The time required for this algorithm is proportional to n2

The Execution Time of Algorithms (cont.)
Cost Times
i = 1; c1
sum = 0; c2
while (i <= n) { c3
i = i + 1; c4
sum = sum + i; c5
}

 So, the growth-rate function for this algorithm is

 The Execution Time of Algorithms (cont.)
Cost Times
i = 1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
i = i + 1; c4 n
sum = sum + i; c5 n
}
T(n) = c1 + c2 + (n+1)*c3 + n*c4 + n*c5

= (c3+c4+c5)*n + (c1+c2+c3)
= a*n + b
 So, the growth-rate function for this algorithm is O(n) // linear time
The Execution Time of Algorithms (cont.)
Cost Times
i=1; c1
sum = 0; c2
while (i <= n) { c3
j=1; c4
while (j <= n) { c5
sum = sum + i; c6
j = j + 1; c7
}
i = i +1; c8
}
 So, the growth-rate function for this algorithm is
 Cost Times
i=1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}
T(n) = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8

= (c5+c6+c7)*n2 + (c3+c4+c5+c8)*n + (c1+c2+c3)

= a*n2 + b*n + c
2
Calculating the time
complexity of various
algorithms:
Exercises
Exercise 1: Algorithm: Linear
Search
Description: Linear search is a simple search algorithm that
finds the position of a target value within a list (array) by
checking each element in sequence until a match is found or
the whole list has been searched.
Exercise 2: Algorithm: Bubble Sort
Description: Bubble sort is a simple sorting algorithm that
repeatedly steps through the list, compares adjacent
elements (next to), and swaps them if they are in the wrong
order. The pass through the list is repeated until the list
is sorted.
Exercise 3: Algorithm: Binary
Search
Description: Binary search is an efficient algorithm for finding a
target value within a sorted array. It works by repeatedly dividing
the search interval in half.
Exercise 4: Algorithm: Merge Sort
Description: Merge sort is a divide-and-conquer algorithm that
divides the array into two halves, sorts each half, and then
merges the sorted halves to produce a sorted array.
Exercise 5
• Analyze the time complexity of the following recursive
function:

int recursiveFunction(int n) {
if (n <= 1) {
return 1;
}
return recursiveFunction(n - 1) + recursiveFunction(n -
2);
}
Exercise 6 solution
• Howto improve the time complexity of Fibonacci
recursive function?