CSA2001 - FUNDAMENTALS IN AI and ML

Unit II Problem Solving Methods

Problem solving Methods - Search Strategies - Uninformed - Informed -
Heuristics - Local Search Algorithms and Optimization Problems -
Searching with Partial Observations – Constraint Satisfaction Problems
– Constraint Propagation - Backtracking Search - Game Playing – Optimal
Decisions in Games – Alpha - Beta Pruning - Stochastic Games.

Prepared by
Dr M.Manimaran
Senior Associate Professor (Grade-1)
School of Computing Science and Engineering
VIT Bhopal University
 Searching with Partial Observations
• Searching with Partial Observation
• Searching with Nondeterministic Actions
• Online Search Agents and Unknown Environments

2
 Searching with Partial Information
• When the environment is fully observable and deterministic, and the agent knows what
the effects of each action are, percepts provide no new information after the agent
determines the initial state.
• In partially observable environments, every percept helps narrow down the set of
possible states the agent might be in, making it easier for the agent to achieve its goals.
• In nondeterministic environments, percepts tell the agent which of the possible
outcomes has actually occurred.
• In both cases, future percepts cannot be determined in advance and the agent's future
actions will depend on those future percepts.
• A solution to this type of problem is a contingency plan (also know as a that specifies
what to do depending on what percepts are received.

3
 Searching with Nondeterministic Actions
• Example: Erratic Vacuum World, same state space
as before. Goal states are 7 and 8.
• Suck action is:
• When applied to a dirty square, the action cleans
the square and sometimes cleans up dirt in an
adjacent square, too
• When applied to a clean square, the action
sometimes deposits dirt on the carpet.

4
 Searching with Nondeterministic Actions
• To formulate this problem, generalize notion of
transition model from before. Use results function
that returns a set of possible outcome states.
• E.g., Results (1, Suck) = { 5, 7 }
• Also generalize notion of a solution to a contingency
plan.
• E.g., From state 1, [ Suck [Right, Suck] else [ ] ]

5
 Searching with Nondeterministic Actions
Augment search trees in the following trees in the following way
• Branching caused by agent's choice of action are called OR nodes. E.g., in vacuum
world, agent chooses Left or Right or Suck.
• Branching caused by environment's choice of action are called AND nodes.
• world, Suck action in state 1 leads to a state in { 5,
• E.g., in erratic vacuum world, Suck action in state 1 leads to a state in { 5, 7 }, so agent
would need to find a plan for state 5 and state 7.
• Two kinds of nodes alternate giving AND-OR tree.

6
 Searching with Nondeterministic Actions
Solution is a subtree that that
• has a goal node at every leaf
• specifies one action at each OR node
• includes every outcome branch at each
AND node.
• OR nodes: In a deterministic environment, the only
branching is introduced by the agent’s own choices in
each state, we call these nodes OR nodes.
• AND nodes: In a nondeterministic environment,
branching is also introduced by the environment’s
choice of outcome for each action, we call these nodes
AND nodes
• AND-OR tree: OR nodes and AND nodes alternate.
States nodes are OR nodes where some action must but
chosen. At the AND nodes(shown in the circles), every
outcome must be handled.
• A solution(shown in bold lines) for an AND-OR search
problem is a subtree that
• 1. has a goal node at every leaf;
• 2. specifies one action at each of its OR nodes;
• 3. includes every outcome branch at each of its AND
nodes. 7
AND-OR Search Algorithms

8
AND-OR Search Algorithms

9
 AND-OR Search Algorithms
• Given algorithm is DFS Can also search tree with BFS or best-first, and generalize
heuristic function for contingency plans for an analog to A*.
• An alternative agent design is for the agent act before it has a guaranteed plan and
deal contingencies only as they arise in execution.
• This type of interleaving of search and execution is useful for exploration problems
and game playing.
• As noted, when an environment is partially observable, an agent can be in one of
several possible states. An action leads to one of several possible outcomes.
• To solve these problems, an agent maintains a belief state that represent the
agent's current belief about the possible physical state it might be in, given the
sequence of actions and percepts up to that point.

10
11
12
 Searching with No Observation
• When an agent's percepts provide no information at
all, this is called a sensor less (or conformant)
problem
• Example: deterministic, assume know geography,
but not location or dirt.
• Initially could be in any state { 1, 2, 3, 4, 5, 6, 7, 8 }

• Do Right, must in { 2, 4, 6, 8 }. Do Suck results in {4,

8 }. Doing Left, then Suck coerces world into state 7
regardless of initial state

13
14
 Searching with No Observation
• Solve sensorless problems by searching space of belief states, which is fully
observable to agent. Suppose physical problem P with Actions , Result , GoalTest and
StepCost.

• Define corresponding sensorless problem: Belief state components

• Belief states: powerset of states of P, although many are unreachable from initial
state. For N states of P, there are 2N possible belief states.
• Initial state: typically, all the states of P
• Actions: if illegal actions are safe, use the union of actions of all states in b; otherwise
use the intersection. Use to predict the next belief states.
• Transition model: b' = Result (Result (b, a) = { s' : s' = Result (s, a) and s' b }.
• Similarly for nondeterministic environments using Results.
• Goal test: all of the physical states in b must satisfy GoalTest P.
• Path cost: Assume that action costs same in all states, so transfer from underlying
problem.
15
 Searching with No Observation
• Complete belief state search
space for deterministic
environment.
• Note only 12 reachable
states out of 28 possible
belief states.

Unit II Problem Solving Methods / Dr Subash Chandra Bose 16

Searching with Observation

17
Searching with Observation

18
Solving Partially Observable Problems

19
20
21
22
23
24
 Online search problems
• Previous problems use offline search algorithms. Complete solution is computed,
then solution is executed.
• Previous problems use offline search
• In online search, agent interleaves computation with action: it first takes an
action, then observes the environment and computes the next action.
• Good idea for dynamic environments where there is a penalty for computing too
long.
• Helpful idea for nondeterministic environments. Don't spend time planning for
contingencies that are rare.
• Necessary idea for unknown environments, where agent doesn't know what states
exists or the results of its actions. Called an exploration problem.
• Agent uses actions as experiments to learn about its environment.

25
 Online search problems
• Assume deterministic, fully observable environment. Agent only knows:
• Actions(S) - list of actions allowed in state s

• Step-cost function c(s, a, s’) - cannot be used until agent knows that is the
outcome of doing a GoalTest(s)

• In particular, it doesn't know Result (s,a) except by actually being in s and doing a,
then observing s'.

26
Online search problems

27
28
29
30
31
32
33
Unit II Problem Solving Methods / Dr Subash Chandra Bose 34
 Online search problems
• Typically, objective is to reach goal state while minimizing cost, where cost is total
path cost of path an agent actually travels.

• Common to compare this cost with path cost of path agent would follow if it knew
the search space in advance, i.e., the actual shortest path. Called the competitive
ratio and would like it to be as small as possible.

• Nice idea, but in some cases competitive ratio is infinite, if online search agent
• reaches a dead-end state from which no goal state is reachable.
• Claim: no algorithm can avoid dead ends in all state spaces. Two states shown are
• indistinguishable, so must result in same action sequence.
• Will be wrong for one.

Unit II Problem Solving Methods / Dr Subash Chandra Bose 35

 Online search problems
• To make progress, we assume that state space is safely explorable. I.e., some goal
state is reachable from every reachable state. State spaces with reversible actions
(mazes, 8-puzzles) are clearly safely explorable.

• Even with this assumption, no bounded competitive ratio can be guaranteed if

there are paths of unbounded cost.
• Common to describe performance in terms of size of entire state space instead of
depth of the shallowest node.

• Online search algorithms are very different from offline search algorithms. Since
agent occupies specific physical node, can only expand immediate successors.

• Need to expand nodes in local order. DFS has this property, but needs reversible
actions to support backtracking.

Unit II Problem Solving Methods / Dr Subash Chandra Bose 36

 Online search problems
• Hill-climbing already is a local search algorithm, but can get stuck in local maxima.
• Add memory to keep track of a "current best estimate", H(s), of the cost to reach
goal from each state that has been visited.

• H(s) starts out with h(s), the heuristic estimate and is updated as agent gains
experience.

37
 CSA2001 - FUNDAMENTALS IN AI and ML
Unit II Problem Solving Methods
Problem solving Methods - Search Strategies - Uninformed - Informed -
Heuristics - Local Search Algorithms and Optimization Problems -
Searching with Partial Observations – Constraint Satisfaction Problems –
Constraint Propagation - Backtracking Search - Game Playing – Optimal
Decisions in Games – Alpha - Beta Pruning - Stochastic Games.

Prepared by
Dr M.Manimaran
Senior Associate Professor (Grade-1)
School of Computing Science and Engineering
VIT Bhopal University
 Constraint Satisfaction Problems (CSPs)
• Constraint means restriction or limitation.
• In AI, constraint satisfaction problems are the problems which must be solved
under some constraints.
• The focus must be on not to violate the constraint while solving such problems.
Finally, when we reach the final solution, CSP must obey the restriction.

• Reference: https://youtu.be/0fQYmHXnWnE

39
40
 Backtracking Search
• Backtracking search:
• A depth-first search that chooses values for one variable at a time and backtracks when
a variable has no legal values left to assign.
• Backtracking algorithm repeatedly chooses an unassigned variable, and then tries all
values in the domain of that variable in turn, trying to find a solution.

Backtracking Search : https://youtu.be/DKCbsiDBN6c

N Queens Problem using Backtracking: https://youtu.be/xFv_Hl4B83A

Reference:
• https://www.cnblogs.com/RDaneelOlivaw/p/8072603.html

41
42