dialogue notations

and design
 Dialogue Notations and Design

• Dialogue Notations
• Diagrammatic
• state transition networks, JSD diagrams, flow charts
• Textual
• formal grammars, production rules, CSP

• Dialogue linked to
• the semantics of the system – what it does
• the presentation of the system – how it looks

• Formal descriptions can be analysed

• for inconsistent actions
• for difficult to reverse actions
• for missing actions
• for potential miskeying errors
what is dialogue?

• conversation between two or more parties

• usually cooperative

• in user interfaces
• refers to the structure of the interaction
• syntactic level of human–computer ‘conversation’

• levels
• lexical – shape of icons, actual keys pressed(Sound & Spell)
• syntactic – order of inputs and outputs(Grammar)
• semantic – effect on internal application/data (Meaning)
structured human dialogue

• human-computer dialogue very constrained

• some human-human dialogue formal too …
Minister: do you man’s name take this woman …
Man: I do
Minister: do you woman’s name take this man …
Woman: I do
Man: With this ring I thee wed
(places ring on womans finger)
Woman: With this ring I thee wed (places ring ..)
Minister: I now pronounce you man and wife
lessons about dialogue

• wedding service
• sort of script for three parties
• specifies order
• some contributions fixed – “I do”
• others variable – “do you man’s name …”
• instructions for ring
concurrent with saying words “with this ring …”

• if you say these words are you married?

• only if in the right place, with marriage licence
• syntax not semantics
… and more

• what if woman says “I don’t”?

• real dialogues often have alternatives:
Judge: How do you plead guilty or not guilty?
Defendant: either Guilty or Not guilty

• the process of the trial depends on the defendants response

• focus on normative responses
• doesn’t cope with judge saying “off with her head”
• or in computer dialogue user standing on keyboard!
dialogue design notations

• dialogue gets buried in the program

• in a big system can we:
• analyse the dialogue:
• can the user always get to see current shopping basket
• change platforms (e.g. Windows/Mac)
• dialogue notations helps us to
• analyse systems
• separate lexical from semantoc
• … and before the system is built
• notations help us understand proposed designs
 graphical notations

state-transition nets (STN), Hierarchical STN,

Petri nets, state charts
flow charts, JSD diagrams
State transition networks (STN)

• circles - states
• arcs - actions/events click on
click on centre circumference
Circle 1 Circle 2 Finish
rubber band draw circle
select 'circle'

Start Menu

select 'line' click on

first point double click
Line 1 Line 2 Finish
rubber band draw last
line

click on point
draw a line
State transition networks -
events

• arc labels a bit cramped because:

• notation is `state heavy‘
• the events require most detail
click on
click on centre circumference
Circle 1 Circle 2 Finish
rubber band draw circle
select 'circle'

Start Menu

select 'line' click on

first point double click
Line 1 Line 2 Finish
rubber band draw last
line

click on point
draw a line
State transition networks -
states

• labels in circles a bit uninformative:

• states are hard to name
• but easier to visualise

click on
click on centre circumference
Circle 1 Circle 2 Finish
rubber band draw circle
select 'circle'

Start Menu

select 'line'
... ... ...
Hierarchical STNs

• managing complex dialogues

• named sub-dialogues Graphics Submenu
select ‘graphics’

Text Submenu
select ‘text’
Main
Menu

Paint Submenu

select ‘paint’
Concurrent dialogues - I
simple dialogue box

Text Style
bold
example italic
underline
Concurrent dialogues - II
three toggles - individual STNs

click on ‘bold’
NO
bold
bold bold

click on ‘italic’
NO
italic
italic italic

click on ‘underline’
NO
u’line
u’line underline
Concurrent dialogues - III
bold and italic combined
Text Style
bold
NO click on ‘bold’ bold example italic
style only underline

click click
on on
‘italic’ ‘italic’

italic click on ‘bold’ bold

only italic
Concurrent dialogues - IV
all together - combinatorial explosion
Text Style
NO ‘bold’ bold bold
style only example italic
‘underline’ ‘underline’ underline
‘italic’ ‘italic’
u’line ‘bold’ bold
only u’line

‘italic’ ‘italic’
italic ‘bold’ bold
only italic
‘underline’ ‘underline’

bold
italic ‘bold’ italic
u’line u’line
escapes

• ‘back’ in web, escape/cancel keys

• similar behaviour everywhere
• end up with spaghetti of identical behaviours
Graphics Submenu ESC
• try to avoid this select ‘graphics’
normal
finish
e.g. on high level diagram
‘normal’ exit for select ‘text’
Text Submenu ESC
each submenu Main
Menu normal
plus separate finish
escape arc active
‘everywhere’ in submenu Paint Submenu ESC

select ‘paint’ normal

finish
help menus

• similar problems
• nearly the same everywhere
• but return to same point in dialogue
• could specify on STN … but very messy
• usually best added at a ‘meta’ level
click on centre click on circumference
from
Circle 1 Circle 2 Finish
Menu rubber band draw circle

press HELP press HELP

button button

Help Subsystem Help Subsystem

Petri nets

• one of the oldest notations in computing!

• flow graph:
• places – a bit like STN states
• transitions – a bit like STN arcs
• counters – sit on places (current state)
• several counters allowed
• concurrent dialogue states
• used for UI specification (ICO at Toulouse)- Interact
cooperation object)
• tool support – Petshop
 Petri net example
Bold On Italic On
user presses user presses
‘Bold’ ‘Italic’

T1 T2 T3 T4

Bold Off Italic Off

user actions transition ‘fires’

represented when all input
as a new counter places have counters
State charts

• used in UML
• extension to STN Standby

• hierarchy
ON OFF
• concurrent sub-nets RESET
• history
• link marked H Sound Channel
goes back to last 1
state on re-entering H
On SEL
subdialogue 2
MUTE
SEL SEL
Off 3
SEL

4
 Delete D1

Flowcharts Please enter

employee no.: ____

C1
read record
• familiar to
programmers
Delete D2 Delete D3

• boxes Name: Alan Dix

Dept: Computing
Name: Alan Dix
Dept: Computing

- process/event delete? (Y/N): _ delete? (Y/N): _

Please enter Y or N

- not state
• use for dialogue answer?
C2 other

(not internal algorithm)

Y N Finish
• Tape - Update C3
delete record

Finish
it works!
• formal notations – too much work?
• COBOL transaction processing
• event-driven – like web interfaces
• programs structure Delete

Please enter
D1

≠ dialogue structure employee no.: ____

• used dialogue flow charts read record

C1

• discuss with clients Delete D2 Delete D3

• transform to code Name: Alan Dix

Dept: Computing
delete? (Y/N): _
delete? (Y/N): _
Please enter Y or N

• systematic testing C2

• 1000% productivity gain Y

answer?

N
other

Finish

• formalism saves time!! delete record

C3

Finish
JSD diagrams( Jackson structure
design)

• for tree structured dialogues

• less expressive Personnel
Record
• greater clarity System

login transaction
* logout

add change display delete

employee employee employee employee
record record record record
textual notations

grammars
production rules
CSP and event algebras
Textual - Grammars

• Regular expressions
sel-line click click* dble-click
• compare with JSD
• same computational model
• different notation
• BNF
expr ::= empty
| atom expr
| '(' expr ')' expr
• more powerful than regular exp. or STNs
• Still NO concurrent dialogue
Production rules

• Unordered list of rules:

if condition then action

• condition based on state or pending events

• every rule always potentially active

• Good for concurrency

• Bad for sequence
Event based production rules

Sel-line  first
C-point first  rest
C-point rest  rest
D-point rest  < draw line >

• Note:
• events added to list of pending events
• ‘first’ and ‘rest’ are internally generated events
• Bad at state!
Prepositional Production System

• State based
• Attributes:
Mouse: { mouse-off, select-line, click-point, double-click }
Line-state: { menu, first, rest }
• Rules (feedback not shown):
select-line  mouse-off first
click-point first  mouse-off rest
click-point rest  mouse-off
double-click rest  mouse-off menu
• Bad at events!
CSP and process algebras

• used in Alexander's SPI, and Agent notation

• good for sequential dialogues
Bold-tog = select-bold?  bold-on  select-bold? 
bold-off  Bold-tog
Italic-tog = . . .
Under-tog = . . .

• and concurrent dialogue

Dialogue-box = Bold-tog || Italic-tog || Under-tog

• but causality unclear

Dialogue Notations - Summary

• Diagrammatic
• STN, JSD, Flow charts
• Textual
• grammars, production rules, CSP
• Issues
• event base vs. state based
• power vs. clarity
• model vs. notation
• sequential vs. concurrent
 Semantics Alexander SPI (i)

• Two part specication:

• EventCSP - pure dialogue order
• EventISL - target dependent semantics

• dialogue description - centralised

• syntactic/semantic trade-off - tollerable

Semantics Alexander SPI (ii)
• EventCSP
Login = login-mess -> get-name -> Passwd
Passwd = passwd-mess -> (invalid -> Login [] valid -> Session)

• EventISL
event: login-mess
prompt: true
out: “Login:”
event: get-name
uses: input
set: user-id = input
event: valid
uses: input, user-id, passwd-db
wgen: passwd-id = passwd-db(user-id)
Semantics - raw code

• event loop for word processor switch ( ev.type ) {

case button_down:
if ( in_text ( ev.pos ) ) {
mode = selecting;
• dialogue description mark_selection_start(ev.pos);
- very distributed }
...
case button_up:
if ( in_text ( ev.pos )
• syntactic/semantic trade-off && mode == selecting ) {
- terrible! mode = normal;
mark_selection_end(ev.pos);
}
...
case mouse_move:
if (mode == selecting ) {
extend_selection(ev.pos);
}
...
} /* end of switch */
Action properties

• completeness
• missed arcs
• unforeseen circumstances
• determinism
• several arcs for one action
• deliberate: application decision
• accident: production rules
• nested escapes
• consistency
• same action, same effect?
• modes and visibility
Checking properties (i)

• completeness
• double-click in circle states? double
?
click
click on
click on centre circumference
Circle 1 Circle 2 Finish
rubber band draw circle
select 'circle'

Start Menu

select 'line' click on

first point double click
Line 1 Line 2 Finish
rubber band draw last
line

click on point
draw a line
Checking properties (ii)
select 'circle'
... ... ...
Start Menu

select 'line' click on

first point double click
Line 1 Line 2 Finish
rubber band draw last
• Reversibility: line

click on point
• to reverse select `line' draw a line

Graphics Sub-menu
select 'graphics'

Main
Menu
select 'text' ... ... ...
... ... ...
select 'paint'
Checking properties (ii)
select 'circle'
... ... ...
Start Menu

select 'line' click on

first point double click
Line 1 Line 2 Finish
rubber band draw last
• Reversibility: line

click on point
• to reverse select `line' draw a line

• click

Graphics Sub-menu
select 'graphics'

Main
Menu
select 'text' ... ... ...
... ... ...
select 'paint'
Checking properties (ii)
select 'circle'
... ... ...
Start Menu

select 'line' click on

first point double click
Line 1 Line 2 Finish
rubber band draw last
• Reversibility: line

click on point
• to reverse select `line' draw a line

• click - double click

Graphics Sub-menu
select 'graphics'

Main
Menu
select 'text' ... ... ...
... ... ...
select 'paint'
Checking properties (ii)
select 'circle'
... ... ...
Start Menu

select 'line' click on

first point double click
Line 1 Line 2 Finish
rubber band draw last
• Reversibility: line

click on point
• to reverse select `line' draw a line

• click - double click - select `graphics'

• (3 actions)
Graphics Sub-menu

• N.B. not undo select 'graphics'

Main
Menu
select 'text' ... ... ...
... ... ...
select 'paint'
State properties

• reachability
• can you get anywhere from anywhere?
• and how easily

• reversibility
• can you get to the previous state?
• but NOT undo

• dangerous states
• some states you don't want to get to
Dangerous States

• word processor: two modes and exit

F1 - changes mode
F2 - exit (and save)
Esc - no mode change

Esc

F1 F2
edit menu exit
but ... Esc resets autosave
Dangerous States (ii)

• exit with/without save  dangerous states

• duplicate states - semantic distinction
F1 F2
edit menu exit

any Esc
update

F1 F2
edit menu exit
F1-F2 - exit with save
Esc
F1-Esc-F2 - exit with no save
Lexical Issues

• visibility
• differentiate modes and states
• annotations to dialogue
• style
• command - verb noun
• mouse based - noun verb
• layout
• not just appearance ...
layout matters

• word processor - dangerous states

F1 F2
edit menu exit

any Esc
update

• old keyboard - OK edit

F1
menu
F2
exit

Esc
Esc 1

tab ...
F1 F2
...
F3 F4
... ...
layout matters

• new keyboard layout

Esc F1 F2 F3 ...

intend F1-F2 (save) edit

F1
menu
F2
exit

finger catches Esc any Esc

update

F1 F2
edit menu exit

Esc
layout matters

• new keyboard layout

Esc F1 F2 F3 ...

intend F1-F2 (save) edit

F1
menu
F2
exit

finger catches Esc any Esc

update
F1-Esc-F2 - disaster! F1 F2
edit menu exit

Esc
Dialogue Analysis - Summary

• Semantics and dialogue

• attaching semantics
• distributed/centralised dialogue description
• maximising syntactic description

• Properties of dialogue
• action properties: completeness, determinism,
consistency
• state properties: reachability, reversibility, dangerous
states

• Presentation and lexical issues

• visibility, style, layout
• N.B. not independent of dialogue
Dialogue Analysis - Summary

• Semantics and dialogue

• attaching semantics
• distributed/centralised dialogue description
• maximising syntactic description

• Properties of dialogue
• action properties: completeness, determinism,
consistency
• state properties: reachability, reversibility, dangerous
states

• Presentation and lexical issues

• visibility, style, layout
• N.B. not independent of dialogue
Digital watch – User Instructions
Time display Stop watch

S MTWTF S S MTW TF S

A STP

• two main modes

Depress
• limited interface A A
button A
for 2 seconds
- 3 buttons
S MTWTF S S MTW TF S

SET A ALM

• button A AM
changes mode
Time setting Alarm setting
Digital watch – User Instructions
Time display Stop watch

S MTWTF S S MTWTF S

A STP

• dangerous states
• guarded
… by two second hold Depress
button A
A A for 2 seconds

• completeness S MTWTF S

SET A
S MTWTF S

ALM

• distinguish depress A AM
and release A
• what do they do Time setting Alarm setting
in all modes?
Digital watch – Designers
instructions Time display Stop watch

S MTWTFS S MTWTFS

STP

and ...
Depress A

Release A Release A

S MTWTFS S MTWTFS

that’s just STP

one button

2 seconds 2 seconds
Depress A

S MTWTFS S MTWTFS
Release A Release A
SET Depress A ALM

AM

Time setting Alarm setting