A Level Computer Science Book 2
A Level Computer Science Book 2
Kevin R Bond
Every effort has been made to trace copyright holders and to obtain their permission for the use
of copyrighted material. We apologise if any have been overlooked. The author and publisher will
gladly receive information enabling them to rectify any reference or credit in future editions.
ISBN 978-0-9927536-6-5
The right of Kevin R Bond to be identified as author of this work has been asserted
by him in accordance with the Copyright, Designs and Patents Act 1988.
The core content of this digital textbook has been approved by AQA for use with
our qualification. This means that we have checked that it broadly covers the
specification and that we are satisfied with the overall quality. We have also approved
the printed version of this book. We do not however check or approve any links or
any functionality. Full details of our approval process can be found on our website.
We approve print and digital textbooks because we know how important it is for
teachers and students to have the right resources to support their teaching and
learning. However, the publisher is ultimately responsible for the editorial control
and quality of this digital book. Please note that when teaching the A-level (7517)
course, you must refer to AQA’s specification as your definitive source of information.
While this digital book has been written to match the specification, it cannot
provide complete coverage of every aspect of the course. A wide range of other useful
resources can be found on the relevant subject pages of our website: aqa.org.uk
Chapter 5.1.1
Figure 1.1.1 “Late Babylonian clay tablet: table of numerals representing
lunar longitudes”, image ID 00851897001, British Museum
Pages 1, 2: Clip art www.123rf.com: green apple: 123rf / 14199537; red apple: 123rf / 1419906;
banana: 123rf / 39056131; orange: 123rf / 38547844; purse: 123rf / 27347756
Chapter 5.1.2
Page 4: Thermometer - “Thermometre froid a plat” Fotolia / 11368653 © Albachiara
Page 5: Cake - 123rf / 33382329 (www.123rf.com)
Chapter 5.1.4
Page 12: Greek character - 123rf / 32698394 (www.123rf.com)
Chapter 5.1.5
Page 14: Road going off into the desert - www.canstockphoto / csp9388362
Chapter 5.1.7
Page 20: Dreaming sheep - Shutterstock / 110338271; Page 20: Ruler - Shutterstock / 198850166
Chapter 5.2.1
Figure 2.1.1 Microsoft® Windows® 7 Calculator screenshot used with permission from Microsoft
Microsoft and Windows are either registered trademarks or trademarks of
Microsoft Corporation in the United States and/or other countries
Figure 2.1.4 Microsoft® Windows® 7 Device Manager version
6.1.7600.16385 screenshot used with permission from Microsoft
Chapter 5.3.1
Page 34: Penguins - Shutterstock / 114208987; Page 34: Peacock - 123rf / 36970936 (www.123rf.com)
Page 34/35: Highway code signs used in question 2, and page 35 are based on Highway Code signs,
© Crown copyright 2007, and are reproduced under Open Government Licence v3.0.
Page 35: Tree rings - Shutterstock / 97674011; Figure 3.1.4 red apples - 123rf / 1419906 (www.123rf.com)
Page 36: pound coin showing head - 123rf / 20150613_ml (www.123rf.com)
Page 36: pound coin showing tail - 123rf / 35831780 (www.123rf.com)
Chapter 5.3.2
Figure 3.2.2 Microsoft® Windows 7 command line window screenshot used with permission from Microsoft
Figure 3.2.3 Apple® MacBookPro® and OS X® are trademarks of Apple Inc., registered in the U.S. and other countries
Chapter 5.4.3
Page 52: CPU - Shutterstock / 222009121
Chapter 5.5
Figure 5.5.1 Shutterstock / 157001045; Figure 5.5.2 Shutterstock / 1226401
Figure 5.5.3 Shutterstock / 185237537; Figure 5.5.5 123rf / 31206024 (www.123rf.com)
Figure 5.5.6 123rf / 32168839 (www.123rf.com)
Chapter 5.6.2
Figure 5.6.2.2 Adapted from www.engineeringtoolbox.com/air-altitude-pressure-d462.html
with kind permission of the editor
7.3 Structure and role of the processor and its components 270
7.3.1 The processor and its components 270
7.3.2 The Fetch-Execute cycle and the role of the registers within it 278
7.3.3 The processor instruction set 280
7.3.4 Addressing modes 286
7.3.5 Machine-code and assembly language operations 288
7.3.6 Interrupts 304
7.3.7 Factors affecting processor performance 308
8.1 Individual (moral), social (ethical), legal and cultural issues and
opportunities 330
8.1 Introduction 330
Index 597
13.1 Aspects of software development - See Unit 1
Glossary - www.educational-computing.co.uk/CS/Unit2/Glossary.pdf
Exam practice questions -
www.educational-computing.co.uk/CS/Unit2/ExamPracticeQuestions.pdf
The structure and content of this book maps to sections 4.5 to 4.12 of AQA’s A-level Computer Science specification
(7517). For example, the chapter number of the first chapter is 5.1.1 and its title is Number systems: Natural
numbers. This chapter maps to section 4.5.1.1 of AQA’s A-level Computer Science specification (7517). The chapters
in the book do not use the leading 4 as this designates Subject content – A-level in the specification.
Flipped classroom
This textbook has been written with the flipped classroom approach very much in mind. This approach reverses the
conventional classroom lesson and homework model of teaching. Instead, chapters in this textbook should be used
to prepare for a lesson so that classroom-time can be devoted to exercises, projects, and discussions.
Key concept Concepts that you will need to understand and to be able to define or
explain are highlighted in blue and emboldened, e.g. Integers. The same
concepts appear in the glossary for ease of reference.
Key principle Principles that you will need to understand and to be able to define or
explain are highlighted in blue and emboldened, e.g. Abstraction. The same
principles appear in the glossary for ease of reference.
Information Background
References information that has the potential to assist and contribute to a student’s learning, e.g. Read Unit 1 section
4.2.2 for more background on sets and set comprehension. Background knowledge that could also contribute to a
student’s learning.
Did you know? Extension Material
“Did you know?” - interesting facts to enliven learning. “Extension Material” - content that lies beyond the
specification.
Task Activity to deepen understanding and reinforce learning.
Programming tasks Practical activity involving the use of a programming language to deepen
understanding and reinforce learning of concepts and principles.
Questions Short questions that probe and develop your understanding of concepts
and principles as well as creating opportunities to apply and reinforce your
knowledge and skills.
If you are reading this book then you will already have So far it has not been necessary to mention digital
chosen to be a part of an exciting future, for Computer computers. Digital computers are just the current
Science is at the heart of an information processing means by which algorithms can be implemented to
revolution. This revolution applies not just to seeking execute on data. Both algorithms and the models on
patterns of meaning in data accumulated on an which they act need to be implemented: algorithms in
unprecedented scale by the huge growth in connected the form of code or instructions that a digital computer
computing devices but also the realisation that all can understand, i.e. a computer program; models in
forms of life are controlled by genetic codes. Genetic data structures in a programming language. Unit 1
codes are instructions in a procedural information sense was largely about the fundamentals of programming,
that together with the environment that they inhabit data structures, algorithms and their efficiency,
control and guide the development of organisms. i.e. algorithms to run quickly while taking up the
minimal amount of resources (e.g. memory, hard disk,
Computer scientists concern themselves with electricity), and the limits of computation.
• representations of information in patterns of
symbols, known as data or data representations, Unit 2 covers the fundamentals of computing devices,
• the most appropriate representation for this data how data is represented and communicated between
• the procedures in the form of instructions devices, the logic gate circuits that enable computing
that can transform this data into new forms of devices to perform operations and to store information.
information. It covers the fundamentals of computer organisation
The procedures themselves are also a form of and architecture, the structure and role of the processor,
information of an instructional kind. the language of the machine, binary (machine code)
and how it is used to program the hardware directly.
The key process in Computer Science is abstraction The fundamentals of networking are covered which
which means building models which represent aspects leads onto data models for storing structured and
of behaviour in the real-world which are of interest. unstructured data that can be accessed from networked
For example, if we wanted to build an automated machines. The simpler case of structured data is
recommendation system for an online book store, we covered first using the relational database model. The
might choose to record the types of book and number limitations of this model are exposed for data that
of each type purchased as well as details that identify lacks structure and which is too big to fit into a single
the respective customer. server. Such data is known as “Big Data”. Machine
learning techniques are needed to discern patterns
Computer Science is not alone in building abstractions, in this data and to extract useful information. Big
mathematics and the natural sciences also build Data also requires a different programming paradigm,
abstractions but their models only serve to describe and functional programming, one that facilitates distributed
explain whereas Computer Science must, in addition, programming.
perform actions on and with the data that has been
modelled if it is to solve problems. These actions are It is right that having journeyed through Unit 1 and
described by algorithms or step-by-step instructions Unit 2 that a student should have an opportunity to
which form what is called the automation stage of discuss using hypotheticals and case studies what kind
problem solving. Whilst it is true that automation of of philosophy of information is appropriate for any
tasks existed before Computer Science, their nature advanced information society. This is explored in Unit
involved concrete, real-world objects, e.g. the Jacquard 2 in the section Consequences of uses of computing
loom, not informational abstractions such as an online where guiding principles of behaviour are explored.
book recommendation system.
Single licence - Abingdon School xii
5 Fundamentals of data representation
5.1 Number systems
Learning objectives:
■■Concept of number
■■The natural numbers
■■ 5.1.1 Natural numbers
What does it mean to count?
■■Numerals We learn to count from an early age. We notice that in the real world objects
can be grouped together in collections, for example
Task three apples. In doing so, we use abstraction in ignoring
Try your hand at counting: the differences between the individual apples in the
1
https://www.youtube.com collection – for example, one of them is green, the other two are red.
/watch?v=vJG698U2Mvo
video.
The concept of number
By considering collections of items we can get an understanding of the concept
of number. For example, a collection of three
Key principle
oranges, a collection of three bananas. If we
Abstraction: choose to ignore the differences between these
An abstraction is a collections and concentrate on their similarity, then we can form a relatively
representation that is arrived
abstract concept of the number three. The same process could lead to the
at by ignoring or removing
unnecessary detail. concept of the number 4, 5 and so on.
(Use http://en.wikipedia.org/
wiki/Babylonian_numerals)
0 0 3
pictorial, but again there is some choice in the
numerals to represent a number. For example,
both 3 and 03 (and indeed 003 and so on) are all
recognised as valid numerals, representing the same number.
Numeral systems
V A numeral system (or system of numeration) is
VI a writing system for expressing numbers, using
VII symbols in a consistent manner.
Questions
1 For the following numbers represented by Roman numerals,
change the symbols from Roman numeral representation to
the equivalent Arabic numeral representation:
(a) VII (b) LXXVII (c) MCMXCVI
Digits of numerals
In a basic digital system, a numeral is a sequence of digits, which may be of
arbitrary length. The most commonly used system of numerals is the Hindu–
Arabic numeral system, based on Hindu numerals. It uses ten symbols called
digits (0, 1, 2, 3, 4, 5, 6, 7, 8, and 9) to represent any number, no matter how
large or how small. This system is referred to as the decimal or denary system.
Key concept
What is a counting number?
Natural number:
We use the counting numbers {1, 2, 3, 4, ... } to keep
Natural numbers are the count-
ing numbers, either track of things such as how much money we have in our
{1, 2, 3, ... }, or {0, 1, 2, 3, ... }. pocket. The braces {} indicate a set (a collection of objects).
The objects in the set are written inside the braces. “…”
The symbol ℕ1 is used to indicates that there are infinitely more objects. Informally,
denote the set {1, 2, 3, ... },
the counting numbers are all the numbers you can get to by counting, starting
and the symbol ℕ0 is used to
denote the set {0, 1, 2, 3, ... }. at 1.
1 + 2 + 3 + 4 + 5 + ....... + 96 + 97 + 98 + 99 + 100
100 + 99 + 98 + 97 + 96 + ....... + 5 + 4 + 3 + 2 + 1
The alternative to Gauss’ method involves laboriously performing 99 addition steps, adding 1 to 2 then 3 to the
resulting sum and so on. This long-winded and laborious calculation is an example of a brute-force approach.
Questions
2 Find the sum of the following range of natural numbers using Gauss’s method
Programming tasks
1 Write a program to find the sum of the natural numbers from 1 to n. Your program should use
the brute-force approach. Test your program with the following values of n
(i) 100 (ii) 1000 000
2 Write a program to find the sum of the natural numbers from 1 to n. Your program should use
the formula approach. Test your program with the following values of n.
(i) 100 (ii) 1000 000
Investigation
1 Compare the execution times of programming tasks 1 and 2 for the two test values. What do
you observe?
So, integers can be negative {-1, -2, -3, -4, -5, … }, positive {1, 2, 3, 4, 5, … },
Information or zero {0}.
ℤ
Read Unit 1 section 4.2.2 for The special symbol ℤ is used to denote the set of integers
more background on sets and
set comprehension: ℤ = { ..., -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, ... }
{m – n: m ∊ ℕ and n ∊ ℕ}
means to generate the members
There are infinitely many elements in this set.
of the set ℤ by subtracting n
from m where m ∊ ℕ means m Questions
is a member of the set ℕ and n
∊ ℕ means n is a member of the 1 Are the following numbers (Hindu-Arabic numerals
set ℕ. representing numbers) integers?
(a) -10 (b) 5⅓ (c) 3.5?
Programming
2 Is the result of evaluating the following expression an integer?
Task
1 What are the maximum and (367 × 42) ⧸ 7
minimum integers for the
programming language that
you use?
If your programming
language has several integer
data types, find these values
for all the supported integer
data types.
■■ Whole numbers
Key concept What are whole numbers?
We will consider whole numbers to be numbers without a fractional part
Whole number:
although other definitions of whole numbers exist which take a different
Whole number is another name
for an integer number. interpretation. A fractional part is a fraction. A
fraction is any number greater than 0 and less
than 1. For example, a slice of cake is a part of the
whole, say ∕₁₀ and clearly not the whole. On the
other hand, we may have 3 whole cakes or 3 whole degrees
of temperature below zero, i.e. -3.
Information
The word “rational” comes
Formally, ℚ is the set of rational numbers.
ℚ
from the word “ratio” because a There are infinitely many elements in this set.
rational number can always be
written as the ratio, or quotient, Questions
of two integers.
1 Determine whether each of the following statements is true
or false:
(a) -13 ∊ ℕ (b) ³⁶∕₇₇ ∊ ℚ (c) -11 ∊ ℚ?
Write a program for this algorithm. Does your program meet the
two conditions? If your programming language supports assertions,
then use assertions to check that the two conditions are met. If not
use appropriate tests.
Table 1.3.1
If you think about it all possible positive rational numbers will be generated,
e.g.¹⁴⁷⁄₉₁₄₅₇ will be in the table at the intersection of row 147 and column Key point
91457 and will get added in turn. Therefore, it is possible to order the rational
ℚ is countable:
numbers of the set ℚ⁺ and to show a one-to-one correspondence with the It is possible to order the
natural numbers as indicated in Table 1.3.2. rational numbers, ℚ,
and to show a one-to-one
correspondence with the natural
1 2 3 4 5 6 7 8 9 10 11 … numbers. Therefore, ℚ is
1⁄1 2⁄1 1⁄2 1⁄3 3⁄1 4⁄1 3⁄2 2⁄3 1⁄4 1⁄5 5⁄1 … countable.
Table 1.3.2
To generate ℚ, we can place zero /1 before 1/1 and insert the negative of each
positive rational number other than zero immediately after the positive rational
number as shown in Table 1.3.3.
1 2 3 4 5 6 7 8 9 …
0⁄1 1⁄1 -1⁄1 2⁄1 -2⁄1 1⁄2 -1⁄2 1⁄3 -1⁄3 …
Table 1.3.3
Table 1.3.3 shows that it is possible to order the rational numbers ℚ and
to place them in a one-to-one correspondence with the natural numbers.
Therefore, the set of rational numbers ℚ is countable. It is also infinite because
the set of natural numbers is infinite.
In addition, all integers are in ℚ because every integer n can be expressed as
n/1.
Representing rational numbers as terminating decimals
Long division is used to convert a rational number into decimal form.
Examples: (a) 189⁄9 = 189 ÷ 9 = 21 (b) 13⁄20 = 13 ÷ 20 = 0.65
9 goes into 189 20 goes into 13
Key point
21 times remainder 0 0 times remainder 13
∙ 20 goes into 130 The rational numbers in lowest
form whose only prime factors
6 times remainder 10
in their denominator are 2 or 5
20 goes into 100 or both, convert to terminating
5 times remainder 0 decimals.
If the final remainder is 0, e.g. 189⁄9, the quotient is a whole number, e.g. 21,
or a finite or terminating decimal, i.e. a decimal with a finite number of digits
after the decimal point, e.g. 0.65.
The rational numbers in lowest form whose only prime factors in their
denominator are 2 or 5 or both, convert to terminating decimals.
The repeating pattern may consist of just one digit or of any finite number
of digits. The number of digits in the repeating pattern is called the period.
The repeating pattern is indicated by placing a period mark or a bar over
each digit in the repeating pattern, e.g.
(a) ¹⁄₃ = 0.333… = 0.3
(b) ⁸⁄₁₁ = 0.727272… = 0.72
Questions
3 Convert the following rational numbers to decimal:
(a) ¹⁶⁄ ₃ (b) ¹⁰∕₇ (c) ¹³∕₁₁
Questions
5 Convert the following decimals to their rational number
equivalent:
(a) 5.25 (b) 0.90
0 1 2
1
1 3
1
1
2 3 5
2 7
1 4 3 5 4 7 9 4 11 13 4 15
8 8 8 8 8 8 8 8
1 2 4 5
3 3 3 3
x = m⁄n
Information
2
Proof by contradiction:
It follows that x2 = m ⁄n2 because x = √2
We assume that what we
want to prove is not true, and
Therefore, m2 = 2n2
then show the consequences
contradict either what we have And so m2 is even. This implies that m is even.
just assumed, or something
we already know to be true (or
both). We may therefore write m = 2k (Multiplying k, any natural number, by 2
ensures evenness).
Substituting 2k for m in m2 = 2n2
Or 4k2 = 2n2
2k2 = n2
Rearranging n2 = 2k2
Thus n is even.
Questions
1 Which of the following numbers are irrational?
Key point
½ √3
�
Figure 1.5.1 The real number line
Real number line:
The real number line is a useful
Ideally, one would like to show and label every point on the number line, but
way of modelling the set of real no matter how dense one makes the points there are always points in between.
numbers. What is the set of real numbers?
The set interpretation of real numbers
ℚ Irrational is an alternative to thinking of real
Key concept
numbers numbers as points on an infinitely long
Set of real numbers:
ℤ line. The special symbol ℝ is used to
The set of real numbers, ℝ, is
ℕ denote the set of real numbers. It is
formed from the union of the a set formed from the union of the
set of rational numbers and the set of rational numbers and the set of
set of irrational numbers.
irrational numbers – Figure 1.5.2.
Figure 1.5.2 The composition of the set of real numbers, ℝ
Questions
Key point
1 Determine whether each of the following statements is true
The set ℝ is an uncountable set. or false:
Real numbers have the property
(a) ¹⁶∕₃ ∈ ℝ (b) 3.142 ∈ ℝ
that between any two of them,
no matter how close, there lies (c) ¹∕₄ ∈ ℚ and ¹∕₄ ∈ ℝ (d) π ∈ ℝ (e) √2 ∈ ℚ
another real number. (f ) √361 ∈ ℕ and √361 ∈ ℝ (g) −3 ∈ ℝ
Investigation
Find out why 0.9999… and 1.0000… represent the same real
number.
■■ The set of real numbers, ℝ, is formed from the union of the set of rational
numbers and the set of irrational numbers
■■ The set ℝ is an uncountable set
■■ Real numbers describe real-world quantities such as distances, amounts of
things, temperature, and so on.
In English, this is
first, second, third, fourth, fifth, etc.
We could also have started at 0 and labelled the first element as the zeroth
element in which case we use the natural numbers including zero as follows
Questions
1 An index is used in programming to specify the position of an
Information
element in an ordered collection, e.g. an array. An index may
An array index is a way of start from 0 or 1 depending on programming language used or
labelling a cell of an array. The programmer preference.
term subscript is also used to
(a) If you were told that i was the 100th element of an array:
mean the same thing.
(i) How many elements would you consider come
before this element?
(ii) What would be the index for this 100th element
if the indexing starts at 0?
(iii) What would be the index for this 100th element if the
indexing starts at 1?
(b) If you were told that there are 100 elements in an array:
(i) What would be the index of the last element if
the indexing starts at 0?
(ii) What would be the index of the last element if the indexing
starts at 1?
Discussion
Programming Task
1 Write a program that accepts a letter of the alphabet typed
at the keyboard, uppercase or lowercase, and outputs its
numeric position in the alphabet followed by either “st”,
“nd”, “rd” or “th” as appropriate.
Clearly, using this ruler, we would not be able to measure the length of a piece
of wood to hundredths of a centimetre, but instead are limited to measuring
Key point to tenths of a centimetre. We conclude that when dealing with fractions, we
When making measurements shall have to approximate some by using a suitably close fraction that does
we are limited to using a have a representation on the ruler, e.g. ¹⁄₉ would be approximated by ¹⁄₁₀
rational number approximation with an error of ¹⁄₉ − ¹⁄₁₀ = ¹⁄₉₀. The more divisions that we have on the
to a real number.
ruler, the better our approximation can be, but the need for approximation
cannot be removed. Between one tenth and two tenths, for example, there are
infinitely many fractions. We are limited therefore to using a rational number
approximation to a real number.
Questions
1 Calculate the decimal expansion of ¹⁄₉.
Key point
2 If the decimal expansion of ¹⁄₉ is restricted to each of the
All fractional representations
following number of decimal places, what fraction results in
run the risk of being imprecise.
each case
(a) 2 (b) 3 (c) 6?
Programming task
1 Write a program that outputs the result of performing the
following calculations:
(a) ¹⁄₃
(b) ¹⁄₇
(c) ¹⁄₅
(d) ¹⁄₁₀
Comment on the results.
Questions
4 For each real number x where x ≥ 0 write down
rn and rn + ¹⁄₁₀n
Rounding off
We usually apply the process of rounding off to real numbers when using a
rational number approximation. The rules for rounding off to n decimal places
are:
■■ If the value of the (n + 1)th digit is less than five (0, 1, 2, 3, or 4), we
leave the nth digit alone.
■■ If the value of the (n + 1)th digit is greater than or equal to five
(5, 6, 7, 8, or 9), we increase the value of the nth digit by one.
Questions
5 Round off the following real numbers to the specified
number of decimal places
(a) x = 0.3245995632……… to 4 decimal places
(b) x = 85.994467285……… to 3 decimal places
(c) x = 5.884467285……… to 3 decimal places
Binary (base 2)
The number base of the binary system is two because it has two digits 0, 1 and
the digit multiplier is a power of two, 2n where n is … −3, −2, −1, 0, 1, 2 , 3, …
Key concept
… 24 23 22 21 20 2−1 2−2 …
Decimal: … 16 8 4 2 1 …
¹⁄₂ ¹⁄₄
The number base of the decimal
1 0 1 1 1 1 1
system is ten because it has ten
digits 0, 1, 2, 3, 4, 5, 6, 7, 8, Table 2.1.3 Place values for the binary system
9 and the digit multiplier is a
power of ten, 10n where n is The number in decimal represented by the binary numeral 10111•11
…, −3, −2, −1, 0, 1, 2 , 3, …
is constructed using the place values in Table 2.1.3 as follows
1 × 16 + 0 × 8 + 1 × 4 + 1 × 2 + 1 × 1 + 1 × ¹⁄₂ + 1 × ¹⁄₄
To indicate the base we use a subscript attached to the numeral, e.g. 10111•112.
Now the “There are 10 types of people in the world those that understand
Key point binary and those that don’t” quote might make more sense because
To indicate the base we use a 10Binary = 2Decimal
suffix attached to the numeral,
e.g. 3410.
Questions
1 Write out each of the following in the form
Key concept (a) 10102 (b) 11112 (c) 100101112 (d) 111111112
Binary:
3 Convert the following numbers expressed in binary to their
The number base of the binary
system is two because it has decimal equivalent:
two digits 0, 1 and the digit
multiplier is a power of two, 2n (a) 10•102 (b) 0•11112 (c) 100•101112 (d) 1•11111112
where n is
…, −3, −2, −1, 0, 1, 2 , 3, …
Key concept
… 161 160 …
… 16 1 … Hexadecimal:
D 4 The number base of the
hexadecimal system is sixteen
Table 2.1.4 Place values for the hexadecimal system because it has sixteen digits 0,
1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C,
The hexadecimal digits A, B, C, D, E and F are in decimal 10, 11, 12, 13, 14
D, E, F and the digit multiplier
and 15 respectively. is a power of sixteen, 16n where
The number in decimal represented by the hexadecimal numeral 38AD4•95 is n is
…, −3, −2, −1, 0, 1, 2 , 3, …
constructed using the place values in Table 2.1.5 as follows
3 × 65536 + 8 × 4096 + 10 × 256 + 13 × 16 + 4 × 1 + 9 × 1/16 + 5 × 1/256
To indicate the base we use a subscript attached to the numeral, e.g. Information
38AD4•9516.
In some programming
languages, e.g. Java, a number
… 164 163 162 161 160 16−1 16−2 …
represented in hexadecimal is
… 65536 4096 256 16 1 ¹⁄₁₆ ¹⁄₂₅₆ …
indicated by placing 0x before
3 8 A D 4 9 5 the numeral, e.g. 0x3c4 or
0x3C4.
Table 2.1.5 Some more place values for the hexadecimal system
Questions
4 Write out each of the following in the form
… 26 25 24 23 22 21 20 …
… 64 32 16 8 4 2 1 …
0 1 0 0 0 1 1
Questions
6 Convert the following numbers expressed in decimal to their binary
equivalent using Method 1.
Questions
7 Convert the following numbers expressed in decimal to their binary
equivalent using Method 2. Show the intermediate results in a table
with structure similar to Table 2.1.7.
∴ 62 = 2 × 31 + 0
We call 0 the remainder. In this example, 2 goes into 62, 31 times exactly.
On the other hand, if a decimal number, n, is odd then there is some integer, k
for which
n = 2k + 1
e.g. n = 63, k = 31
∴ 63 = 2 × 31 + 1
We call 1 the remainder. In this example, 2 does not divide 63 exactly.
The first 1 or 0 remainder is the least significant bit of the decimal number’s
binary equivalent and the final remainder 1 or 0 remainder, the most significant
bit of the binary equivalent.
Successive division algorithm decimal to binary
For decimal number, n.
Make k the value of n
If k is equal to 0 write down the answer 0
While k is not equal to 0
Make the new value of k the old value
divided by 2 using integer division
If this is the first pass write down remainder
Else write down remainder to the left
of the previous remainder
Programming task
1 Code this successive division algorithm in a programming language
with which you are familiar. Test your program by converting the
following decimal numbers
(a) 010 (b) 2410 (c) 5910 (d) 12710 (e) 3310
Questions
Programming task
2 Code this successive division algorithm in a programming language
with which you are familiar. Test your program by converting the
following decimal numbers
(a) 010 (b) 4710 (c) 30210 (d) 6551710 (e) 28556210
You could use this calculator to check your answers to questions about number
bases.
B47A16 = 10110100011110102
1011 0100 0111 1010
The method relies on the fact that the hexadecimal digits 0 to F map to 0 to
15 in decimal and this decimal range can be coded by just four binary digits.
When a number represented in four binary digits is multiplied by 1610, it
becomes a number represented by eight binary digits with zeroes in the least
significant four bit positions, twelve binary digits when multiplied by 1610 again
and so on.
1 0 1 1 × 1610 = 1 0 1 1 0 0 0 0
Questions
9 Convert the following numbers expressed in hexadecimal to their
binary equivalent using the method described above.
10110100011110102 = B47A16
B47A
Questions
10 Convert the following numbers expressed in binary to their
hexadecimal equivalent using the method described above.
(d) 1100111000112
Figure 2.1.4 Microsoft® Windows® Device manager showing the allocation of memory
Task
Questions
Key principle
A representation is a pattern of
1 State the information conveyed by the following symbols:
♩ ♪
symbols that conveys information,
e.g. a pattern of 1s and 0s. He Ar
Bits
We have seen how to represent the
natural numbers in the decimal Figure 3.1.4 A collection of apples Did you know?
and the binary numeral systems.
Bit is derived from the b of binary
In the binary system, we use just two symbols, the binary digits 0 and 1 and the it of digit.
to represent a natural number, for example, the number of apples shown
in Figure 3.1.4 in binary is 1012. Rather than use the term binary digit
we can abbreviate it to bit. So we need three bits to count five apples.
This digital data 1012 encodes the information that we have five apples. If
we remove one apple, our digital data must change to 1002 to convey the
new information that we have four apples. Note that digital data always
changes in discrete steps. The minimum step in our apples’ example is one.
Removing more apples, say three, leaves just one apple and to convey this
information we must change our digital data to 12. Removing the last apple
leaves none and our digital data becomes 02 to convey this information.
In a similar manner, the datum True and the datum False in
Figure 3.1.3 could be encoded as 1 and 0, respectively.
Coin tossing
Now rather than counting objects and recording their number, let’s suppose
that we wish to convey the outcome of a coin tossing experiment. For the
experiment, assume our coin will land either head up (h) or tail up (t) with h
equal probability when tossed and we will call a single toss of the coin, a trial.
Now before we toss the coin, we cannot say that the outcome is (h) or the
outcome is (t). We are in a state of data deficit and therefore, in possession t
of no information about the outcome. However,
2-bit
ht
Outcome as soon as the coin is tossed we have an outcome,
encoding
(h) or a (t) which we can represent as 1 or 0. We
(tt) 00
now have data, a single bit of data which conveys
(th) 01
information, the result of the trial. We say that
(ht) 10
to represent the outcome we require one bit
(hh) 11
of information or one bit of information per
Table 3.1.1 symbol where a symbol is either (h) or (t).
Questions
3 Complete a copy of Table 3.1.2 by replacing the blanks in the
Alphabet column and entering the missing information in the Bits of
information per symbol column.
No of Bits of information
Alphabet
coins per symbol
2 equiprobable symbols
1 1
(h), (t)
4 equiprobable symbols
2 2
(hh), (ht), (th), (tt)
8 equiprobable symbols
3 (hhh), (___), (___), (___)
(thh), (___), (___), (ttt)
16 equiprobable symbols
(hhhh), (____), (____), (____)
4 (hthh), (____), (____), (____)
(thhh), (____), (____), (____)
(tthh), (____), (____), (tttt)
Table 3.1.2
Key principle
The bit is the fundamental unit of information
Unit of information:
The bit is the fundamental unit of Imagine a machine that can answer only “42” to any question. There
information. is no data deficit because the answer to any question can be predicted
with absolute certainty, it is always the symbol “42”. Therefore, the
machine produces an amount of information which is zero.
Task
1 Investigate whether or not the programming language that you use
has a byte data type. If it doesn’t how could one be created for use in
programs that you might write?
16 ways
2 ways
4 ways
8 ways
5 How many different values are possible for the following number of
bytes
(a) 1 (b) 2 (c) 8?
Express your answers as powers of 2.
Questions
6 Write all possible bit patterns for 4 bits and their corresponding
decimal natural number values in table format.
Figure 3.2.3 shows the use of Gi and Ki for reporting disk storage capacity
using the command df –h in terminal mode on an Apple® MacBook Pro®
running operating system OS X® 10.8.5. The About This Mac window is also
shown.
Task
Powers of 2
Table 3.2.1 shows some numbers in decimal numerals expressed as powers of
2 and their equivalent binary numeral. In 2 raised to the power of 10, 10 is
known as the exponent. The exponents 10, 20, 30, 40 specify the number of
zeroes in the binary numeral.
Decimal Power
Binary numeral
numeral of 2
1024 210 10000000000
1048576 220 100000000000000000000
30
1073741824 2 1000000000000000000000000000000
40
109951162776 Table
2 3.2.1100000000000000000000
Decimal numerals expressed as powers of 2
00000000000000000000
Questions
1 Express the following binary numerals in the form 2n.
(a) 10002 (b) 10000002 (c) 10000000000000002
Questions
3 Convert the following to bytes
(a) 1MiB (b) 1.5KiB (c) 1.75GiB
101 10 1
Table 3.2.5 shows how to express a decimal numeral which is a power of 10 in
Figure 3.2.4 Powers of 10 units of k, M, G and T.
If the decimal numeral refers to a quantity of bytes then we can express the
quantity using the units of k, M, G and T.
Questions
6 Express the following decimal numerals in the form 10n
(a) 1000 (b) 1000000 (c) 10000000
7 Convert the following quantities in bytes to kB
(a) 1000 (b) 10000
8 Convert the following quantities in bytes to MB
(a) 500000 (b) 2000000 (c) 30000000
Questions
9 Convert the following data transfer rates to bits per second
(a) 1Mb/s (b) 100kb/s (c) 1Gb/s
Figure 4.1.1 Table of unsigned binary codes in four bits and their
decimal equivalent values.
When coding numbers in unsigned binary, the weights of each binary position
in decimal are as shown in Figure 4.1.2. Notice that the next significant digit
weighting in decimal is obtained from the previous one by multiplying by 2.
Figure 4.1.2 shows that the number with decimal representation 12 has an
unsigned binary representation of 00001100 in 8 bits.
Decimal
Most significant weighting Least significant
binary digit binary digit
128 64 32 16 8 4 2 1
0 0 0 0 1 1 0 0
Questions
1 Convert the following decimal values to unsigned binary using 8
bits:
(a) 5 (b) 129 (c) 253
minimum = 002
maximum = 112
The minimum number is always zero but the maximum varies with the number
of bits used to represent the number. Figure 4.1.3 shows the maximum binary
numeral for 1, 2, 3, 4, 5, 6, 7 and 8 bits and the weighting in decimal for each
bit position.
128 64 32 16 8 4 2 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1
1 1 1 1 1
1 1 1 1
1 1 1
1 1
1
Figure 4.1.3 Maximum binary numeral for a given number of bits
Figure 4.1.4 shows the maximum number expressed as a decimal numeral for
1, 2, 3, 4, 5, 6, 7, 8 and n bits.
Minimum value = 0
Maximum value = 2n – 1
Questions
02 + 02 = 02
02 + 12 = 12 27 26 25 24 23 22 21 20
12 + 02 = 12 0 1 1 0 1 1 0 0
12 + 12 = 02 carry 12 0 0 1 0 1 0 1 0
02 + 02 + carry 12 = 12 1 Carry 1 0 Carry 1 0 1 Carry 1 0 1 1 0
02 + 12 + carry 12 = 02 carry 12
12 + 02 + carry 12 = 02 carry 12 Least
12 + 12 + carry 12 = 12 carry 12
significant digit
This column uses the rule
column
02 + 02 + carry 12 = 12
This column uses the rule
12 + 12 = 02 carry 12
02 + 02 = 02
02 + 12 = 12
12 + 02 = 12
If we have a carry from the previous column then the carry must be added to
the sum of the two digits of the current column. So we have the additional
rules
02 + 02 + carry 12 = 12
02 + 12 + carry 12 = 02 carry 12
12 + 02 + carry 12 = 02 carry 12
12 + 12 + carry 12 = 12 carry 12
Questions
multiplier x 4 3
1 9 6 0 8 456 x 40 = 18240
product
multiplier x 1 1
Notice that because each digit of the multiplier 112 in the above example is a 1,
the multiplicand 1012 is just copied and then shifted either zero or one places
to the left to produce the corresponding partial product, 1012 or 10102. The
number of shifts to perform is the same as the exponent of the weighting, i.e. 0
or 1.
Extending this to a multiplication of larger numbers we see that binary
multiplication consists of copying and shifting the multiplicand,
e.g. 110111012 x 10112 = 1001011111112 as shown in Figure 4.2.2.
Key principle
Multiplication of two unsigned
211 210 29 28 27 26 25 24 23 22 21 20
binary integers:
For each 1 in the multiplier, copy
1 1 0 1 1 1 0 1 22110
the multiplicand and place below
the last partial product but shifted
x 1 0 1 1 1110
left by a number of columns equal
to the exponent of the weight for
1 1 0 1 1 1 0 1
this 1.
1 1 0 1 1 1 0 1 0 For each 0 in the multiplier,
change every digit in a copy of the
0 0 0 0 0 0 0 0 0 0 multiplicand to 0 and place the
copy as before.
1 1 0 1 1 1 0 1 0 0 0 Sum the partial products to obtain
the product.
1 0 0 1 0 1 1 1 1 1 1 1 243110 12 + 12 + carry 12 = 12 carry 12
Questions
Table 4.3.2 shows Bin using 4 bits to represent integers in binary. Again,
Bin Dec
negative numbers are represented in binary with the most significant digit 1
0000 0
and positive numbers with the most significant digit 0.
0001 1
0010 2 Whatever the number of bits:
0011 3
• 1 in the most significant bit position indicates a negative integer and a 0, a
0100 4
positive integer
0101 5
0110 6 • The most negative integer occurs with 1 in the most significant bit position
0111 7 and all 0s in the other positions
1000 -8
• For -1, every bit is 1 -8 4 2 1
1001 -7
1010 -6 • For the most positive value every bit is 1 0 0 0 0
1011 -5 except the most significant bit which is 0. 0 0 0 1
1100 -4 0 0 1 0
The sign bit in two’s complement is always the
1101 -3 0 0 1 1
most significant digit. 0 1 0 0
1110 -2
1111 -1 To achieve the range -810 to +710 the weighting for 0 1 0 1
each bit must be as shown in Table 4.3.3. Notice 0 1 1 0
Table 4.3.2 Two’s complement
that the most significant digit, the sign bit, also 0 1 1 1
representation of negative and
1 0 0 0
positive integers has magnitude or size, i.e. a weighting of -810.
1 0 0 1
Table 4.3.4 shows how the weighting of each bit 1 0 1 0
position varies for integers in two’s complement 1 0 1 1
Key fact binary for a given number of bits, e.g. for 5 bits 1 1 0 0
Two’s complement sign bit: the most significant bit has a weighting of -16 in 1 1 0 1
The most significant bit is the decimal. The most significant digit is always the 1 1 1 0
sign bit. 1 1 1 1
sign bit and its weighting is always negative.
Its weighting is always negative. Table 4.3.3 4-bit
The bit positions are labelled starting with the
two’s complement
least significant digit which is given bit position 0.
representation of integers
n – 1
The most significant bit has weighting, -2
where n is the number of bits,
e.g. n = 8, -28 – 1 = -27 = -12810.
No of
Weighting
bits
8 -128 64 32 16 8 4 2 1
7 -64 32 16 8 4 2 1
6 -32 16 8 4 2 1
5 -16 8 4 2 1
4 -8 4 2 1
3 -4 2 1
2 -2 1
7 6 5 4 3 2 1 0
Bit position
Table 4.3.4 Two’s complement representation of integers showing weighting
of bit positions for different numbers of bits
Questions
1 What is the weighting in decimal of the most significant bit if the
following number of bits are used to represent integers in two’s
complement binary
(a) 3 (b) 5 (c) 8 (d) 10 (e) 16?
3 What is the binary numeral for the most negative integer in two’s
complement binary when the number of bits for the numeral is as
follows
(a) 3 b) 5 (c) 8 (d) 10 (e) 16?
4 What is the binary numeral for the most positive integer in two’s
complement binary when the number of bits for the numeral is as
follows
(a) 3 (b) 5 (c) 8 (d) 10 (e) 16?
Change to
unsigned
decimal Convert Insert 0 Flip bits Add 1
-1310 → 1310 → 11012 → 011012 → 100102 → 100112
Questions
5 Convert the following integers expressed in decimal to 5-bit two’s
complement binary.
(a) +12 (b) -12 (c) +7 (d) -7 (e) -1
1 x -128 + 1 x 64 + 1 x 32 + 1 x 16 + 0 x 8 + 0 x 4 + 0 x 2 + 1 x 1 = -1510
Questions
7 Convert the following integers expressed in two’s complement
binary to decimal.
The addition carried out is just binary addition but this can generate a carry (1)
which is ignored because we restrict the answer to the same number of bits as
we started with, i.e. 4.
Check: 01012 – 00112 = 510 – 310 = +210 = 00102
Another example, 01012 – 11112 would be evaluated as follows
01012 – 11112 → 01012 + (-11112) → 01012 + 00012 → 01102
Questions
8 Evaluate the following 4-bit two’s complement binary integer
expressions using steps that involve only binary.
-27 26 25 24 23 22 21 20
-12810 6410 3210 1610 810 410 210 110
1 0 0 0 0 0 0 0
and the most positive integer is 011111112 whose bits are associated with the
following decimal weightings:
-27 26 25 24 23 22 21 20
-12810 6410 3210 1610 810 410 210 110
0 1 1 1 1 1 1 1
Questions
9 What is the range in decimal of integers represented in two’s
complement binary using
■■Binary to decimal floating The number 136 • 75 represents 1 hundred, 3 tens, 6 units, 7 tenths and 5
point hundredths.
Figure 5.4.4.1 shows how unsigned numbers with a fractional part can be
represented in binary using 8 bits. The weighting of each bit has been selected
to allow three bits for the fractional part but we could have chosen a different
number of bits for the fractional part, if we had wanted to. Notice that the
weighting decreases by a factor of 2 between adjacent columns as shown.
÷2 ÷2 ÷2 ÷2 ÷2 ÷2 ÷2
Weighting 16 8 4 2 1 ½ ¼ ⅛
16 8 4 2 1 0.5 0.25 0.125
1 0 1 1 0 1 1 1
This coding is known as fixed point coding because the binary point is fixed in
position, in this example between the third and fourth bits from the right.
Questions
1 Given 8 bits with the binary point fixed in position between the
third and fourth bits from the right as in Figure 5.4.4.1, what is the
decimal representation for each of the following unsigned binary
numbers?
(a) 00011•100 (b) 00101•110 (c) 10000•101
(d) 11111•111
Questions
2 Given 8 bits with the binary point fixed in position between
the fourth and fifth bits from the right what is the decimal
representation for each of the following unsigned binary numbers?
-16 8 4 2 1 ½ ¼ ⅛
1 0 1 1 0 1 1 1
Questions
4 Given 8 bits with the binary point fixed in position between the
third and fourth bits from the right as in Figure 5.4.4.2, what is
the decimal representation for each of the following signed binary
numbers?
where A is any real number greater than -10 and less than +10 and B is any
Using the letter E instead of 10,
integer, e.g. A = 1.356, B = 1. and omitting the multiplication
For example, sign, the real no above can be
rewritten as follows
6•58723 x 104 = 65872•3 where A = 6•58723 and B = 4, the
13.56 = 13.56E0
number of decimal places to shift the point right
= 1.356E1
= 0.1356E2
6 • 5 8 7 2 3 = 65872•3 = 135.6E-1
= 1356E−2, etc
-8•0000 x 10 = -8000•0 where A = -8•0000 and B = 3
3
These numerals are sometimes
6•0 x 10 = 0•0006 where A = 6•0 and B = -4, the number of
-4 called floating point numerals as
opposed to the decimal numerals
decimal places to shift the point right, i.e. shift right -4 places
such as 13.56 that are called fixed
which translates to a shift left of 4 places. point numerals.
A similar notation is used when two’s complement binary is used to represent
Floating point numeral:
signed numbers that range from small to large
Consists of an integer numeral,
M x BaseE e.g. 1, followed by a fraction
numeral, followed by an exponent
where M is called the mantissa or significand, E the exponent and the number
part.
base Base equals 2 in decimal.
The mantissa is any real number greater than or equal to -110 and less than +110. Fraction numeral:
A fraction part is a decimal point
The exponent expresses the number of binary places to shift the point right
followed by a positive integer
or left. numeral, eg .356
For example,
Exponent part:
M = 0•10000002 and E = 00102 = 210 An exponent part is the letter E
followed by an integer numeral,
e.g. E1.
0 • 1 0 0 0 0 0 02 = 10•0000002
The true value represented is Figure 5.4.4.3 shows mantissa and exponent fields of a floating point
(-1)S x f x 2e number, 0•110 x Base0101 where Base equals 2 in decimal
S = sign bit
(-1)S → (-1) 0 = +1
The most significant bit of the mantissa is the sign bit. Its weighting is always
and (-1)1 = -1 -1. Therefore, the binary point is situated between the most significant bit and
where the next most significant bit of the mantissa.
e= E – bias
f = F/2n + 1
Likewise, the most significant bit of the exponent is a sign bit. The exponent
For single precision numbers is always integral, i.e. a whole number, either negative or positive or zero.
n=23, bias=127.
Example 1: To evaluate 0•110 x Base0101 in decimal, where Base equals 2 in
decimal, first calculate exponent in decimal
Exponent = 01012 = 4 + 1 = +510
Then move the binary point of the mantissa +510 places to the right and then
convert the mantissa to decimal (plus sign means move binary point right)
0•1102 → 011000•02 = +2410
Exponent = 10002 = -810 (minus sign means move binary point left)
Move binary point of the mantissa 810 places to the left and convert mantissa
to decimal
0•1102 → 0•00000000112 = ¹⁄₅₁₂ + ¹⁄₁₀₂₄ = + 0.002929687510
Questions
5 Given 8 bits to represent a signed number in two’s complement floating point form, with 4 bits for the
mantissa and 4 bits for the exponent as shown in Figure 5.4.4.3, write down separately, for each of (a) to
(f ), the binary forms of the mantissa and the exponent then the decimal expansion
(a) 01000100 (b) 10100100 (c) 01001111 (d) 01110011 (e) 10000000 (f ) 11111100
2. Convert the whole number part, W, from decimal to binary using, for 2. Convert the whole number
example, the repeated division by two algorithm. part, W, from decimal to
binary using the repeated
3. Convert the fractional part, F, from decimal to binary in a given number
division by two algorithm.
of bits using the following algorithm.
Repeated multiplication by two algorithm: 3. Convert the fractional part,
F, from decimal to binary in
n ← 0 a given number of bits using
OrigF ← Fractional part F the repeated multiplication by
two algorithm.
Repeat
R ← F x 2
n ← n + 1
F ← R - D
Until F = 0 Or F = OrigF
Or n = AllocatedFractionalNoOfBits
Table 5.4.4.1 shows how the fractional part 0.7510 is converted to 0.112 by this
algorithm. The algorithm terminates for 0.7510 on the condition F = 0. It
will always terminate on F = 0 when the denominator of the fractional part
involves the prime factor 2 only, if a sufficient number of bits are allocated to
the fractional part.
Fractional
R Digit D
part, F
0.75 0.75 x 2 = 1.5 1
0.5 0.5 x 2 = 1.0 1 Most
0 significant
Fractional
R Digit D
part, F
0.8 0.8 x 2 = 1.6 1
0.6 0.6 x 2 = 1.2 1 Most
0.2 0.2 x 2 = 0.4 0 significant
0.4 0.4 x 2 = 0.8 0 bit
(Previous 4 steps now
0.8
repeat)
Fractional
R Digit D
part, F
0.1 0.1 x 2 = 0.2 0
0.6 0.2 x 2 = 0.4 0 Most
0.2 0.4 x 2 = 0.8 0 significant
0.4 0.8 x 2 = 1.6 1 bit
0.6 0.6 x 2 = 1.2 1
0.2 0.2 x 2 = 0.4 0
0.4 0.8 x 2 = 1.6 1
0.6 0.6 x 2 = 1.2 1
(Previous 3 steps now
0.2
repeat)
Repeating bit patterns occur whenever the denominator of the fractional part
involves prime factors other than 2, e.g. 0.8 = ⁸⁄₁₀ = ⁴⁄₅ so the denominator
has a prime factor (5) other than 2.
Questions
6 Convert the following decimal numbers to fixed point binary
using the repeated multiplication by two algorithm.
Questions
7 Convert the following decimal numbers to fixed point binary
using the repeated division by two algorithm for the whole number
part and the repeated multiplication by two algorithm for the
fractional part.
Programming tasks
1 Write a program that implements the repeated multiplication by
two algorithm. Test your program for cases (a) to (e) in Question 6.
→ 11100.0012 + 12 → 11100.0102
Questions
8 Convert the following decimal numbers to 8-bit fixed point
binary in which three bits are allocated for the fractional part.
For example, 3.7510 is in two’s complement 8-bit floating point binary with 3
bits for the exponent and 5 bits for the mantissa
0.11112 0102
as shown in Table 5.4.4.4, which starts from the fixed point form 011.11002.
Number 0 1 1 • 1 1 0 0 0 0 0 3.7510
Most
-2 1 • ¹⁄₂ ¹⁄₄ ¹⁄₈ ¹⁄₁₆ Increment exponent
significant
0 1 • 1 1 1 0 0 0 1 (1 + ¹⁄₂ + ¹⁄₄ + ¹⁄₈ ) x 21 = 3.7510
bit
Move point -1 • ¹⁄₂ ¹⁄₄ ¹⁄₈ ¹⁄₁₆ Increment exponent
Questions
9 Convert the following decimal numbers to floating point binary
storing the mantissa in 5 bits in two’s complement form and the
exponent in 3 bits, also in two’s complement form. The binary
point should be between the most significant bit and the next most
significant bit of the mantissa.
For example,
M = 0•1102 (two’s complement)
E = 01012 (two’s complement)
-1 ½ ¼ ⅛
Md = + ( ¹⁄₂ + ¹⁄₄) = ³⁄₄ = 0.7510
0 1 1 0
-8 4 2 1
Ed = 01012 = +510 0 1 0 1
Questions
10 Using method 1 convert the following floating point binary
numbers which store the mantissa in 4 bits in two’s complement
form and the exponent in 4 bits, two’s complement form, into
decimal. The binary point is between the most significant bit and
the next most significant bit of the mantissa.
(a) 0•101 0111 (b) 1•000 0110 (c) 0•100 1000
(d) 0•111 1011 (e) 1•001 1111
Method 2:
Alternatively, convert the exponent to decimal and move the binary point of
the mantissa right if exponent positive, left otherwise.
For example,
M = 0•1102 (two’s complement)
E = 01012 (two’s complement)
Ed = 01012 = +510
Shift binary point 510 places to the right, but first add trailing zeroes
0•1102 → 0•1100002 -32 16 8 4 2 1 ¹⁄₂
→ 011000•02 0 1 1 0 0 0 0
Questions
11 Using method 2, convert the following floating point binary
numbers which store the mantissa in 4 bits in two’s complement
form and the exponent in 4 bits, two’s complement form, into
decimal. The binary point is between the most significant bit and
the next most significant bit of the mantissa.
(a) 0•101 0111 (b) 1•000 0110 (c) 0•100 1000
(d) 0•111 1011 (e) 1•001 1111
Questions
1 Write down the following decimal numerals as sums of the
decimal units 1000, 100, 10, ….., shown above
Questions
2 Write down the following binary numerals as sums of the
decimal units 32, 16, 8, 4, …., shown above
(a) 1100.11 (b) 101.0101 (c) 11.1011
Notice in binary the only possible prime factor for the denominator is 2.
(2 x 2)
However, the denominator of a decimal such as 0.8 = ⁄ doesn’t consist
5
of multiples of 2. There are in fact many decimals that cannot be broken down
x
into the form ⁄ n .
2
Questions
4 Write down the factors of the following fixed point decimal
Key fact numbers in Factors⁄Factors format, for some you may want to use a
factoring calculator
In binary, the only possible prime
factor for the denominator is 2. (a) 0.1 (b) 0.7 (c) 5.7 (d) 8.75 (e) 67.03125
x
We break a decimal into ⁄ n
2
when we convert it exactly to its
5 For each of the fixed point decimal numbers in Q4, state whether
binary equivalent. x
There are in fact many decimals
it can be represented in the form ⁄ n.
2
that cannot be broken down into
x 6 Using a decimal to binary converter, write down the fixed point
the form ⁄ n .
2
binary equivalent of the decimal numerals in Q4. Comment on
your answers.
Information
Fixed point Fixed point Table 5.4.5.1 shows the fixed point binary
Decimal to binary converter
and binary to decimal decimal binary equivalent of some unsigned fixed point
converter: 0.1 0.00011 decimals.
http://www.exploringbinary.com/ 0.2 0. 0011 Notice that some binary equivalents
binary-converter/
0.25 0.01 contain a repeating sequence of 1s and 0s
0.3 0.01001 indicated with a line.
0.4 0.0110 Table 5.4.5.2 shows the same decimals
0.5 0.1 expressed in rational form x/y with the
0.6 0.1001 numerator and denominator factored. It
0.7 0.10110 also shows the closest binary fixed point
0.75 0.11 numeral in 8 bits with binary point
0.8 0.1100 between the most significant digit and the
0.9 0.11100 next.
1.0 1.0
Table 5.4.5.1 Fixed point decimals and their fixed point binary equivalent
Unsigned x Closest
Closest ⁄ n Closest
Fixed 2 8-bit binary
x/y form for 8-bit fixed point
point numeral in
0≤n≤7 binary form
decimal decimal form
0.1 1⁄ 13⁄ 0.0001101 0.1015625
(2 x 5 ) 27
Questions
7 For each of the following values of x, write down the nearest
x
8-bit fixed point binary numeral to ⁄128 using Figure 5.4.5.1
weighting
When these fixed point binary numerals with recurring bit patterns are limited
to a given numbers of bits, e.g. 9, we can just drop the bits in bit positions
greater than this given number. This is known as truncation or rounding down.
For example, 0.110 truncated to 9 bits becomes 0.000110012 just by dropping
10011 from the 10th bit position onwards.
Rounding off
Alternatively, we can choose to round off. This means
• Add 1 to the last retained digit if the following digit is 1 otherwise leave
unaltered.
For example, if we round off 0.00011 to 8 bits then we need to look at the
binary expansion as far as the 9th bit which is 0.00011001.
The 9th bit is 1 so we drop the 9th bit but add 1 to the 8th bit so arriving at
0.00011012, the rounded off result in 8 bits.
On the other hand, if we round off 0.01001 to 8 bits then we don’t add 1
because the binary expansion to 9 bits is 0.010011002 and the 9th bit is 0.
Therefore, 0.01001 rounded off to 8 bits is 0.01001102.
Rounding off is usually used where the representation is inexact, because less
error can result compared with rounding down.
Questions
10 For each decimal in Table 5.4.5.4, write down the nearest
unsigned fixed point binary numeral in 10 bits, rounding off. Also
write down the rounding error if any.
01
10
00
01
00
11
10
11
11
00
00
11
11
00
01
00
00
00
00
01
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0001100110011....
Figure 5.4.6.1 Binary number line showing 8-bit unsigned
fixed point binary
For example, if only 8 bits are available as shown in Figure 5.4.6.1 then
0.110 which in unsigned fixed point binary is 0.00011 will be represented by
0.00011002 if truncated and 0.00011012 if rounded off because it lies between
these two values as shown.
Absolute error
Key principle
The difference between the actual number and the nearest representable value is
Absolute error: known as the absolute error. For example, 0.110 is stored as 0.00011012 in 8-bit
The difference between the unsigned fixed point binary form which is 0.101562510. Therefore,
actual number and the nearest
representable value. Absolute error = 0.101562510 - 0.110 = 0.001562510
Absolute means that the sign is ignored e.g. differences of 0.0015625 and
-0.0015625 are the same absolute error.
Questions
Questions
2 Calculate the percentage relative error when the following fixed point decimal numbers are stored in 8-bit
unsigned fixed point binary as shown in Figure 5.4.6.1. Round off if a number cannot be represented
exactly.
Questions
3 The percentage relative error is 1% for the following decimal numbers. Calculate the absolute error.
• speed of calculation In a simplified example of fixed point, we allocate 8 bits and use these as shown
in Figure 5.4.7.1.
128 128
values values
00
00
00
11
00
00
00
00
00
00
00
11
00
000
00
00
00
00
10
10
11
00
00
10
00
10
0.0
0.0
0.1
0.1
0.1
1.0
1.0
1.1
1.1
If we think of the number line shown in Figure 5.4.7.1 as a ruler then the
precision with which we can record measurements with this ruler is to the
nearest ¹⁄₁₂₈ .
Thus, for any positive or negative number inside the range that can be
represented, the maximum absolute error in a measurement in this coding
scheme will be ¹⁄₂₅₆ (one half of ¹⁄₁₂₈ because we round off). The
maximum percentage relative error will be ¹⁄₂₅₆ ⁄ x 100 = 0.39%
¹²⁷⁄₁₂₈
and the largest ¹⁄₂₅₆⁄¹⁄ x 100 = 50%.
₁₂₈
Questions
1 With the binary point placed between the sign bit and
the next bit what is the smallest positive number that can be
represented in two’s complement fixed point binary for the
following number of bits ?
f x 2e
We are still restricted with 8 bits to choosing from 28 or 256 different bit
patterns but we achieve a much greater range of representable numbers than
with fixed point coding. If we use the weightings shown in Figure 5.4.7.3 then
the range of representable numbers is
⁻⁸⁄₈ x 27 to ⁺⁷⁄₈ x 27 (i.e. -128 to 127)
or
⁻¹²⁸⁄₁₂₈ x 27 to ⁺¹¹²⁄₁₂₈ x 27
This range is larger than the 8-bit fixed point coding by a factor 27.
Mantissa Exponent
⁻⁸⁄₈ ⁴⁄₈ ²⁄₈ ¹⁄₈ -8 4 2 1
0 0 0 0 0 0 0 0
Figure 5.4.7.3 Weightings for 8-bit floating point, two’s complement 4-bit
mantissa and 4-bit exponent
Figure 5.4.7.4 shows the range of expressible negative and positive numbers
for a normalised 4-bit mantissa and a 4-bit exponent weighted as described
above in Figure 5.4.7.3. The negative value closest to zero is 1011 10002 in
4-bit mantissa 4-bit exponent form or -0.625 x 2-8 in decimal. The mantissa has
5 0 1
-128 - + +112
2048 512
Figure 5.4.7.4 The range of expressible negative and positive numbers for a normalised
4-bit mantissa and a 4-bit exponent weighted as shown in Figure 5.4.7.3
We consider zero as a special case. For zero, f is 0 and exponent e is also 0. This
means that for our 8-bit example we will use only 128 bit patterns excluding
zero, but 129 bit patterns if zero is included. For zero to be excluded then the
bit before the binary point must always be different from the bit after this
point.
The 129 different representations are distributed unevenly across the range as
illustrated in a section of the number line in Figure 5.4.7.5.
0.100 x 21
0.100 x 22
0.100 x 23
0.101 x 20
0.110 x 20
0.111 x 20
0.101 x 21
0.110 x 21
0.111 x 21
0.101 x 22
0.110 x 22
0.111 x 22
relative error is ⁸⁄₂₇ x 100 = ⁸⁄₁₂₈ x 100 = 6.25% for the largest positive
number.
For the smallest positive numbers, the difference between the representations
is as little as ¹⁄₈ x 2-8 or ¹⁄₂₀₄₈, a small absolute error of half this. The
approximate percentage relative error is ¹⁄₄₀₉₆ ⁄ -8 x 100 = 6.25% for the
2
smallest positive number. The relative error is similar across the whole range.
Questions
2 Evaluate the following where the mantissa is in two’s complement form
(a) 0.1112 x 22 (b) 0.1012 x 26 (c) 1.0112 x 22 (d) 0.1012 x 2-6
3 Explain why the gaps between adjacent numbers expressed in floating point form are not constant.
RepresentationFor example, Table 5.4.7.1 shows how fixed point Rangerepresentation compares
The fixed point range in decimal forwith floating
n bits wherepoint
n for 32 bits.
= 32, and using two’s complement representation
+8.4 x 106 to +3.9 x 10-3, 0, -3.9 x 10-3 to -8.4 x 106
with 8 of the 32 bits allocated to the fractional part is
approximately
The floating point range in decimal, for the same
number of bits 32, for an 8-bit two’s complement
+ 1.7 x 1038 to + 1.5 x 10-39, 0, -1.5 x 10-39 to -1.7 x 1038
exponent and 24-bit normalised mantissa, is
approximately
Table 5.4.7.1 Comparison of ranges of fixed point and floating point representations
where the number of bits is 32 in each case
Questions
4 What are the most positive and most negative numbers that can be represented using the following fixed
point two’s complement representations? Give your answers in decimal.
5 What are the most positive and most negative numbers that can be represented using the following
floating point two’s complement representations? Give your answers in decimal.
(a) 16 bits of which 8 bits are allocated to mantissa and 8 bits to the exponent
(b) 32 bits of which 16 bits are allocated to mantissa and 16 bits to the exponent
(c) 64 bits of which 32 bits are allocated to mantissa and 32 bits to the exponent
The first 11.1 cm is also 111 mm. This indicates that this measurement has been made to the nearest millimetre. The
second, 120 cm is ambiguous. It is not clear whether the measurement is exactly 120 cm or the measurer was just
measuring to the nearest 10 cm.
The third, 12.23 km in metres is 12230 m. The measurement was performed to the nearest 10 m, i.e. the real value
lies somewhere between 12225 and 12235. The last measurement 12.2 km in metres is 12200 m, a measurement
performed to the nearest 100 m, i.e. the real value lies somewhere between 12150 and 12250.
The latter two measurements, 12.23 km and 12.2 km, clearly show a different degree of precision, one is a
measurement to the nearest 10 m and the other to the nearest 100 m. We call the digits which provide information
about the precision of a measurement, significant digits or significant figures and the more significant digits
used the greater the precision of the measurement.
Table 5.4.7.2 shows the number of significant digits for some measurements.
Questions
6 State the number of significant figures for each of the following
measurements made in the decimal system
(a) 12.23 (b) 130.04 (c) 0.03 (d) 0.00450 (e) 34 (f ) 0254.
1111
The number of significant digits is unambiguous, it is 3, i.e. 101. This
representation is known as scientific form. Floating point representation
Figure 5.4.7.6 Section of a resembles scientific form in structure but differs slightly on detail. The
binary ruler significant digits are contained in the mantissa. The exponent records how
many places to shift the binary point, left or right to obtain the fixed point
form of the number.
Questions
7 State the number of significant figures, if you can, for each of the following measurements made in fixed
point binary. If you cannot, explain why.
(a) 10.12 (b) 0.1112 (c) 0.0112 (d) 0.1012 (e) 0.11001002 (f ) 10102
if the same total number of bits are used for each form. floating point form will be stored
with less precision than fixed
However, floating point, for a given number of bits, can store numbers point representation which uses
chosen from a greater range than fixed point, but only at the expense of the same total number of bits.
precision.
For example, Figure 5.4.7.7 shows a floating point number representation in
32 bits using a 24-bit mantissa and an 8-bit exponent. Key fact
The mantissa has storage space for 23 significant digits + 1 sign bit. Range of floating point vs
fixed point:
However, the same 32 bits could store 31 significant digits + 1 sign bit in fixed
For a given number of bits,
point form. floating point represents a much
Questions
8 How many significant digits can two’s complement floating
point store if the representation is as follows
Questions
9 How many significant digits can two’s complement fixed point store
in the following number of bits
(a) 16 bits?
(b) 24 bits?
(c) 64 bits?
Speed of calculation
In general, calculations take longer with numbers stored in a digital computer
Key fact in floating point form than they do with numbers stored in fixed point form
Speed of calculation of
because floating point inevitably involves floating or shifting the decimal or
floating point vs fixed point: binary point of operands whereas fixed point doesn’t.
In general, calculations take longer
If the central processing unit (CPU) does not have circuitry to perform floating
with numbers stored in floating
point form than they do with
point operations directly then the steps of the operations have to be written in
numbers stored in fixed point software and accessed by the CPU. Fetching and executing code to perform
form. calculations is considerably slower than doing these calculations directly in
hardware designed just for this task.
Although modern general purpose computers contain CPUs that have access to
hardware floating point units, even with a hardware floating point unit floating
point calculations still take longer than fixed point calculations.
0.323142 x 103
0. 00323142 x 105
Questions
1 Write down the floating point form of 323.142 for which the
power of ten multiplier is
Having more than one representation is not a good idea. For example, adding
323.142 to itself using two different floating point representations is not
straight forward. Try this for yourself
32314.2 x 10-2
+ 0.323142 x 103
-----------------
-----------------
Neither is comparing two numbers straightforward if they use different floating
pointing representations, e.g. 323.142 x 100 and 0.00323142 x 105.
It makes good sense therefore to allow just one floating point representation
of a number to ensure the uniqueness of the representation and to reduce the
effort required to perform arithmetic operations.
Normalisation in decimal
Computer memory is finite and therefore it is necessary to allocate a fixed
number of bits to each representation of a number. Just for the moment,
imagine that the computer is able to store the decimal digits, sign and decimal
point as shown in Figure 5.4.8.1 with the position of the mantissa’s decimal
point fixed to ensure uniqueness of representation. This memory is shown to
store up to six significant decimal digits in the mantissa.
Mantissa Exponent
+ 3 2 3 1 4 2 + 2
Figure 5.4.8.1 Mantissa-exponent store for decimal digits, sign and decimal
point in fixed position as shown to ensure uniqueness of representation
Now imagine that after a floating point number calculation the result obtained
is
+ 0.00516838 x 106
We now face a problem. Although the mantissa of the answer has six significant
digits, and our fixed point memory arrangement can accommodate six
significant digits, if we do not adjust the exponent of the result we will lose
precision as shown in Figure 5.4.8.2 where rounding off has been applied
Mantissa Exponent
+ 0 0 0 5 1 7 + 6
Questions
2 Using the form of representation shown below, in which six
significant digits are allowed in the mantissa and one in the
exponent, normalise the following floating point decimal numbers
Mantissa Exponent
sign d d d d d d sign d
0.187510 is 0.00112 in binary fixed point form. In un-normalised floating point This is achieved by ensuring
that bit to the immediate right
form, this is
of point is a significant digit,
0.00112 00002 1 for positive mantissas, 0 for
negative mantissas
where the mantissa is 0.00112 and the exponent is 00002.
2. Ensures that the representation
What if only 4 bits instead of 5 bits were available for the mantissa? Do we just is unique.
round off to 0.0102? No, we normalise the 5 bits first and then round off, if we
have to.
Figure 5.4.8.4 shows in stages how this is done, for a 4-bit mantissa and a Key fact
4-bit exponent for the result, by shifting the bits of the mantissa left whilst
Increment:
decrementing the exponent. Shifting/incrementing stops when the bit before
Means increase by 1, or more if
the binary point is different from the bit after the point. The mantissa bit
size of increment specified.
shown in red is unavailable to use for the result. The normalised representation
Decrement:
is
Means decrease by 1, or more if
0.112 11102 size of decrement specified.
0 • 0 0 1 1 0 0 0 0 ³⁄₁₆ x 20 = 0.187510
0 • 0 1 1 1 1 1 1 ³⁄₈ x 2-1 = 0.187510
0 • 1 1 1 1 1 0 ³⁄₄ x 2-2 = 0.187510
Figure 5.4.8.4 Normalising a positive number expressed in un-normalised floating point binary form
Key principle
The normalisation process essentially identifies where the significant digits of
the representation begin and then adjusts the mantissa so that digit positions Normalisation process:
Move significant digits so
are not wasted by being taken up by non-significant digits, For example, in the
that bit before binary point in
two’s complement floating point representation 0.00112 00002 , the mantissa mantissa is different from bit
is 0.00112 and has significant digits that begin at the first 12 after the point, after point, adjust exponent
i.e. 112. The 02 before the binary point is the sign bit so needs to be retained. accordingly.
The 002 between the point and 112 is not significant. So moving the point two
places to the right results in a mantissa of 0.112. To compensate we need to take
2 away from the exponent, 00002 → 11102.
Questions
Did you know?
3 Using the form of representation shown below for your answer in
Arithmetic Shift Left: which six bits are assigned to the mantissa and four to the exponent,
This operation is supported by normalise the following un-normalised floating point binary
most CPUs as a built-in machine
numbers whose mantissa and exponent are shown below in
operation belonging to the
instruction set of the processor. (a) to (c)
In assembly language it usually is
Mantissa Exponent
given the mnemonic ASL. It shifts
the bits left one position whilst -1 ¹⁄₂ ¹⁄₄ ¹⁄₈ ¹⁄₁₆ ¹⁄₃₂ -8 4 2 1
preserving the value of the sign d d d d d d d d d d
bit.
(a) 0.0000111012 00112 (b) 0.00101012 11002
(c) 0.000000111012 01112
Algorithm:
While bit before point = bit after point
Do
Remove bit after point
then shift all remaining mantissa bits left one place and
insert a zero in least significant bit position
Decrement exponent
EndWhile
Figure 5.4.8.5 shows this algorithm applied to 1.11012 00002. Four bits
are allocated to the mantissa and four to the exponent for the result. The
fifth mantissa bit shown in red is not available to use in the result. For each
iteration of the algorithm a 1 in the ¹⁄₂ column is removed and then the lesser
significant bits are shifted left one bit position and a zero inserted in the least
significant bit position. The normalised form shows that we have avoided
rounding off and thereby a loss of precision.
1 • 1 1 0 1 0 0 0 0 -³⁄₁₆ x 20 = -0.187510
1 • 1 0 1 0 1 1 1 1 -³⁄₈ x 2-1 = -0.187510
1 • 0 1 0 0 1 1 1 0 -³⁄₄ x 2-2 = -0.187510
Figure 5.4.8.5 Normalising a negative number expressed in un-normalised floating point binary form
Also note that the algorithm for normalizing a negative mantissa is exactly the
same as for normalizing a positive mantissa.
The normalisation process for a negative mantissa essentially identifies where
the significant digits of the representation begin and then adjusts the mantissa
so that digit positions are not wasted by being taken up by non-significant
digits. For example, in the two’s complement floating point representation
1.11012 00002, the mantissa is 1.11012 and has significant digits that begin
at the first 02 after the point, i.e. 012 (this is the opposite of the case when the
mantissa is positive). The 12 before the binary point is the sign bit so needs to
be retained. The 112 between the point and 012 is not significant. So moving
the point two places to the right results in a mantissa of 1.012. To compensate
we need to take 2 away from the exponent, 00002 → 11102. Notice that when
the mantissa is negative we look for the occurrence of the first 02 after the
binary point as this is where the significant digits begin.
By a similar argument, if the mantissa had been 111.11012 , the first two 1’s are
insignificant, they are equivalent to leading zeroes in a positive mantissa, e.g.
000.10102, and therefore can be dropped producing 1.11012 which then needs
to undergo normalisation.
Questions
4 Write down the result of eliminating unnecessary digits, if
possible, from the following two’s complement representations
Therefore, the least positive and the least negative decimal numbers that can be
represented in two’s complement normalised binary floating point form are as
shown in Table 5.4.9.3 where n is the number of bits assigned to the exponent.
n = no of
Magnitude Example n = 3
exponent bits
(n – 1)
½ x 2-2
(n – 1) (3 – 1)
Least positive number ½ x 2-2 = 0.5 x 2-2
2
= 0.5 x 2-2 = 0.5 x 2-4
< -½ x 2-2(n – 1)
less negative than (3 – 1)
Least negative number (n – 1) = -0.5 x 2-2
-½ x 2-2
2
< -0.5 x 2-2 = -0.5 x 2-4
Table 5.4.9.3 Least positive and negative numbers
This means that there is a range of numbers either side of zero for which
a floating representation does not exist as shown in Figure 5.4.9.1. Zero
can be treated as a special case by assigning it with its own un-normalised
representation, e.g. an all zeroes mantissa and exponent or we have to use the
least positive or least negative number representations for zero. For the specifics
of how it is done in current computer systems look up the IEEE standard.
Negative Positive
Underflow Underflow
Expressible Expressible Positive
Negative
Negative Positive Overflow
Overflow
Numbers Zero Numbers
(n -1) -1 (n -1) (n -1) (n -1) -1
- 1 x 2+2 approx. -0.5 x 2 -2 +0.5 x 2 -2 approx. +1 x 2+2
0
Figure 5.4.9.1 Number range for floating point two’s complement binary where n is the
number of exponent bits
If a calculation produces a final result that lies in this range then it cannot be
represented and underflow has said to have occurred. Intermediate results
of the calculation may encroach into this range but after normalisation the
final result is restricted to the range of representable numbers for the specified
number of storage bits.
Simplified example
Figure 5.4.9.2 shows the least positive and the least negative normalised
Key fact
floating point binary representations in 7 bits for a 4-bit mantissa and a 3-bit
Underflow can occur when exponent, 01002 1002 and 10112 1002, respectively, or in decimal +½ x 2-4 and
dividing a small number by a very -⅝ x 2-4.
large number or when subtracting
two numbers of the same sign
Mantissa Exponent
and close in magnitude. -1 ½ ¼ ⅛ -4 2 1
Least positive normalised 0 1 0 0 1 0 0
Least negative normalised 1 0 1 1 1 0 0
Figure 5.4.9.2 7-bit two’s complement floating point binary least positive
and least negative number normalised representations
If we represent in two’s complement floating point form, the fraction ¹⁄₆₄
which is in fixed point two’s complement representation 0.000001002, we
obtain 0.1002 10112 or in decimal ½ x 2-5. This requires a 4-bit exponent.
Therefore, ¹⁄₆₄ cannot be represented in the 4-bit mantissa, 3-bit exponent
form shown in Figure 5.4.9.2. This is an example of underflow.
Circumstances when underflow can occur
Underflow can occur when dividing a small number by a very large number
or when subtracting two numbers of the same sign which are close in
magnitude.
Questions
1 Using the form of representation shown below in which six
bits are assigned to the mantissa and four to the exponent, state
whether underflow occurs when attempting to represent the
following numbers expressed in fixed point two’s complement
binary.
Mantissa Exponent
-1 ¹⁄₂ ¹⁄₄ ¹⁄₈ ¹⁄₁₆ ¹⁄₃₂ -8 4 2 1
d d d d d d d d d d
Overflow
Overflow occurs when a number is too large in magnitude to be represented.
Key principle
Questions
3 State whether performing the following arithmetic with integers represented in 8-bit two’s complement
form will result in overflow.
n = no of
Magnitude Example n = 3
exponent bits
(3 – 1)
<+1 x 2(2 - 1)
less than 2
Most positive number = +1 x 2(2 - 1)
+1 x 2 (2(n – 1) - 1)
= +24-1 = +23 = 8
(3 – 1)
-1 x 2(2 - 1)
more negative than 2
Most negative number = -1 x 2(2 - 1)
-1 x 2 (2(n - 1) - 1)
= -24-1 = -23 = -8
Table 5.4.9.4 Two’s complement binary floating point representation of
most positive and negative decimal numbers
If a calculation produces a final answer which is more positive than the most
Key fact
positive number then positive overflow has occurred.
Circumstances when
Conversely, if a calculation produces a final answer which is more negative
overflow can occur:
Overflow can occur when adding than the most negative number that can be represented then negative
two very large numbers of the overflow has occurred.
same sign or in multiplication
Circumstances when overflow can occur
involving large numbers or when
dividing a number by a very small Overflow can occur when adding two very large numbers of the same sign or
number. in multiplication involving large numbers or when dividing a number by a
very small number.
Questions
4 Using the form of representation shown below in which six bits are assigned to the mantissa and four
to the exponent, state whether overflow occurs when attempting to represent the following numbers
expressed in signed fixed point binary.
Mantissa Exponent
-1 ¹⁄₂ ¹⁄₄ ¹⁄₈ ¹⁄₁₆ ¹⁄₃₂ -8 4 2 1
d d d d d d d d d d
Questions
5 The most negative number that can be represented in a particular computer system is -2127.
The following expression is to be evaluated by this computer system
-2127 x 220∕230
In which order should the evaluation be arranged and why?
6 Discuss whether it is preferable to use the right hand side of the identity shown below, or the left, when
writing a program to evaluate x2 - y2. State the circumstances for your choice(s).
Both x and y represent real numbers.
x2 - y2 = (x + y) ( x – y)
ASCII was invented in the 1960s so that information could be exchanged over telephone wires between data
processing equipment. ASCII stands for American Standard Code for Information Interchange. Messages were
prepared on paper tape, similar Code
Character
Code
Character
Code
Character
Code
Character
in decimal in decimal in decimal in decimal
to that shown in Figure 5.5.3, by
32 Space 56 8 80 P 104 h
punching holes in the tape (a hole
33 ! 57 9 81 Q 105 i
= 1, an absence of a hole = 0) and 34 “ 58 : 82 R 106 j
then the tape was read by a sending 35 # 59 ; 83 S 107 k
machine connected to a telephone 36 $ 60 < 84 T 108 l
line. At the other end of the line 37 % 61 = 85 U 109 m
was another machine that would 38 & 62 > 86 V 110 n
interpret the received ASCII codes 39 ‘ 63 ? 87 W 111 o
40 ( 64 @ 88 X 112 p
and then print the corresponding
41 ) 65 A 89 Y 113 q
message in symbol form on paper
42 * 66 B 90 Z 114 r
for a human to read.
43 + 67 C 91 [ 115 s
44 , 68 D 92 \ 116 t
45 - 69 E 93 ] 117 u
46 . 70 F 94 ^ 118 v
47 / 71 G 95 _ 119 w
Figure 5.5.3 5-bit Punched paper
48 0 72 H 96 ` 120 x
tape large black dot = a hole = 1,
49 1 73 I 97 a 121 y
absence of a hole = 0 50 2 74 J 98 b 122 z
51 3 75 K 99 c 123 {
Table 5.5.2 shows a lookup table 52 4 76 L 100 d 124 |
for ASCII control codes, 0 to 31. 53 5 77 M 101 e 125 }
The codes with a blank character 54 6 78 N 102 f 126 ~
field are codes used for controlling 55 7 79 O 103 g 127 DEL
12 Form feed 28
13 Carriage return 29
14 30
15 31
Table 5.5.2 ASCII code lookup table for some control codes
97 Single licence - Abingdon School
5.5 Information coding systems
European languages which have accented letters, and special symbols such as £, When a key on a keyboard is
pressed a scan code is generated.
as it was designed for the North American market and it certainly doesn’t cater
Scan codes are binary codes
for Asian languages which are logogram-based (symbols represent concepts), as well. The scan code that is
not alphabetic. The 95 ASCII codes for characters found in text are wholely generated is converted into an
inadequate for a universal standard for information interchange. ASCII code that corresponds
to the current setting for the
Unicode was designed to provide a single character set that covers the languages
keyboard’s keys. The mapping
of the world. Unicode UTF-16 uses either one or two 16-bit code units for
between scan codes and ASCII
its character codes. A single 16-bit unit supports 216 or 65536 different codes. codes can be changed. For
Unicode UTF-32 uses 32-bit code units each representing a single character example, the mapping for a key
code. Unicode includes all the ASCII codes in addition to codes for characters marked in one currency symbol
can be changed so that when
in foreign languages (e.g. complete sets of Chinese characters), and many
pressed it maps to the ASCII code
mathematical and other symbols. for another currency symbol,
UTF-8 encodes each of the 1,112,064 valid code points in the Unicode code e.g. $ to £ (extended ASCII code
16310). This was the only way
space using one to four bytes. The first 128 characters of Unicode, which
to overcome ASCII’s limited
correspond one-to-one with ASCII, are encoded using a single byte with the character set until the adoption
same binary value as ASCII. of Unicode.
Figure 5.5.4 From decimal numeral to ASCII codes and back to decimal numeral
The ASCII codes 50, 54, 49 are called the character code form of the decimal
digits 261 e.g. 50 is the character code form of the decimal digit 2. To convert
this character code form 50 into the number 2 we need to subtract 48. The
character code form of the decimal number 2 in 7-bits is 0110010 whereas its
pure binary representation is 000 0010 in 7-bits.
Symbolically, the character code form 50, 54, 49 can be written as ‘2’ ‘6’ ‘1’. The
single apostrophes around each digit are used to differentiate the character form
from the decimal digit form.
Questions
7 What needs to be done to convert the following ASCII codes to their equivalent decimal digit
(a) 53 (b) 48 (c) 57?
8 What is the ASCII character code form of the following decimal digits and combination of decimal digits
(a) 6 (b) 34 (c) 908 (d) 444?
9 Why is it difficult to do arithmetic with the character form of a decimal numeral?
10 What would need to be done with the character form of a decimal numeral in order to do arithmetic in the
conventional way?
11 What is the ASCII character code form of the following characters and character strings
(a) '6' (b) '54'?
Key concept If error detection rather than error correction is sufficient then the parity bit
method can be used. The parity bit is computed from a group of n data bits
Parity bit: and then added to the group, making it n + 1 bits long. For example, a 7-bit
The parity bit is computed from
ASCII code becomes 8 bits long after a parity bit is added. The parity bit is
a group of n data bits and then
added to the group, making it computed by counting the number of
n + 1 bits long. ones in the n bit data group, and then
The parity bit is computed by setting the parity bit to make the count
counting the number of ones in
for the n + 1 group (parity + data) either
the n bit data group, and then
setting the parity bit to make
even or odd. The former is called even
the count for the n + 1 group parity and the latter odd parity.
(parity + data) either even or
For example, the count of 1s for the Figure 5.5.6 7-bit ASCII
odd. The former is called even
7-bit ASCII code 0101101 is 4. character code
parity and the latter odd parity.
A transmission or disk read is With even parity this becomes the 8-bit
judged reliable if the parity bit code 00101101 with the parity bit set to 0 to make the count of 1s across the
regenerated from the n data bits 8 bits an even number. With odd parity this becomes the 8-bit code 10101101
agrees with the received parity
with the parity bit set to 1 to make the count of 1s across the 8 bits an odd
bit.
Parity bit checking only works number. Now suppose that even parity is used and 00101101 is sent. If the
if an odd number of bits have pattern 01101101 is received then an error has occurred because the count of
been flipped. 1s is now odd.
The parity bit can be computed by applying the exclusive-OR (XOR) to the n
data bits because an XOR operation performs modulo-2 addition. Thus a series
of XOR operations can perform the counting.
Suppose the data is the 7-bit ASCII code, 0101101, and the XOR operation is
denoted by ⊕ then
0 ⊕ 1 ⊕0 ⊕1 ⊕1 ⊕0 ⊕1 = 1 ⊕1 ⊕1 ⊕1 = 0 ⊕0 = 0
Information
The XOR-computed parity bit for the 1011010 is 0 for even parity. Inverting
Exclusive-OR (XOR): the computed XOR-computed result gives 1 for odd parity. The result for
This performs modulo-2 addition, parity + data is thus as follows
i.e. integer addition modulo 2.
EVEN parity: 00101101 ODD parity: 10101101
+ 0 1 Now suppose that the byte 00101101 (most significant bit(MSB) a parity bit)
0 0 1 is read from disk and even parity is used. To check that this byte has been read
1 1 0 reliably, the parity bit for its 7 data bits is computed (by hardware or software)
using XOR (0) and compared with the MSB using XOR again (0 ⊕ 0 = 0). The
transmission or disk read is judged reliable if the regenerated parity bit agrees
with the received parity bit. This judgement is not always correct as two bits or
an even number of bits may be corrupted during transmission. However, use
101 Single licence - Abingdon School
5.5 Information coding systems
of a single parity bit is usually sufficient except when circumstances dictate that
full error-detection capability is required.
Questions
12 Calculate
the parity bit using even parity for the following 7-bit codes (a) 0111000 (b) 1110010
13 Calculate the parity bit using odd parity for the following 7-bit codes (a) 0111000 (b) 1110010
14 Explain how a receiver of a data transmission consisting of one parity bit and 7 data bits can detect that an
error has occurred affecting an odd number of the 8 received bits.
Checksums
Parity checking is good for checking asynchronous serial transmission of data Key concept
over short distances but not very good for synchronous serial transmission over
Checksum:
long distances. For the latter, checking must be applied to a block of data. The
The checksum is a number
data is treated as a sequence of fixed size numbers, e.g. each one byte in size. appended to the end of a block
These numbers are added together to form a total which is then truncated to of data and used for error
the same size as the number size, e.g. one byte, often by hashing. This truncated detection and correction. The
total is known as the checksum. The checksum is appended to the end of the block of data is treated as a
sequence of fixed size numbers,
block and for this reason is also referred to as a block check character. When
e.g. each one byte in size which
the data arrives at the receiver, the checksum is regenerated and compared are added together to form a
with the transmitted checksum. In this way the received data can be checked total which is then truncated
for errors that have arisen in to the same size as the number
Parity bit
Questions
15 What is a checksum?
1 1 0 0 1 0 1 0
16 The checksum for the block of data opposite uses even vertical parity. 0 0 1 0 1 1 0 1
1 1 1 0 1 0 0 0
Horizontally the most significant bit is an even parity bit. There is a 1 0 1 0 1 1 1 0
single bit error in this block. Can you identify which bit is in error? 0 0 1 1 1 1 1 1
1 1 0 0 0 1 0 1
1 1 0 1 1 1 1 0
1 0 0 1 0 1 0 1
Check digits
Check digits and parity bits are special cases of checksums. The maths used for
Key concept parity bits works for binary numbers but not decimal numbers. Thus different
methods must be used for making decimal number data such as credit card
Check digit:
numbers and book ISBNs reliable.
A check digit is a decimal digit
added to a number (either at A check digit is a decimal digit added to a number (either at the end or the
the end or the beginning) to beginning) to validate the number, e.g. a valid book ISBN.
validate the number, e.g. a valid
book ISBN. For example, the check digit in ISBN 978-0-
The main task of a check digit 9927536-2-7 shown in Figure 5.5.8, is the
is to detect a single corrupted rightmost 7 digit. This 7 is computed by an
digit and a transposition of two
algorithm applied to the information digits
adjacent digits.
of the number, i.e. 978-0-9927536-2. On Figure 5.5.8 ISBN-13
entering this ISBN into a computer, the check- book code
digit generating algorithm is applied to the 978-0-9927536-2-7
information digits of the ISBN as before, and showing check digit 7
the re-computed check digit compared with the
check digit that was entered (see later for a more
efficient way of doing this). In this way it is possible to check that the book
ISBN has been read correctly.
The three most common errors made by humans when keying numbers
into a computer, or reading and saying them, are omitting or adding a digit,
transposing adjacent digits and changing a single digit.
For example, transposing the digits 2 and 7 in 978-0-9927536-2-7 or changing
the triplet 992 to 922. The omission or addition of a digit is easily detected
without a check digit. Therefore, the main task of a check digit is to detect a
single corrupted digit and a transposition of two adjacent digits. Other types of
error are rare.
Check digits normally use modular arithmetic. The mathematical function
a mod b returns the remainder of the integer division a /b, an integer in the
range 0 to b - 1. Given a number N that consists of decimal digits d1 d2 d3 ...,
the simplest way to compute a check digit C for N is to solve the equation
(C + d1 + d2 + d3 + ...) mod p = 0
choosing an appropriate value for p. Note that for the lefthand side of this
equation to be 0
(C + d1 + d2 + d3 + ... ) must be a multiple of p
Therefore, this equation can be solved by first computing the sum S as follows
S = (d1 + d2 + d3 + ... ) mod p
and then using the fact that if C is restricted to the range 0 to p - 1,
C+S=p
Rearranging, C=p-S
Example
Suppose N is a three-digit number and each digit is in the range 0 to 4, inclusive, then a good choice for
p is 5.
If N = 342, S = (3 + 4 + 2) mod 5 = 4
then C = 5 - 4 = 1
The check digit 1 is appended to the number N and the 4-digit number 3421 is given over the
telephone, stored in a computer or transmitted over a communication line. At the receiving end, the
4-digit number is checked. If no digits have been corrupted, the calculation (3 + 4 + 2 + 1) mod 5 will
yield 0 (remember (C + d1 + d2 + d3 + ...) mod p = 0). However, if the received 4-digit number has been
corrupted in a single digit, e.g. it became 3221, then the calculation (3 + 2 + 2 + 1) mod 5 yields 3 when
it should be 0. Detection of single-digit errors are possible with this simple check digit mechanism.
Example
Suppose N is a three-digit number and each digit is in the range 0 to 4, inclusive, then a good choice for p
is 5. The weights chosen are 2, 3, and 4 because they are relatively prime to 5, i.e. 5 does not divide any of
them evenly.
If N = 342, S = (2∙3 + 3∙4 + 4∙2) mod 5 = 1
then C = 5 - 1 = 4
The check digit 4 is appended to the number N and the 4-digit number 3424 is read over the telephone,
stored in a computer or transmitted over a communication line. At the receiving end, the 4-digit number is
checked. If no digits have been corrupted, the calculation (2∙3 + 3∙4 + 4∙2 + 4) mod 5 will yield 0. However,
if two adjacent digits of the 4-digit number have been swapped because of an error, e.g. it became 3244,
then the calculation (2∙3 + 3∙2 + 4∙4 + 4) mod 5 yields 2 when it should be 0, thereby detecting an error.
ISBN
ISBN-13 has a total of 13 digits and includes a check digit. It conforms to EAN-13, the
European Article Numbering barcode system. The commonly used ISBN-10 book codes
have been turned into ISBN-13 by prepending 978. ISBN-13 book codes can use an EAN-13
barcode and therefore be barcode scanned. For example, ISBN-13 book code 978-0-9927536-
2-7 has 978 followed by language/country code 0, publisher code 9927536, book number 2,
and check digit 7.
To calculate the check digit:
Add up all the even numbered positions and multiply the sum by 3.
Sum the odd numbered positions.
Total the two sums.
Add a number that rounds up this total to the nearest multiple of ten.
This number is the check digit.
Algebraically
S = (1∙9 + 3∙7 + 1∙8 + 3∙0 + 1∙9 + 3∙9 + 1∙2 + 3∙7 + 1∙5 + 3∙3 + 1∙6 + 3∙2 ) mod 10 = 3
C=p-S
C = 10 - 3 = 7
The check digit C is therefore 7.
Questions
17 What is a check digit?
18 What are the three most common errors made by humans when keying numbers into a computer, or
reading and saying them?
Using an example, describe how a check digit is calculated so that it can be used to detect two of these
19 commonest errors?
Questions
1 Explain why instructions, e.g. calculate square of 9, and data,
e.g. speech, must be transformed before the hardware of a digital
computer is able to process these instructions or data.
Graphics
One way of structuring bit patterns is the Joint Photographic Experts Group
(JPEG) method for images produced by digital photography.
A JPEG file stores a digitised image as a sequence of bit patterns obtained,
for example, from a digital camera that captures a scene photographically by
sampling the brightness (or intensity) of the colour components of the scene
before digitising the result 0 255 255 255 255 255 255
Information in numbers to produce a 255 255 255 255 255 255 255
MatLab: JPEG formatted digital image 255 255 255 255 255 255 255
255 255 255 255 255 20 0
http://uk.mathworks.com/ representation of the scene –
255 255 255 255 255 255 255
products/matlab see Figure 5.6.1.2. 255 255 255 255 255 255 255
GNU Octave: When this JPEG file’s contents 255 255 255 255 255 255 255
255 255 255 21 0 255 255
http://mxeoctave.osuv.de/ are accessed and processed
correctly the digitised Figure 5.6.1.2 Image data taken from a
Redang.jpg:
recording of the original section of the JPEG formatted file, Redang.jpg
www.educational-computing.
co.uk/CS/Images/Redang.jpg
scene can be displayed as and displayed in decimal for ease of viewing.
shown in Figure 5.6.1.3. The
sequence of bit patterns serves to convey both
the digitised image itself plus information
(metadata) about the image such as its
dimensions, in this case 600 x 800.
Questions
3 Outline a method by which an image of a scene can be captured in Figure 5.6.1.3 600 × 800
digital form so that it can be displayed on an image display device. digital image stored in file
Redang.jpg
A relatively easy way to explore digital images is to use either Matlab from
Information MathWorks or GNU Octave, an open source system. The same scripts and
Octave: commands execute in either. For example, the following script
You will need to add the Z = imread('Redang.jpg');
command disp(info); to output
the value of the info. info=imfinfo('Redang.jpg');
image(Z);
Using MatLab’s Pixel region Image Tool as shown in Figure 5.6.1.7, the Red
(R), Green (G) and Blue (B) sample values of any region of the displayed image
can be retrieved.
Figure 5.6.1.7 Pixel Region Image Tool showing Red (R), Green (G)
and Blue (B) values in a region of the image.
Questions
4 A digital image file stores bit patterns representing intensity values
of samples of the scene captured by the imaging device. What other
information about the image is also stored in the image file and why?
imwrite(C, 'RedangChanged.jpg');
Figure 5.6.1.8(a) Array W rendered. Figure 5.6.1.8(b) Array C rendered showing the
effect of doubling every red value in W
Questions
5 Explain how each of the red and the green components of an RGB image can be reduced by 50% in
MatLab or GNU Octave.
PlaneGrey.jpg:
www.educational-computing. clear all;
co.uk/CS/Images/PlaneGrey.jpg C = imread('PlaneGrey.jpg’);
figure(1);
image(C);
C(:,:) = 255 - C(:,:);
Information
figure(2);
Octave:
imshow(C);
You will need to add the
command disp(info); to output imwrite(C, 'PlaneGreyNegative.jpg');
the value of the info.
If the intensity values in array C are subtracted from 255 then an intensity value
of 255 becomes an intensity value of 0, and an intensity value of 0 becomes an
intensity value of 255, and so on.
Thus we get the negative of this image when we update C as follows
C(:,:) = 255 - C(:,:);
Figure 5.6.1.9(a) 480 × 640 greyscale image Figure 5.6.1.9(b) 480 x 640 negative greyscale image
Programming tasks
1 Whenever the red, green and blue components of an image sample have the same value, the colour
displayed is a shade of grey. This means that a digitised image of sampled red, green and blue colours has
the potential for 256 shades of grey if each colour is encoded with 8 bits (0..255). We can use the intensity
of the overall colour, i.e. red + green + blue, to assign a shade or level of grey. The intensity of a colour
called the luminance is calculated as follows
Write a program or script in MatLab or GNU Octave that uses this formula to set the colour of each pixel
of an RGB image to a shade of grey to produce an equivalent greyscale image.
2 If you succeeded in turning an RGB image into a greyscale image you may have noticed that the result is
not as expected. This is because the formula method used in Programming task 1 did not take into account
the way that the human eye perceives luminance, e.g. the eye is less sensitive to blue light than red. We
need to adjust for this by weighting as follows
3 Write a program or MatLab/GNU Octave script to rotate an image through 180 degrees, i.e. turn the
image upside down.
A digital image can be created without using a camera. We can instead create a digital coloured image
by creating a three-dimensional array of numbers, D, as shown in Figure 5.6.1.10. D is populated with
values, 0 and 255 or in binary 00000000 and 11111111, representing the intensity of red, green and blue
with 255 being the strongest and 0 the weakest.
The MatLab/GNU Octave script to generate this array, to render it as an image and write the data to a
file Squares.jpg is as follows
[0 255 0 255; 255 0 255 0; 0 255 0 255] is the way that MatLab/GNU Octave creates a two-
dimensional array, each sequence of numbers is a row vector with rows separated by ';' so putting the
row vectors together we get, in this instance,
0 255 0 255
255 0 255 0
0 255 0 255
As we have three primary colours, three of these 2-D arrays are required, one for each colour, Red, Green,
Blue. (3) BLUE
0 255 0 255
(2) GREEN
0
0 255 0 255
(1)RED 255
0
0 255 0 255
255
255 0 255 0
0 255 0 255
The command:
imwrite(D, ‘Squares.jpg’)
scans array D,as it does so writing its values to a bit stream for file Squares.jpg
using the format required by JPEG.
(3) BLUE
0 255 0 255
(2) GREEN
0
0 255 0 255
(1)RED 255
0
0 255 0 255
255 Figure 5.6.1.11 Outcome
255 0 255 0 1 0 0 1 of executing image(D), an
255 255 0 1
0 0
1 1 image of 3 by 4 squares.
DISK
Figure 5.6.1.13 Python 3.4 script to read byte by byte contents of a bitmap file Fruit1.bmp
where ImageWidth is expressed in pixels. A pixel is a picture element and is the smallest area of the picture that is
sampled and digitised.
The total number of bytes to store an array of pixels, ArraySize, is
RowSize × ImageHeight
Programming tasks
5 Write a program that opens a BMP image file as a file of byte. The program should copy the first 1078
bytes of the file into a new file, then write the 8-bit ASCII codes for "HELLO WORLD" to the new file after
this. It should skip copying the next 11 bytes of the original file (which are effectively replaced by "HELLO
WORLD") and then copy the rest of the data in the original file into the new BMP file. Note where the
message starts. View the new BMP file in an image viewer. Can you detect where the original image has
been altered?
Now write a program to extract the message that has been stored in the image file. The program should
use the same message starting position as was used in the program that stored the message.
Tasks
1 Investigate steganography and digital watermarking.
Figure 5.6.1.14 JES GUI showing the Command window and the Sound
Tool window and sample 326368 whose value is -2963.
The command makeSound(bitStream) reads the bit patterns from bit
stream bitStream which itself is connected to WAV file Me2.wav. It
extracts the sampling rate, the number of bits per sample and the type of
recording (mono or stereo) all of which are stored in this file. With this
information, makeSound(bitStream) constructs either a one-dimensional
Information array (mono) or a two-dimensional array (stereo) and then stores the bit stream
bit patterns in the constructed array.
For sound file I/O by far the
Figure 5.6.1.15 shows Me2.wav opened by a Python 3.4 script running in
best add-in module for Python
is “pysoundfile”. Spyder. It extracts the sampling rate and assigns this to variable samplingRate
http://pysoundfile.readthedocs. and the sound data which it assigns to variable soundData. Before printing
org/en/0.8.1/ both, samples per second (44100) and array soundData [9 -2 12 …, -50,
-48 -47].
Information
Spyder:
https://store.continuum.io/
cshop/anaconda/
Spyder is part of the Anaconda
system that gives access to
scientific routines including
support for arrays and digital
signal processing.
Figure 5.6.1.15 WAV file Me2.wav opened in Spyder by a Python 3.4 script.
Programming tasks
6 Using JES, Spyder Python 3.4 or another programming/scripting system that supports exploration of
digitally recorded sound, write a program/script/commands to open WAV files, read the stored sampled
sound values and display these. Try also to extract the sampling rate and bits per sample.
Figure 5.6.1.16 Generating mathematically a sequence of numbers that represent a time sequence of samples of
a continuous tone of frequency 1000 Hz sampled at a rate of 20000 samples per second or one every 1⁄20000th
of a second. The sequence is written together with the sampling frequency and the bits per sample, to file
Tone.wav.
Programming tasks
7 Using MatLab or GNU Octave, mathematically generate separate WAV files of the following tones (use
trigonometric function cosine and then repeat using trigonometric function sine)
Use sampling rate 20000 samples per second, bits per sample 16 and collect 20000 samples
(0 to 1 − 1⁄20000 in time steps of 1⁄20000 second).
1.0, 0.9239, 0.707, 0.3826, 0.0, -0.3826, -0.707, -0.9239, -1.0, -0.9239, -0.707, -0.3826, 0.0, 0.3826, 0.707, 0.9239, etc
1th
The values in this sequence are separated in time by of a second because the sampling rate used was 20000
20000 1th
samples per second. The “sampling interval” or “sampling period” for this sample rate is .
20000
If we read this sequence from the beginning and write the sequence to a new WAV file, ToneFreqDoubled.wav,
omitting every other value, then the sequence in the new file is
1.0, 0.707, 0.0, -0.707, -1.0, -0.707, 0.0, 0.707, 1, etc
Figure 5.6.1.19 Spyder Python 3.4 script to double frequencies of digitally recorded sounds in a WAV file.
Play Tone.wav and ToneFreqDoubled.wav in a media player such as Windows Media Player and note the
difference in frequency.
Questions
6 Writing every other sample is one way of doubling frequencies of digitally recorded sound. Can you think
of another way that this could be done without having to omit sampled values and which could alter
frequencies by factors other than 2?
Programming tasks
8 Using JES, Spyder Python 3.4 or another programming/scripting system that supports exploration of
digitally recorded sound, write a program/script/commands to double frequencies of digitally recorded
sounds in WAV files.
Figure 5.6.1.21 Spyder Python 3.4 script to transfer sound samples to a text file
Likewise, it is possible to convert a text file into a sound file. Using Tone.txt for convenience, the Spyder Python
3.4 script shown in Figure 5.6.1.22 creates a WAV file, TextToSound.wav, of digitised sound samples. It sets
the sampling rate to 20000 samples per second but this can be changed easily to change the frequency of the tone
represented by this file and it sets the number of bits per sample to be 16.
Figure 5.6.1.22 Spyder Python 3.4 script to create a sound file from a text file
The Matlab command audioinfo can be used as shown in Figure 5.6.1.23 to obtain the metadata stored in file
TextToSound.wav.
Information
>> info = audioinfo('TextToSound.wav')
audioinfo: info = Filename: 'TextToSound.wav'
The audioinfo command is not
CompressionMethod: 'Uncompressed'
yet implemented in Octave
NumChannels: 1
SampleRate: 20000
Principle TotalSamples: 20000
Duration: 1
Text, digitised sound and
images: Title: []
Text, digitised sound and Comment: []
images are all just bits or bit
Artist: []
patterns under the hood. As
such they can be mapped BitsPerSample: 16
between each other by
transforming the way that the
bit patterns are arranged and Figure 5.6.1.23 MatLab command line >>info =
interpreted. audioinfo('TextToSound.wav')
Programming tasks
9 Using JES, Spyder Python 3.4 or another programming/scripting system that supports exploration of
digitally recorded sound and text files, write a program/script/commands to convert WAV files to text files
and vice versa.
Questions
7 It has been demonstrated that it is possible to transform sound and image files to text files and back again.
Give three reasons why this is useful.
120
100
80
60
40
20
0
Figure 5.6.2.2 Relationship between air pressure and height above sea level (Adapted from
www.engineeringtoolbox.com/air-altitude-pressure-d462.html with kind permission of the editor)
Information that is recorded in a continuous form and that is similar to its
Key concept
source’s original structure is also analogue data. The phonograph invented in
Discrete data: 1877 by Thomas Edison, known today as a record player, recorded speech
Information represented by directly onto wax cylinders by making physical deviations of a groove,
separate values is discrete. We
impressed into the wax, a replica of the variation in air pressure caused by the
say that these values are discrete
data. speech.
the set of all positive integers. Table 5.6.2.1 shows discrete data in the form of Key concept
temperature readings taken at hourly intervals.
Digital data:
What is digital data?
Digital data is discrete data
To store data digitally in a computer, it has first to be represented in discrete form, which has been encoded in
and then converted (encoded) to digital (binary) values. digital form, i.e. binary, using
some algorithm.
Figure 5.6.2.3 shows discrete data being encoded in binary by a process which
Since discrete information is
represents each discrete datum by a specific binary value, e.g. 4.7010 and 4.9310 conveyed by the sequence in
are both represented by 1002. This digitising process introduces errors called which the encoding symbols
quantisation errors, e.g. 4.7010 is represented by 1002 which is 410. are ordered, there must be
some way of determining the
beginning of a sequence. This
Quantisation is known as synchronisation.
Synchronisation is a property of
Discrete data digital data that distinguishes it
111 from analogue data. Machine
6.60 110 communications typically
use special synchronization
5.65 5.38 5.44 101 sequences to enable machines
4.70 4.93 100 to extract discrete information
Digital data
3.40 011 represented by digital data.
Signals are used for all sorts of purposes. Essentially a signal is that which conveys a message or information from
one place to another. As such, signals are subject to the laws of Physics, in particular Einstein’s special theory of
relativity that states that signals or the information that they carry cannot travel faster as a group than the speed of
light which is 3 × 108 metres per second in a vacuum.
The information carried by a signal is in the form of energy that can activate a detector or sensor in a receiver
of the signal. For example, the light from a warning beacon is conveyed as photons or light particles, each of
which carries a certain amount of electromagnetic energy, enough to stimulate cells in the retina of the eyes of
the receiver. This stimulation of the retina results in an electrical signal to the receiver’s brain which responds
accordingly.
Analogue
0 voltmeter
R showing a
fluctuating
Microphone voltage,
Sound waves Battery V = IR
Figure 5.6.2.4 Electrical circuit for converting sound energy into electrical energy
Information
Speech:
Pressure/Pa
0 0 000
1 2 3 4 5 6 7 8 9 10 11 12 Time/milliseconds 0 100
-2.5
−2.5 101
-5
−5 110
-7.5 −7.5 111
Figure 5.6.2.7 Digital signal
Table 5.6.2.2 Using digital
signals to encode binary data
It is possible to use just two distinguishable levels of voltage, 0 volts and 5 volts as shown in Figure 5.6.2.8. The
digital signal is then a binary digital signal. The shape in Figure 5.6.2.8 is called a voltage pulse.
Each voltage represents a binary datum, binary datum 1 by +5 volts and binary datum 0 by 0 volts as shown in
Table 5.6.2.3.
Voltage level Binary
5 1
0 0
Table 5.6.2.3 Using binary digital signals to encode binary data
The stream of voltage pulses shown in Figure 5.6.2.8 encodes the binary data 0110101010 (the least significant
digit is the first pulse to be produced).
Voltage/volts
0
0 1 2 3 4 5 6 7 8 9 10 Time/milliseconds
Questions
1 What is analogue data?
3 What is a signal?
Key principle 0
Time/milliseconds
Pulse Amplitude
Modulation(PAM):
Pulse Amplitude Modulation
is a process of measurement
of the amplitude (height) of
an analogue signal at fixed
and regular intervals of time Figure 5.6.3.1 Analogue signal plotted on a voltage-time graph
determined by the sampling
The analogue to digital conversion process consists of several stages:
frequency. The process outputs
a series of pulses whose 1. The analogue signal is sampled at fixed and regular intervals of time
amplitudes correspond to using sample and hold circuitry – see Figure 5.6.3.2 - to produce an
these measurements and whose
equivalent digital signal as shown in Figure 5.6.3.3.
duration in time is the time
elapsed between one sampling This form of digital signal is known as a Pulse Amplitude Modulation
and the next (the sampling (PAM) signal.
interval).
0
Time/milliseconds
Figure 5.6.3.2 Analogue signal sampled at fixed and regular intervals of time
Figure 5.6.3.2 shows this analogue signal sampled at fixed and regular time
Questions intervals. A sample is a single measurement of amplitude. The number
of measurements of amplitude per second is known as the sampling rate.
1 What is a sample?
Sampling rate is expressed as number of samples per second, e.g. 1000
samples per second. Sampling rate is also called sampling frequency. Sampling
2 What is the sampling rate
frequency is expressed in Hz, e.g. 1000 Hz is the equivalent of 1000 samples
for the following sampling
per second, and 1 KHz, which is the equivalent of 1000 samples per second.
frequencies
Figure 5.6.3.3 shows the digital signal produced from the sampled analogue
(a) 20000 Hz (b) 40 kHz
signal by a circuit that holds the current sampled value steady until the next
(c) 44.1 kHz?
sampled value is obtained.
Equivalent digital signal
7
4.5
Voltage/volts
0
Time/milliseconds
-2.4
Figure 5.6.3.3 Digital signal produced from the sampled analogue signal.
A 4-bit ADC is helpful in explaining how the measurements of voltage are converted to binary but not very useful
in practice; commercially available ADCs use a higher number of bits, e.g. 8, 10, 12, 16.
If we are dealing with a bipolar signal, i.e. one where the voltage may be positive or negative then the ADC must
be set to work across a voltage range that includes both positive and negative values. For the conversion shown in
Figure 5.6.3.4 the range is set from −8.5 to +7.5 volts i.e. 16 volts.
4-bit ADC
Voltage range ± 8 volts
No of bit patterns or levels = 16
Resolution = Voltage range = 1 volt
No of levels Amplitudes here all
+7.0 0111 recorded as 0100
+6.0 0110
+5.0 0101
+4.0 0100
+3.0 0011
+2.0 0010
+1.0 0001
+0.0 0000 Questions
-1.0 1111
-2.0 1110 3 Describe the stages of
-3.0 1101 the analogue to digital
-4.0 1100
-5.0 1011 conversion process.
-6.0 1010 Amplitudes here all
-7.0 1001 recorded as 1110
-8.0 1000
Figure 5.6.3.4 Levels for a 4-bit ADC and voltage range -8.5 to + 7.5 volts
coded in 4-bit two’s complement binary
Encoding samples using 4 bits gives 16 different bit patterns from 0000 to 1111. To cover both positive and negative
values of voltage, these bit patterns are interpreted as representing two’s complement binary, so voltages in the range
−0.5 to +0.5 are coded as 0000, −0.5 to -1.5 volts are coded as 1111, +6.5 to +7.5 volts as 0111 and −7.5 volts to
−8.5 volts as 1000. Table 5.6.3.1 shows the correspondence between voltage and binary code.
We can imagine that the ruler shown in Figure 5.6.3.5 has been used to measure the amplitude of the digital signal
shown in Figure 5.6.3.4, rounding up or down to a value from the set {−8.0, −7.0, …, +6.0, +7.0}. For example,
3.6 volts would be rounded up to 4.0 volts and coded as 0100, as would 3.5 volts. But 3.4 volts would be rounded
down to 3.0 volts and coded as 0011.
-8.0 -7.0 -6.0 -5.0 -4.0 -3.0 -2.0 -1.0 0.0 +1.0 +2.0 +3.0 +4.0 +5.0 +6.0 +7.0
1000 1001 1010 1011 1100 1101 1110 1111 0000 0001 0010 0011 0100 0101 0110 0111
Figure 5.6.3.5 Ruler for a 4-bit ADC
The ADC stores the binary code, e.g. 0110, for the current measurement of amplitude in an internal register before
transfer to the processor of the computer to which the ADC is connected. ADCs may be connected by serial (SIP
or I2C) or parallel interface depending on its design. Figure 5.6.3.6 shows the pulse code form of one 4 bit sample.
Binary code 1 is a 5 volts high pulse and binary code 0 is a 0 volts high pulse.
0 1 1 0
Figure 5.6.3.6 Pulse Code Modulation (PCM) form of one four bit sample.
Questions
4 Draw a ruler for a 3-bit ADC to measure an analogue signal that varies from −9.0 to +7.0 volts over a
range of 16 volts. The ruler uses two’s complement representation (HINT: see Figure 5.6.3.5).
Resolution
The purpose of an ADC is to output a PCM digital signal that represents measurements of the amplitude of
an analogue signal at fixed and regular intervals of time. The accuracy of the measurements are determined by
the number of bits that the ADC uses for its measurements, the more bits the greater the accuracy. Stating the
resolution of the ADC is one way of expressing this accuracy.
Resolution of an ADC is measured in terms of the number of bits per sample. The number of bits per sample is
referred to as the bit depth of the ADC or word length.
Resolution for a given analogue signal is defined in terms of the range of voltage measured and the number of levels
or bit patterns available as follows
Voltage range
Resolution =
No of levels
where No of levels = 2No of bits
Table 5.6.3.2 shows resolution for a voltage range 0 to +8 volts and various number of bits
No of bits No of Resolution
available to levels in volts
ADC 2No of bits
4 16 0.5
8 256 0.03125
12 4096 0.001953125
16 65536 0.0001220703125
00 01 10 11
Figure 5.6.3.7 Ruler for measuring voltages in range 0 to 4 volts in 2 bits.
A voltage measurement that lies between 0 and 1 volts would be therefore be coded as 00, a voltage measurement
between 1 and 2 volts as 01, and so on. The resolution is 4 volts = 1 volt .
4
However, given, for example, 01 as the coded measurement, we can only say that Key concept
the analogue signal’s amplitude at the time of measurement was in the range 1 to 2 Quantisation error:
volts or 1.5 ± 0.5 volts. The error in measurement
introduced by an ADC because
If the actual amplitude was 1.7 volts then the measurement would be rounded
of rounding down or up when
down to 1.5 volts and coded as 01. measuring the amplitude of an
If the actual amplitude was 1.2 volts then the measurement would be rounded up analogue signal.
The effects of quantisation errors produced by an ADC are most apparent when the number of bits is small. The
greater the number of bits the smaller the effects of quantisation error. Unfortunately, more bits means the quantity
of digital data is greater and therefore file sizes that store this data are greater too. Quantity of data is calculated as
follows
Quantity of data = No of bits per sample × Sample rate × Length in time of analogue signal
Music CDs are PCM recordings of analogue signals sampled at 44,100 samples per second using ADCs with a
resolution of 16 bits. So three minutes of mono sound would occupy 15.876 MB of storage according to Table
5.6.3.3. If two channels are used then three minutes of stereo sound would occupy 31.752 MB of storage.
Table 5.6.3.3 Quantity of data for various no of bits per sample and sample rates
Questions
7 (a) Draw a ruler for a 4-bit ADC to measure an analogue signal that varies in the voltage range 0 to 4
volts.
(b) What is the resolution of this ADC?
(c) What is the resolution in volts for the measurements of this analogue signal?
(d) What is the maximum quantisation error?
8 An ADC with a resolution of 10 bits is used to digitize an analogue signal of duration 180 seconds using
a sampling rate of 40000 samples per second. How many bytes will the ADC’s PCM produce?
9 A CD-ROM has a capacity of 737 MB (1MB = 1000000 bytes). How many 3 minute two-channel stereo
music recordings can be stored on this CD-ROM if the recordings were made in PCM from an ADC
with a resolution of 16 bits using a sampling rate of 44,100 samples per second per channel?
0
Time/milliseconds
Figure 5.6.3.8 DAC reconstructed analogue signal and the original analogue signal
Questions
10 What is the purpose
of a digital to analogue
converter?
Voltage/volts
0
Time/milliseconds
Physical quantities
Chemical
Biological
Sensor
Analogue sensors
The output signal from the majority of sensors is analogue so the signal must first be converted into a digital
signal before it can be passed to a digital computer system for recording and further processing.
Key concept To convert the analogue signal from an analogue sensor, the signal is fed to an
analogue to digital converter (ADC). The output of the ADC is a PCM
Analogue sensor:
signal (digital) suitable for transmission to a digital computer system.
A sensor whose output is an
analogue signal. Transmitting the PCM signal to a digital computer system is usual done through
a serial interface such as a UART or I2C or SPI (see Chapter 9.1.1).
The analogue signal may also need to undergo some conditioning before being applied to the ADC.
This signal conditioning takes the form of filtering
• to remove unwanted frequency components
• signal conversion to ensure its voltage range is correct for the ADC
• signal isolation for safety reasons in healthcare applications where there may be direct contact between a
patient’s body and the sensor.
The need to perform sense-transduce-signal condition-signal convert-output PCM onto a serial bus with analogue
sensors has led to the development of integrated circuits called MEMS that do all this.
Information
MEMS
MEMS stands for microelectromechanical systems. They consist of mechanical microstructures, microsensors,
microactuators, and microelectronics, all integrated onto the same silicon chip. Figure 5.6.3.11 shows a schematic
for a MEMS integrated circuit digital gyroscope and an actual MEMS 3-axis gyroscope that can be connected to a
Raspberry Pi.
Directions of + Ωz Yaw
+ Ωy
detectable Yaw
angular rates Ω
Roll
D OS
G
IG CO
YR
IT P
AL E
+ Ωx
Pitch
MEMS are known as smart sensors because they incorporate into a single integrated package or chip,
• sensing + transduction with an analogue signal conditioning interface circuit
• an integrated analogue-to-digital converter (ADC)
• a microcontroller and an I/O bus to provide serial output to other computer systems.
Figure 5.6.3.12 shows a simplified block diagram of a smart sensor on a chip.
136 Single licence - Abingdon School
5.6.3 Analogue/digital conversion
Sound
waves Parallel
Amplifier DAC to Serial
converter
Loudspeaker
Questions
11 What is the most common use for a Digital to Analogue Converter (DAC)?
12 What is a sensor? Why is an Analogue to Digital Converter (ADC) often required before the signal from a
sensor can be processed by a digital computer?
15 Why is a Digital to Analogue Converter (DAC) needed in order to play digitally recorded sound?
16 With the aid of diagrams, describe the process of converting a PCM signal into its equivalent analogue
signal.
17 An audio signal from a microphone was converted into a PCM signal using an 8-bit ADC and replayed
through a loudspeaker via a sound card employing an 8-bit DAC. A listener complained that the quality
of the reproduced sound was inferior to a PCM signal generated from the same audio signal using a 16-bit
ADC and replayed through the same loudspeaker via a sound card employing a 16-bit DAC. Explain why
the quality of the reproduced sound could have been perceived as different for the two systems.
■ Be aware that bitmap image This analogue electrical voltage must then be
files may also contain digitised to produce digital output.
metadata
In a digital camera, many such photosensors Figure 5.6.4.1 Sensor and its
■ Be familiar with typical array of photosensors
are arranged as shown in Figure 5.6.4.1. The
metadata
whole array is just called a sensor.
Did you know? Sampling and quantisation
Digital single-lens reflex
When, for example, a digital camera takes a picture of an object such as shown
cameras: in Figure 5.6.4.2, light from the object is projected through the imaging
These use an aspect ratio of system onto an array of light sensitive sensors (photosensors).
3:2. Aspect ratio is the ratio
of the width of the image to
its height. The Canon EOS
600D (released February 2011) Real world object
Imaging system
uses an APS-C CMOS sensor
consisting of a sensor array of
dimensions 5184 x 3456.
Pixel
Key concept
Figure 5.6.4.3 shows the result of sampling the image in Figure 5.6.4.2 at
Pixel: discrete coordinate positions ranging in the X-direction from 0 to 27 and the
A pixel is the smallest Y-direction from 0 to 19, and digitising the analogue voltage representing the
addressable region or element of
intensity of the light of the primary colours Red(R), Green(G) and Blue(B) in
a digital image. Each pixel is a
sample of the original image.
this light, e.g. at X = 18, Y = 12.
X
0 27
0
Key concept
Pixel:
A pixel is also the smallest Y
controllable element of a digital
image represented on the screen Pixel
i.e. the smallest element or at
X = 18, Y = 12
region of a digital image that Colour = A8231616
can be changed or edited when
19
editing bitmapped images
using software such as iPhoto, Pixel
or Picture element
Photoshop, Windows Paint, and
at position X = 3, Y = 14
other “paint” style packages. Colour = FFFFFF16
Its weakness is in changing size. At position X = 3 and Y = 14, the corresponding red, green and blue intensities
Pixel images can be reduced in are each represented by FF16 or 25510, the maximum value.
size, but lose quality when they
Figure 5.6.4.4 shows a smiley face drawn in pixel mode using Photoshop. The
are increased in size.
pixels for the eyes, nose and mouth were drawn individually by selecting the
relevant pixel and changing its colour.
Questions
1 What is a pixel?
The stored bits in memory are a digital representation of the image or just a
bitmap. We say that the image has been bitmapped.
A bitmapped image is a pixel-based digital image.
MEMORY
MAPPED TO
MEMORY
307
308
309
Questions
310
311
312 5 Figure 5.6.4.6 shows an image of a section of a chequered board
313
314 and a section of memory for storing the bitmap for this image.
315
316 The pixel size is defined as the size of a single square. Each
317
318 memory cell, e.g. cell 311, can store one byte. A bitmap is to be
319
320
321
created in the memory that will record the colour of each pixel as
322
any one of 256 different colours. White will be coded as 255,
Figure 5.6.4.6 black as 0 and the red used in the image as 125.
(a) Show how this image could be stored as a bitmap. Use the
given memory cells in your explanation.
(b) With each memory cell still representing one pixel the
memory is changed so that each cell can store 3 bytes. How
many different colours can be coded in one memory cell?
Express your answer as a power of 2.
Questions
This is approximately the angle
6 How many different intensities of colours can be represented for
subtended by a penny viewed
an individual pixel if the number of bits used for each quantised at 52 km, meaning it could
sample is be distinguished from another
(a) 12 (b) 15 (c) 18? penny immediately next to it.
Express your answer as a power of 2.
Resolution
A popular convention is to describe the resolution of a bitmapped image as
Key principle
the number of pixel columns (width) by the number of pixel rows (height), for
Resolution of a bitmapped example, 3264 × 2448.
image:
Image resolution = width of image in pixels × height of image in pixels
Image resolution = w x h
where For a given dimension of image, say 1.5 inches by 1 inch, the more samples
w = width of image in pixels that are taken across the image the smaller the pixels and the greater the
h = width of image in pixels
recorded detail.
Two objects
that cannot be resolved by sensor Voltage
To resolve the detail in a barcode, for as two objects out
example, it must be possible to pick
Key principle out both white and black bands in the
image. If the photosensors are too big Sensor
Displayed or printed
then this detail will be missed. Imaging
resolution of a bitmapped system
Power
image: However, the pixel dimensions of a in
This is expressed as pixels per bitmapped image, such as 5184 × Figure 5.6.4.8 Sensor too
inch or ppi.
3456 pixels, doesn’t give the image a large to resolve two objects
The choice of ppi for a clear, physical size. The bitmapped image’s
sharp image depends on the dimensions in pixels state only how many pixels there are, not how big each is.
viewing distance. Display and print resolution
The size of each pixel is set by specifying how many pixels should be fitted into
an inch when the bitmapped image is displayed or printed.
This is expressed as pixels per inch or ppi.
Key fact The choice of ppi for a clear, sharp image depends on the viewing distance.
Scanned and digital camera At the viewing distance, it should not be possible to see individual pixels.
images:
For images produced by For example, a 4"× 6" standard photographic print, printed at 300 ppi and
scanning or by a digital camera, viewed at about 11 inches appears fine, whereas a billboard-sized photograph
the clarity and resolution of a also appears fine because its viewing distance is so much greater, even though it
captured image is determined
is only printed at about 15 ppi.
by the size of the photosensors
(one photosensor = one pixel). Figure 5.6.4.9 shows a bitmapped image originally prepared for printing at (a)
This can be expressed as ppi. 10 ppi, (b) 50 ppi and (c) 300 ppi.
The image in (a) of 40 × 30 pixels still reveals its pixels at normal viewing
distance. We say that the
image is pixelated because
the individual pixels are
visible. Image (b) is 200
× 150 pixels and shows
much less pixelation but
(a) (b) (c) the sharpest image is (c)
Figure 5.6.4.9 bitmapped image printed at (a) 10 ppi, (b) 50 ppi and (c) 300 ppi which has dimensions
1200 × 900 pixels.
horizontal direction per unit measurement, with 1920 x 1080 pixels. Retina displays use 326 ppi
When introducing the iPhone 4,
e.g. inch, or the number in the vertical
Steve Jobs said that the number of
direction per unit measurement which is pixels needed for a Retina Display
the same thing for square pixels. The ppi of a computer display is therefore is about 326 ppi for a device held
related to the size of the display in inches and the total number of pixels in the 10 to 12 inches from the eye. At a
Image resolution is measured in samples per inch or loosely in horizontal pixels Did you know?
× vertical pixels. Display screens used by desktop computers typically have a
Screen resolution:
resolution of 96 ppi or lower.
The iPhone 5s (released
For the LCD monitor in Figure 5.6.4.11, 96 ppi is the maximum pixel density. September 2012) has a screen
Display devices usually allow the display settings to be changed, e.g. for the given resolution of 326 ppi and a
display, choosing 1280 pixels by 720 pixels changes the pixel density or resolution screen size of 1136 by 640-pixel.
Sony’s Xperia Z’s screen has
to
1280 pixels/20 inches or 720/11.25 = 64 ppi a screen size of 1080 × 1920
pixels and a resolution or pixel
for the same screen real estate of 20 × 11.25 inches. density of approximately 441
ppi.
Figure 5.6.4.12 shows the same section of a digitised image at different screen
resolutions or ppi. The dimensions of the image in pixels is unchanged but the
image’s size in the display goes from small to large as the resolution of the screen is reduced. The displayed pixels
become larger as the screen resolution is lowered.
Figure 5.6.4.12 The same section of a digitised image at different screen resolutions, with the pixel density of
the screen or ppi decreasing from left to right.
Questions
7 Calculate the screen resolution in number of pixels per inch for the following
(a) Apple MacBook Pro Retina 13.3"
screen size in pixels = 2560 × 1600 screen dimensions = 11.3 × 7.04 inches
(b) Microsoft Surface Pro 3 12"screen size in pixels = 2160 x 1440 screen dimensions = 10.0 × 6.67 inches
(c) Dell Ultrasharp U2414M 24 inch monitor
screen size in pixels = 1920 × 1200 screen dimensions = 20.43 × 12.77 inches
(d) Google Nexus 6
screen size in pixels = 2560 × 1440 screen dimensions = 5.19 × 2.92 inches
8 Large screens can get away with lower pixel densities because viewing distance is important with regards to
resolution. Use your answers for question 7 to justify this statement.
Camera resolution
The iPhone 5s’ iSight camera uses a chip with light sensors of width and height 1.5 × 10-6 metre in size, giving
dimensions for each pixel of 1.5 × 10-6 metres (1.5µm where µm is a micrometre) . This camera uses an aspect ratio
of 4:3 and therefore is 3264 pixels across by 2448 pixels down (3264 × 2448) or 7990272 pixels in total. Expressed
to 2 decimal places this is 7.99 megapixels or to none, 8 megapixels.
Apple actually increased pixel size in the iPhone 5s to 1.5µm (from 1.4 µm in iPhone 5) and kept the pixel count
the same by using a 15% larger sensor. The slightly larger sensor size and therefore pixel size improved low light
sensitivity and reduced the ratio of image signal to noise emanating from within the sensor in low light conditions.
This actually improved the quality of the image.
Sony Xperia ’s primary camera uses a chip of width and height 4128 × 3096 pixels. The total quoted number of
pixels is 13.1 megapixels. This is actually greater than the number which contribute to the final image because some
pixels are unused or are shielded from the light because they are around the edges of the sensor.
Questions
9 Why is it useful for smartphone cameras to have 4 megapixels or greater?
Questions
10 An image of size 640 × 480 pixels is to be printed at 300 pixels per inch. What will be the size of the
printed image in inches to one decimal place?
11 The size of a photographic print printed at 300 ppi is 4.2 × 3.2 inches. What was the size in pixels of the
digital image that was printed?
(a) How many pixels are printed in a 9.4 inch wide row?
(b) If the scanning resolution is n samples per inch, how many samples, in terms of n, would be taken
in a scan of one row across the film slide (0.94 inches)?
(c) What must the minimum value of the scanning resolution be in samples per inch to produce a print
of acceptable quality?
(d) Using your answer to part (c), what is the size of the bitmap in pixels produced by the scanner?
Express your answer in the form of number of pixel columns (width) by the number of pixel
rows (height). Both numbers are integers.
Figure 5.6.4.16, byte 10 states that the bitmap data begins at byte 54. Byte 77 0 Top row fourth pixel
34 states that the length of the bitmap data is 24 bytes, which indeed it is 76 0 BLACK
as Figure 5.6.4.15 shows. Microsoft chose to store the bytes in little-endian 75 0 0, 0, 0
fashion. In little-endian, the least significant byte is stored in the smallest 74 0 Top row third pixel
address. For example, "Where the data starts" is four bytes long and has value 73 0 BLUE
0, 0, 0, 54 which in decimal is just 54. 72 255 0, 0, 255
Information
Reading and writing bytes:
The source code of programs
to read and write a file of bytes
can be downloaded from www.
educational-computing.co.uk.
}
1 identifier must be ‘BM’. 77 ‘M’
2 file size The size, in bytes, of the bitmap file. 78
4 bytes,
3 file size 0
4 file size 0 0, 0, 0, 78
5 file size 0
6 reserved Reserved; must be zero. 0
7 reserved 0
8 reserved 0
}
9 reserved 0
10 bitmap data offset The offset, in bytes, from the 54 Where the data
11 bitmap data offset beginning of the 0 starts
12 bitmap data offset BITMAPFILEHEADER 0
0, 0, 0, 54
}
13 bitmap data offset structure to the bitmap bits. 0
14 bitmap header size 40
15 bitmap header size 0 4 bytes,
16 bitmap header size 0
0, 0, 0, 40
}
17 bitmap header size 0
18 Horizontal width of bitmap in pixels 4
19 Horizontal width of bitmap in pixels 0 4 bytes,
20 Horizontal width of bitmap in pixels 0 0, 0, 0, 4
}
21 Horizontal width of bitmap in pixels 0
22 Vertical width of bitmap in pixels If Height is positive, the 2
23 Vertical width of bitmap in pixels bitmap is a bottom-up DIB. 0 4 bytes,
24 Vertical width of bitmap in pixels DIB = Device Independent 0 0, 0, 0, 2
25 Vertical width of bitmap in pixels Bitmap. 0
26 Number of planes in the bitmap The number of planes for the target 1
27 Number of planes in the bitmap device.This value must be set to 1. 0
28
29
30
Bits per pixel
Bits per pixel
Compression The type of compression for a
24
0
0
} 2 bytes,
0, 24
}
33 Compression JPEG = 0x0004, PNG = 0x0005 0
34 Bitmap data size Size in bytes 24
35 Bitmap data size 0 4 bytes,
36 Bitmap data size 0 0, 0, 0, 24
37 Bitmap data size 0
Programming Tasks
1 Using Microsoft Paint (or equivalent), create and save a 4 × 2 pixels 24-bit uncompressed Windows
bitmap similar to Figure 5.6.4.15. Use the pencil tool to change the colour of individual pixels. You will
need to zoom in to make the pixels large enough to manipulate.
2 Write a program that opens and reads the contents of this file, byte by byte, displaying each byte as a
decimal integer on the console. Number these bytes starting from 0 so that the console output displays
number followed by byte value read from file. Check that your output shows similar values to those shown
in Figures 5.6.4.14 and 5.6.4.16.
3 Using Paint, change the colours of the pixels in the bitmap, noting the RGB values and re-run your
program. Check that the output from your program now reflects the new RGB values of the colours.
4 Using Paint, create and save an 8 x 2 pixels 24-bit uncompressed Windows bitmap, with differently
coloured pixels.
5 Re-run you program and note the relevant changes in the header (metadata) and the data part of the
bitmap displayed on the console. Do the displayed changes agree with what you expect?
6 Edit your program so that it also writes each byte that it reads from the opened bitmap file to a new file.
Save your edited program under a new name. The new bitmap file produced by running the new program
should be given a suitable name and the extension “.bmp”. At the moment it should be just a copy of the
original.
7 Edit your new program so that it alters the three bytes of a chosen pixel before writing these to the new
bitmap file. Now open the changed bitmap file in Paint and check that your program has changed the
colour of a pixel.
8 How could a short sequence of 8-bit ASCII character codes be placed in a bitmap file? Choose a much
larger image bitmap file than you have been working with, e.g. 640 x 480 pixels, and use your program
suitably modified to replace pixel bytes in this bitmap with a sequence of 8-bit ASCII codes. Check the
result by displaying the new image bitmap file in an image viewer, e.g. Paint. Can you detect the ASCII
codes?
9 Write a program that reads an altered image bitmap file and recovers the sequence of 8-bit ASCII codes.
Display these as characters on the console.
Questions
14 What is the minimum file size in bytes, for a bitmapped image that has a colour depth of 12 bits and
dimensions 640 × 480 pixels?
• colour depth
• size in pixels
Vector graphic:
■■ points in space,
A vector graphic can be ■■ straight lines or curves connecting these points
represented as a list of objects or
a list of drawing commands that ■■ shapes formed from closed paths (such as rectangles, circles, ellipses,
reference geometric objects triangles, other polygons, and non-regular shapes created from
straight and curved lines).
■■ Other commands apply fill operations to shapes or set the thickness
of lines.
All of these points, lines, curves and shapes make up what is seen as the image.
newpath
moveto 100 300
lineto 200 250
lineto 200 200
setlinewidth 2
stroke
showpage
Figure 5.6.5.2 Vector graphic image and the commands that produced it
Figure 5.6.5.2 shows a vector graphic image of two black lines and the
SCRIPT commands that produced it.
A command in this vector graphic scripting language consists of operands
(parameters) and an operator written operator operand1 operand2.
For example,
■■ moveto 100 300, moves a “phantom” pen to the point whose
coordinates are 100,300.
■■ The command lineto 200 250 creates the path for a straight
line from the position of the current point to the new current point
at 200,250.
■■ The command setlinewidth 2 sets the pen to draw lines with a
thickness of 2 units which means 2/72 inch if the unit is 1/72 inch
(72 ppi).
■■ The command stroke causes the pen to move along the defined path
inking it in.
■■ The final command in the script, showpage displays the page on the
screen.
The parameters supplied to the commands, e.g. 200 and 250 in command
lineto 200,250 are just numbers. Therefore, to apply a change, such as
resize or reshape, to a part of a vector graphic image we simply do arithmetic
Key fact on these numbers and then draw the vector graphic again.
Image quality: This means we can resize a vector shape as many times as we like, making it any
A vector graphic can be scaled size we need, without any loss of image quality.
without any loss of image
quality. This is why vector graphic images retain crisp, sharp edges to graphic elements/
As a consequence, the quality shapes no matter by how much we enlarge them, and, unlike pixel-based
of a vector graphic image is graphics, vector graphic images are resolution-independent.
independent of the resolution
of the display device, i.e. This means that when printing, the resolution of the printed image is
vector graphic is resolution determined by the highest resolution that the printer can print at.
independent.
Figure 5.6.5.3 shows the original vector graphic and a separate scaled by 2
version.
In Figure 5.6.5.3, the scale 2 2 command is applied to all the operand
numbers in the other commands, e.g. moveto 100 300 becomes moveto
200 600, setlinewidth 2 becomes setlinewidth 4.
newpath
scale 2 2
moveto 100 300
lineto 200 250
lineto 200 200
setlinewidth 2
stroke
showpage
Figure 5.6.5.3 Vector graphic image from Figure 5.6.5.2, its scaled
counterpart, and the commands that produced the scaled version.
Questions
1 How does a vector graphic represent images?
Display devices are mainly digital so to display a vector graphic image, it must
be rasterised, i.e. turned into pixels. There are some display devices that can
display the vector graphic image without needing to rasterise at all. These
essentially are Cathode Ray Tube-based (CRT) displays with a single colour
continuous phosphor coating applied to the screen. These are ideal for being
controlled by commands in a vector graphic script. Some radar displays,
monitors used in medicine and laser shows are operated directly in this way as
are X-Y plotters which are a kind of printer that uses vector data.
Questions
3 Explain why a vector graphic image is still displayed without any loss of image quality on a digital display
when it is scaled.
Tasks Information
These activities are designed to familiarise you with a vector graphic Postscript interpreter and
format. viewer Ghostscript:
Download from
Download and install Ghostscript 9.15 or later for Windows and
http://www.ghostscript.com/
GSview 5.0 or later. download/gsdnld.html and
1 Create the following file in a text editor, e.g. Microsoft Wordpad, and instal then download and install
GSview 5.0 or later: Download
save with extension .ps
from
newpath http://pages.cs.wisc.edu/~ghost/
0 1 0 setrgbcolor gsview/get50.htm
256 500 100 0 360 arc fill
newpath Download postscript language
1 0 0 setrgbcolor tutorial and cookbook,
100 500 50 0 360 arc fill PSBlueBook.pdf from
showpage http://partners.adobe.com/
public/developer/ps/sdk/
Open this file in GSview.
sample/index_psbooks.html
(a) What do you see?
(b) Change the fill colour of the first object to blue.
(c) Change the dimension of the second object from 100 to 50
(d) Change the arc from 0 360 to 0 180.
Tasks
Figure 5.6.5.4 Scalable Vector Graphics image produced from XML and rendered in a browser
The root element of all SVG images is the <svg> element. Here is how it looks:
<svg xmlns=”http://www.w3.org/2000/svg” xmlns:xlink=”http://www.w3.org/1999/xlink” >
Namespaces are defined in the <svg… /> part to avoid the names in the rest
of this file clashing with similar names elsewhere:
xmlns=”http://www.w3.org/2000/svg” xmlns:xlink=http://www.w3.org/1999/xlink
So the position of the first rectangle will be measured from x = 10, y = 10 and
therefore be drawn starting from x = 10+10, y = 10.
The position of the second rectangle will be measured from x = 300, y = 10
and therefore be drawn starting from x = 300 + 10, y = 10.
The result is shown in Figure 5.6.5.6.
Figure 5.6.5.7 shows another SVG image produced from XML and rendered
in a browser. The XML defines two shapes, a green rectangle and a red circle.
The XML-text of this SVG image was also produced with a text editor and is
shown in Figure 5.6.5.8.
Figure 5.6.5.7 SVG image
Tasks
5 Using a text editor such as Wordpad, create a copy of the vector graphic script shown in Figure 5.6.5.5
and save as TwoRects.svg. Experiment with changing the properties of each rectangle and observe the
outcome by opening TwoRects.svg in a web browser.
6 Repeat activity Task 5 for the vector graphic script shown in Figure 5.6.5.8. Save as RectAndCircle.svg.
7 Using a text editor such as Wordpad, create a copy of the vector graphic script shown below, save it as
Text.svg.
<svg
xmlns=”http://www.w3.org/2000/svg”
xmlns:xlink=”http://www.w3.org/1999/xlink”>
<text x=”10” y=”30” fill=”red”
font-family=” ‘Lucida Grande’,sans serif”
font-size=”32” transform=”rotate(90 20,30)”>
I love SVG!
</text>
</svg>
(a) Display Text.svg in a web browser.
(b) Experiment with changing the properties of the text and observe the outcome by
opening Text.svg in a web browser.
Tasks Information
8 Inkscape is an open-source vector graphics editor similar to Adobe Inkscape:
Download and install https://
Illustrator, Corel Draw, Freehand, or Xara X but it differs from these
inkscape.org/en/download/
because it uses Scalable Vector Graphics (SVG), an open XML-based
W3C standard, as the native format. Download and install.
Questions
4 List three types of object that could be found in a vector graphic image file.
Figure 5.6.6.1 Vector graphic image and its equivalent bitmap image,
original size and magnified 7 times
GSview was also used to produce the 24 bit, 72 ppi bitmap image from its
vector graphic equivalent.
Bitmap, or raster images as they are sometimes called, use a rather primitive
form of representation consisting of no more than a lattice of small rectangular
areas called pixels. In a bitmap file, the only information that is stored about
a pixel is its colour. Therefore, for the bitmap image of a black circle on a
white background displayed in Figure 5.6.6.1, there is no stored black circle
as such in the bitmap file. All that the computer knows about the image from
the bitmap file is that some of its pixels are black and some are white. When
the bitmap image is rendered on screen it appears as a black circle to the viewer
because that is how the viewer perceives the black and white pixels, but the
larger the pixels, in the displayed image, the harder this task for the viewer.
As the computer does not see the bitmap image representation as a circle object with properties, such as position,
radius, stroke (outline) colour, it cannot move the circle or transform it by, for example, changing its radius. All it
can do is to change the colour of pixels, e.g. all white pixels to red or all black pixels to green, say.
Even using a pixel editing package, moving the black circle is difficult to do precisely, especially if the edge of the
circle is antialiased so that some pixels on the edge have intermediate values between
black and white as in Figure 5.6.6.2.
Questions
1 For a bitmapped graphic, explain why it is difficult to move objects
from one position in the image to another.
However, storing the black circle on a white background image as a vector graphic
avoids using pixels altogether. Instead, the actual definition of the circle is stored as Figure 5.6.6.2 Enlarged
a circle object with its properties specified. Its properties are its (x, y) centre position antialiased bitmap
in the coordinate system of the image, its radius, its stroke (or outline) colour, stroke
thickness and its fill colour.
The immediate advantage is that the circle object can be treated separately from the rest of the image and its
properties adjusted to move or transform it.
Indeed, if the vector graphic image contains many circles, each can be selected separately and individually
manipulated. Also, with such an image, a computer can automatically, say, delete all circles or scale all black circles
to three times their size or paint all green objects red. There is no pixel selection taking place only object selection.
Questions
2 For a vector graphic, explain why it is relatively easy to move objects from one position in the image to
another and to manipulate objects in other ways
As an example of the way that a vector graphic allows easy selection of image
Information
objects, Figure 5.6.6.3 shows the contents of the part of an SVG file produced
Inkscape:
in Inkscape by drawing a red circle. If there was another circle in the image then Download and install
another <circle ……./> block of text would be stored in the file. https://inkscape.org/en/download/
bitmapped image or shape too much and it will lose its sharpness. Enlarge it
even more and the pixels that make up the image or shape can become visible,
resulting in a blocky or pixelated appearance.
Vector images tend to produce smaller file sizes than bitmapped image files
The original vector graphic image file for the black circle on a white background consisted of 44 bytes. Its
bitmapped equivalent was 1400 bytes. Thus, a vector image usually takes up less storage space than a bitmapped
image.
Bitmapped (raster) images always have a fixed size in pixels, e.g. 300 x 240 pixels. In an uncompressed bitmap,
doubling the dimensions, e.g. from 300 x 240 to 600 x 480 pixels, quadruples the size of its file because extra pixels
need to be stored. This is not the case for vector graphic image files because the number of vector shape definitions
remains the same no matter by what factor the image is scaled. All that changes are the property values for each
shape; the file size, or computer memory requirements, remain the same.
Vector images are easy to create, read and edit
Vector graphic image files are (generally) plain text files and as such they can easily be generated by hand or as the
output of user written programs. As text files, they are human-readable as well as generally free format, that is,
the text can be split across lines and indented to highlight its logical structure. A vector graphic image file can be
directly edited in a text editor. This is not the case with bitmap images.
Vector image has no resolution
Unlike bitmaps, vector graphic images are resolution-independent. Vector graphic images always print at the highest
possible resolution of the printer that they are sent to. They also display without loss of image quality at different
screen resolutions, keeping their dimensions the same, more or less.
On the other hand, when bitmapped images are printed, how they look always depends very heavily on their
resolution. Even though the bitmapped image may look good on the computer screen, printing requires much
higher resolution than computer screens display at. If the bitmapped image doesn’t have enough pixels to print at
the size required, its printed equivalent will be of poor quality.
Vector objects are reusable
It is very easy to pick an object from one vector graphic image,
Questions
transform it or restyle it without any loss of quality, and then to
insert it into another vector graphic. 3 State four advantages that vector graphics
has over bitmapped graphics.
Sinusoids
If we imagine the red dot inside the circle in Figure 5.6.7.2 is a cyclist cycling
round the circle at constant speed of, say, one complete loop of the circle
per minute then we are imagining
One cycle Sine
a periodic system because it has a
Time repeating pattern. If we were to lie
down in the plane of the circle, i.e.
view the circle sidewise-on, then
looking along the time axis across the
When the red dot returns to its starting page we would see the vertical distance
position after completing a revolution of of the cyclist from this axis vary in time
Cosine the circle it is said to have gone through
one cycle as shown in red. Mathematically, this
waveform is called a sine waveform or
Time sine wave.
Figure 5.6.7.2 Generating sinusoids - sine and cosine waveforms
If we look along the time axis going
down the page we see the vertical
Task distance of the cyclist from this new axis also vary in time as shown in
To see a demonstration of red. Mathematically, this waveform is called a cosine waveform or cosine
generating a sinusoid from wave. Both sine and cosine waveforms are called sinusoids. A sinusoid is
circular motion visit characterised by three quantities:
http://treeblurb.com/dev_math/
sin_canv00.html
1. Peak amplitude or just amplitude which is the maximum vertical
distance of the waveform from the time axis. In circle terms this is the
radius of the circular path that the cyclist follows.
Task 2. Frequency which in circle terms is the number of cycles of the circle
per second, e.g. 60 loops of the circle per second. In pitch or frequency
Download and install
SFS/ESynth from terms it is expressed in Hz, e.g. 60 Hz.
https://www.phon.ucl.ac.uk/ 3. Phase which in circle terms is where the red dot starts. Conventionally,
resource/sfs/esynth.php this is measured in angle from the time axis, e.g. for the sine waveform
Experiment with generating
time axis, angle = 0 degrees; for the cosine waveform time axis this is
sinusoids using ESythn.
90 degrees.
Figure 5.6.7.3 shows waveforms generated by ESynth with frequency 200 Hz,
amplitude 0.5 and phases 0 and 90 degrees.
Sampling a waveform Key concept
Sampling rate and bit depth (sample resolution) Sampling rate:
We have learned that the term periodic refers to any waveform that can be Sampling rate or sampling
described in terms of going round in a circle. frequency is the number of
samples taken per second.
Figure 5.6.7.4 shows one cycle of a sine wave generated along the time axis
by recording the vertical distance of the clock hand from this axis in time. The Bit depth:
clock hand rotates anticlockwise and the recording starts when this hand is Bit depth or sampling
resolution is the number of bits
pointing along the time axis (phase = 0 degrees).
allocated to each sample.
To create a sine-wave generator (oscillator) in a digital computer all that is
needed is to store, at
successive intervals of time, Positions marked | on clockface
the vertical distance of when waveform sampled Sample point
One cycle
the clock hand from the per second B
sampling. C Time/s
C A
0 0.25 0.5 0.75 1.0
The positions marked A
D
in red on the clock face
indicate the moments in Sampling clock D
Sample point
B
Figure 5.6.7.7 One cycle of Figure 5.6.7.7 shows one cycle of the
Time
reconstructed wave, sampling rate A
C
reconstructed wave for a sampling rate of 4
4 samples per second samples per second - see Figure 5.6.7.4.
D
Sampled B
Z
Spurious frequency Time/s
C A
frequency
D
Sampling clock
1 Hz
0 1 2 3 4 Hz 2.5 cycles
Sampling frequency 2.5 Hz 2.5 Hz
The fundamental frequency of the 2.5 Hz signal Reconstructed
when reconstructed from the samples does not
match the original’s
Figure 5.6.7.11 Sampling a waveform at a sampling frequency which is less than twice the waveform’s
frequency results in an alias (spurious) frequency replacing the sampled waveform’s frequency
However, the waveform constructed from these samples has a repeating pattern
Key principle
frequency which is not 2.5 Hz. Its frequency is approximately 1.25 Hz. This is
Nyquist’s theorem: an artifact called a spurious or alias frequency, i.e. one that does not really exist.
When sampling a (complex)
This known as aliasing. However, the waveform that could be constructed
periodic waveform, we must
sample at twice the highest from samples of the 1 Hz sine waveform (red dotted curve) does have a
frequency present in the repeating pattern frequency of 1 Hz.
waveform, at least, if all the
frequencies present in the It turns out that when sampling a (complex) periodic waveform, we must
(complex) periodic waveform sample at twice the highest frequency present in the waveform, at least, if all
are to be preserved. the frequencies present in the (complex) periodic waveform are to be preserved.
This is known as Nyquist’s theorem. Figure 5.6.7.12 illustrates this with a
sinusoid (cosine waveform) of frequency 1 Hz. The sampling rate is twice this
Task at 2 samples per second but it is still possible to construct a waveform with
Sampling a rotating image at fundamental frequency, 1 Hz, the same frequency as the original.
too low a frequency can result in
the rotating image appearing to Sample points A and B Sampling rate = 2 samples in 1 cycle
rotate at a lower frequency than it Signal = 2 samples per cycle
actually is. amplitude Reconstructed signal
A
Try observing rotating ceiling fan
blades whilst blinking your eyes.
The fan blades may appear to Time
rotate at a lower frequency than
they really are. We call the false
frequency of rotation a spurious
B
frequency. It is a consequence of
Nyquist’s theorem.
Original signal
Questions
1 Explain the terms (a) sampling rate (b) bit depth or sample resolution.
2 An analogue waveform made up of the following sinusoids with frequencies 1 kHz, 5 kHz, and 10 kHz is
sampled and the samples digitised. When the digitised result is processed, it is discovered that it is made up
of sinusoids with frequencies, 1 kHz, 5 kHz and 7.5 kHz but not 10 kHz. Suggest the most likely reason
why this has happened and suggest one possible solution.
3 Why are music CDs recorded at a sampling rate of 44100 samples per second?
at 44100
samples per
Did you know?
The most general term for a
second because
frequency component of a
what makes complex tone is a “partial” or
the note sound “overtone”. All harmonics are
like a violin partials, but partials can be
note are the either harmonic or inharmonic.
harmonic Instruments such as bells, and
frequencies other metallophones, generate
Amplitude (dB)
a multitude of inharmonic
that are also
partials. Waveforms containing
present. This Figure 5.6.7.13 Spectral content of the inharmonic partials will not
is called the sound “Laa” recorded at 44100 samples per
be periodic at the fundamental
second
quality of frequency, but may or may not
the note or be over a longer time.
timbre that
distinguishes Did you know?
Frequency (kHz)
it from other Modified Shannon-Nyquist
Theorem:
sounds of the same pitch and volume, e.g. a violin note from a trumpet note. The
States that the highest frequency
fundamental plus harmonics of a sound are called the spectral content. Figure
component in the source must
5.6.7.13 shows the spectral content of a recording of the sound “Laa”. A Discrete be less than half the sampling
Fourier Transform(DFT) has been applied to the recorded waveform to reveal frequency.
a fundamental at 155 Hz and 22 harmonics, some of which are not displayed
because their amplitude is too small. The highest harmonic frequency is 3895 Information
Hz. Fourier analysis is the process of finding which sine waves need to be added Discrete Fourier Transform:
together to make a particular waveform shape. The DFT works with digital Converts from the time domain
samples. If the sound “Laa” had been sampled at, say 4000 samples per second, to the frequency domain and lists
which is not at least twice the frequency of the highest frequency and ten other the frequency components of the
harmonics, the recording would have been distorted by including frequencies not signal.
Korg Wavestation
synthesiser
emulator running
on Windows 7
USB cable
Figure 5.6.8.1 Korg 61-key keyboard connected to Korg’s Wavestation synthesiser emulator
Information Pressing a key on the Korg keyboard sends a message to the computer program
Introduction to computer to play the note corresponding to this key. A note number is assigned to each
music: key on a MIDI keyboard. For the keyboard in Figure 5.6.8.1, note numbering
http://www.indiana.edu/~emusic/ starts at 36 and runs consecutively up to 95 as shown in Figure 5.6.8.2.
etext/toc.shtml
MIDI note number 60 has been assigned to middle C on this keyboard. Note,
number 60 corresponds to frequency 261.63 Hz but the MIDI specification
allows this mapping to be changed.
Single licence - Abingdon School 171
5.6.8 Musical Instrument Digital Interface(MIDI)
Table 5.6.8.1 shows the usual correspondence between MIDI note number and frequency for this and some other
MIDI note numbers. The name given to each note is also shown. Note that musical pitch (note frequency) is
not embedded in any way in MIDI Note messages, thereby allowing mapping from note number to pitch to be
changed. Nor is note name tied to a specific frequency (tuning a musical instrument adjusts frequency).
37 39 42 44 46 49 51 54 56 58 61 63 66 68 70 73 75 78 80 82 85 87 90 92 94
Middle C
D4 E4 F4 G4 A4 B4
C2 C3 C4 C5 C6 C7
36 38 40 41 43 45 47 48 50 52 53 55 57 59 60 62 64 65 67 69 71 72 74 76 77 79 81 83 84 86 88 89 91 93 95 96
Figure 5.6.8.2 Some notes from an octave (white and grey keys only), their MIDI note
number and one possible assignment of frequencies
MIDI 60 62 64 65 67 69 71 72
note no
Frequency Hz 261.63 293.67 329.63 349.23 392.00 440 493.88 523.25
Note name C4 D4 E4 F4 G4 A4 B4 C5
Table 5.6.8.1 MIDI note no, its usual corresponding frequency in Hz and its usual note name
Information
MIDI itself does not make sound. It is just a series of messages to turn notes on
and off, etc. These messages are interpreted by a MIDI instrument to produce Octave:
The range from C4 to C5 is an
sound. A MIDI instrument can be a piece of hardware (a synthesizer) or a
octave. The grey keys indicate
software tool (Wavestation emulator, MuLab, Logic Pro). the musical interval of an octave
The most common tool used to generate MIDI messages is an electronic (12 semitones) between notes.
keyboard. These messages may be routed to a digital synthesiser inside the An octave has the property that
the ratio of the frequencies at the
keyboard or they may be patched (wired) to some other MIDI instrument such as
ends of the range is 2:1. In equal
a computer running synthesiser software. Almost all MIDI devices are equipped to temperament, an octave is defined
receive MIDI messages on one or more of 16 selectable MIDI Channel numbers, to be 12 equal semitones in the
labelled 1 to 16 (supports “multi-timbral” performance). modern scale. Each semitone
1/12
therefore has a ratio of 2
MIDI messages
(approximately 1.059). Note A4
The most common MIDI messages are Voice Channel messages. Voice Channel is assigned frequency 440 Hz.
messages convey information about whether to turn a note on or off on a Therefore, the frequency of the
particular channel, what instrument sound to change to, and so on. nth semitone above or below A4
Voice Channel MIDI messages consist of two or three bytes as shown in Figure is
2n/12 x 440 Hz.
5.6.8.3 (Status byte followed by one or two Data bytes). For the serial hardware
interface, each byte is surrounded by a start bit and a stop bit, making each packet
10 bits long. Within a MIDI software system data is 8-bit bytes. The first byte, called the Status byte, takes on
values ranging from 0x80 to 0xFF in hexadecimal or 128 to 255 in decimal - most significant bit (MSB) is ‘1’. The
Data bytes, take on values in the range 0x00 to 0x7F or 0 to 127 - most significant bit of each byte is a ‘0’.
The transmission bit rate of the hardware interface in the MIDI standard is 31,250 bits per second. Therefore, one
start bit, eight data bits, and one stop bit result in a maximum transmission rate of 3125 bytes per second.
MIDI uses the fact that the Status byte is in a different range from the Data bytes. If MSB = 1, the byte is a “Status”
byte. If MSB = 0, the byte is a “Data” byte. The first four bits of a Status byte are the code for the command, and
the last four bits the channel to which the command applies (e.g. 00002 is Channel 1, 11112 is Channel 16).
For example, when a key is pressed, the keyboard creates a “Note On” (Status byte = 0x91, 14510, 100100012)
message for Channel 2 consisting of three bytes, e.g. 145 45 100. The first four bits of the Status byte (10012) tell
MIDI that the message is a Note On command, while the last four bits tell MIDI what MIDI channel the message
is for (00012= MIDI Channel 2).
Loudspeaker
Virtual Machine
Channel 2 Wavestation
Piano Keyboard
MIDI
(VMPK) 145 45 100
instrument
LoopMIDI
virtual MIDI
connection
144 43 58 Channel 1
To generate messages that use different velocities requires a MIDI keyboard. Computer keyboards are not velocity
sensitive. Using a computer’s keys to play notes into a software synthesiser will generate note messages that all have
the same velocity.
When a key is released the keyboard creates another MIDI message, a
Information
“Note Off” message, e.g. 129 60 85. The first byte, 129, is the Status
byte - the first four bits of the Status byte (10002) correspond to “Note LoopMIDI:
Off”, the second four bits (00012) to the channel, i.e. Channel 2 (0000 is Information on LoopMIDI can be viewed
at http://www.tobias-erichsen.de/software/
Channel 1). The second byte is the key, 60 (middle C) in this example, loopmidi.html where it can also be
and the third byte is the velocity which indicates how quickly the key downloaded.
was released. The MIDI instrument can use the velocity value of 85 to
know how quickly it should dampen the note.
Figure 5.6.8.3 shows an on screen piano keyboard (VMPK) connected Information
via a virtual MIDI connection (LoopMIDI) to a running copy of the
Korg Wavestation:
Wavestation emulator. A Korg 61-key keyboard is connected via USB to Information on the Korg Wavestation is
the computer and Wavestation emulator. The output from Wavestation available at http://www.korg.com/us/products/
goes to a loudspeaker. The input channels 1 & 2 are monitored by a software/korg_legacy_collection/. It is not free
software. MULAB is an alternative that is free
piece of software called MIDI-OX.
- http://www.mutools.com/mulab-downloads.
In Figure 5.6.8.3 MIDI-OX shows that the “Note On” and “Note Off” html
Status bytes for Channel 1 have codes 144 and 128, respectively, whilst
for Channel 2, these are 145 and 129, respectively. Wavestation uses the
Channel values to route the messages on each Channel to different voices.
Messages can also have other purposes, e.g. to change the instrument sound. Figure 5.6.8.4 shows messages that do
this, e.g. 193 41 which is a two-byte code to change the MIDI instrument for Channel 2 (code 193 - 1100 00012,
11002 is the change code and 00012 is the Channel) to a viola (code 41). Such messages are control messages.
Pitch Bend (control to vary pitch) is another type of control message that a MIDI controller can send, e.g. 225 43
0 (Figure 5.6.8.4), causes pitch bend in a Channel 2 note. For Channel 1 the same control message would use a
leading byte with value 224 (1110 00002). There are many more ways to control the playing of a note and each has a
corresponding control code.
The playing of multiple notes “together” events in MIDI are sent as a string of serial commands so, for example, a
2-note chord will be transmitted as two separate messages, Status(Note
While True
On, Ch 1) key1-velocity Status(Note On, Ch 1) key2-velocity unless the
Do
synthesiser supports Running Status. In this case, a single Status byte’s
Wait for message
action is allowed to persist for an unlimited number of Data byte pairs
Process message
which follow.
MIDI messages Figure 5.6.8.5 Event handler
A MIDI message is the means by which an event in one system, e.g. key
pressed on a keyboard, is communicated or transported to another to produce an Information
event in the receiving system, e.g. a synthesiser plays a note. Chuck download:
http://chuck.cs.princeton.edu/
Keyboard-event → MIDI message → synthesiser event.
release/
For example, the “Note On” message sent by a MIDI controller to a MIDI
instrument causes an event to take place, i.e. the synthesiser plays the note
specified in the message by note number. The Virtual MIDI Piano Keyboard shown in Figure 5.6.8.3 is a MIDI
events generator and receiver. Event-driven systems rely upon a piece of software called an event handler which
consists of a non-terminating loop that “sleeps” when there are no messages to process, i.e. is suspended - Figure
5.6.8.5 - and springs into action when there is (in Chuck the loop takes the form of a “polling loop”).
Extension material
Chuck is an open-source and freely available programming language for real-time sound synthesis and music creation.
Figure 5.6.8.6 shows an event handler written in Chuck.
MidiIn midiIn; //create an event object
0 => int port; // select MIDI port 0
if( !midiIn.open(port) ) // if MIDI port 0 not open exit
{
<<< “Error: MIDI port did not open on port: “, port >>>;
me.exit();
}
MidiMsg msg; // makes object to hold next MIDI message
Wurley piano => dac; // select Wurley piano to play with MIDI controller
while( true ) // loop forever
{
midiIn => now; // wait on MIDI event, shred suspended but time advances
while( midiIn.recv(msg) )
{
if (msg.data1 == 144) //check that status byte = 144 which is Note On Channel 1
{
Std.mtof(msg.data2) => piano.freq; //convert MIDI no to corresp. frequency
msg.data3/127.0 => piano.gain; //set piano gain (data3 in range 0 to 127)
1 => piano.noteOn; //trigger note on
}
else //status byte not equal to 144 so switch note off
{
1 => piano.noteOff; //trigger note off
}
} Figure 5.6.8.6 Event handler written in Chuck programming language
}
Chuck
Virtual
LoopMIDI Virtual Loudspeaker
keyboard
Machine
Figure 5.6.8.7 Playing music via an executing Chuck event handling program
Figure 5.6.8.7 shows the use of VMPK virtual piano connected via LoopMIDI to the running Chuck event handler shown in
Figure 5.6.8.6. The output of the Chuck program is sent to a loudspeaker connected to the computer.
Questions
1 What is MIDI?
3 The following MIDI message consisting of three bytes is generated when a key is pressed on a MIDI
keyboard: 144 60 64
Explain the purpose of each of three bytes.
4 Note On is one example of a MIDI message. Give three other examples, each must be a different type.
5 Explain the statements “A MIDI keyboard is an events generator” and “MIDI messages are associated with
events”. What is the fundamental structure of event-handling software such as that found running in a
MIDI instrument?
6 State three main advantages of MIDI file representation of music over audio data file representation, e.g.
.wav file
The redundancies in data depend on the type of data (text, images, audio, etc)
which is why different compression methods have been developed. Each works
best with a particular data type.
Questions
1 What does it mean to compress data?
However, a disadvantage with lossy compression is that the lost data are not retrievable. The compressed data will
have very limited potential for adjustments or changes and every time the compressed data is uncompressed, edited,
compressed again and saved, more data is lost.
With lossless compression the original uncompressed data is always recoverable.
Online high-quality image retailers often display their images in low quality form, i.e. they use a lossy, compressed
version of high compression ratio, so potential customers can view what is on offer before purchasing. This protects
against theft of data as it prevents customers from accessing and downloading a higher-quality version. It is the
ability of lossy compression methods to allow the compression ratio to be varied from low to high that supports this
way of marketing images. Alternative, an uncompressed or lossless version can be made available to customers on
receipt of payment.
Questions
6 What is meant by lossless compression?
8 State one advantage that lossy compression has over lossless compression.
9 State one advantage that lossless compression has over lossy compression.
10 For each of lossless and lossy compression, give one example where it is used and why.
where the red values indicate counts. The problem is to distinguish a byte containing a greyscale value (such as 15)
from one containing a count (such as 8). There are several possible solutions.
In one solution, the 256 different greyscale values are reduced to 255 so that the 256th can be used as a flag to
precede every byte containing a count. Suppose this flag value is 255 then the sequence above becomes
255, 8, 15, 46, 81, 123, 58, 255, 4, 98, 255, 3, 7, 8
RLE works well with images that contain large areas of the same colour e.g. black and white images which are
mostly white, such as the page of a book. This is due to the large amount of contiguous bytes that are all the same
colour.
However, an image with many colours and relatively few runs of the same colour such as a photograph containing a
high degree of colour variation will not lend itself to compression using RLE so well.
The direction of scan can also affect the compression ratio. For example, an image that has lots of vertical lines will
not compress well if it is scanned horizontally for same-pixel runs but will if scanned vertically. A good RLE image
compressor should be able to scan a bitmap by rows, columns, or in a zig-zag pattern and be able to choose the scan
output that produces the best compression ratio.
Questions
11 Explain the principles of run length encoding lossless compression.
12 The following numbers, restricted to the range 0..254, represent the intensities of a contiguous block of
pixels in a greyscale bitmap
15, 112, 112, 112, 98, 76, 76, 15, 46, 46, 46, 46, 46, 19, 101, 6, ...
Using run length encoding, compress this block of pixels using 255 as the flag that prefixes an RLE packet.
13 Run length encoding works well, i.e. achieves a good compression ratio, with some images but not others.
Why?
14 Why is run length coding normally not a good choice for text compression?
Dictionary-based methods
We compress naturally in everyday life when, for example, referring to months of the year by number, e.g.
September by the number 9. Dictionary-based methods compress by using this technique. The dictionary is a kind
of look-up table, e.g. entry 9 is September. Dictionary-based compression methods vary in how the dictionary is
constructed and represented but they all use the principle of replacing substrings in a text, e.g. ‘th’ in ‘the’ with a
codeword, e.g. 1, that identifies that substring in a dictionary or codebook - see
Key concept
Table 5.6.9.1. The substring is called a phrase. Codewords for the dictionary
Token: are chosen so that they need less space than the phrase that they replace, thus
A unit of data written on
achieving compression. The process of compression is called encoding. The reverse
the compressed file. A token
consists of two or more fields.
process is called decoding. The compressor is an encoder and the decompressor is
In LZ78, the token consists of a decoder.
two fields, the first is a pointer
If we have to use a dictionary containing a large number of entries then the
to an entry in the dictionary
and the second is the code of a
overhead of storing or transmitting the dictionary is significant, and choosing
symbol, e.g. “A”. which substrings to place in the dictionary to maximise compression is also
A token is sometimes written difficult.The solution is to use an adaptive dictionary scheme based on methods
surrounded by chevrons < and > developed by Jacob Ziv and Abraham Lempel in the 1970s.
e.g. <2, A>
In the 1978, Ziv and Lempel described a dictionary-based algorithm (LZ78) that encodes a phrase (substring) of
n characters from the input in a codeword that points back to an earlier phrase in the input which it matches in
all but the last character, e.g. the B in BA matches B at
Input string A B AA BABAA
position 2 in the example shown in Figure 5.6.9.2 so
Dictionary
is encoded as the two-field token 2, A. The first field of
A B AA BA BAA
the token is the pointer 2 and the second the code of the Index 1 2 3 4 5
symbol, e.g. ASCII A. A B AA BABAA
The next symbol read from the input is the character B but there is no entry yet in the dictionary for this phrase,
and so B is added at position 2.
The third character read from the input, is A. This is matched with the A at position 1 in the dictionary. The goal
of dictionary encoding is to find the longest dictionary substring that matches the input so the next symbol is read.
This is another A. The dictionary is now searched for an entry containing the two-symbol string AA.
None is found, so the string AA is added to the next available position in the dictionary which is 3, and the token
1, A is output. This is the “compressed” version of the substring AA. We actually need to build up phrases in the
dictionary with at least three characters before we can replace a phrase with something shorter. Figure 5.6.9.3
shows that this happens at entry 5 in the dictionary.
An efficient way of representing the dictionary is a tree-like structure called a trie as shown in Figure 5.6.9.3 which
grows as more characters of the input are processed. All strings that start with the empty string (dictionary index 0)
are added as children of the root which is labelled 0.
Information
In the example, all strings that begin with A are located in the subtree with node
Trie:
labelled 1 (index 1 in the dictionary).
A tree data structure in which the
All strings that begin with B are located in the subtree with root labelled 2. string of characters represented
by node n is the sequence of
All strings that begin with AA are located in subtree with root labelled 3 and so
characters along the path from
on. the root to n. Given a string, the
The example in Figure 5.6.9.2 is too trivial to achieve a reduction in the number trie consists of nodes for exactly
those substrings that are prefixes
of bytes (assuming each character is represented in one byte). We need a much
of some other substring.
longer input string to achieve compression. The word ‘trie’ comes from the
The dictionary shown in Figure 5.6.9.2 and Figure 5.6.9.3 is constructed as middle of the word ‘retrieval’.
the input string is parsed (processed). This dictionary is empty when the first
Dictionary Encoded Empty string character, A, of the input string is read. The characters about to be
output
0 encoded are used to traverse the tree until the path is blocked, either
0 Empty
string
1 A <0, A> A B
because there is no onward path for the current character or because a
2 B <0, B> 1 2 leaf is reached.
3 AA <1, A> A A
The node at which the block occurs gives the index/phrase number to be
4 BA <2, A> 3 4
used in the output, e.g. 1 in <1, A>.
5 BAA <4, A>
A A new node, e.g. 3, is added and joined by a new branch to the node at
5 which the block occurred.
Figure 5.6.9.3 Data structure for The new branch is labelled with the last character of current string, e.g.
LZ78 coding - numbers in nodes refer A.
to dictionary index
For example, suppose we append BAB to the input string to form the
new input ABAABABAABAB. We need to add a new node 6 to Figure 5.6.9.3 and connect it to node 4. The path
from node 4 to node 6 is labelled BAB.
The encoder output is now the string of tokens
0, A 0, B 1, A 2, A 4, A 4, B
If we extend the input with BABB to form input ABAABABAABABBABB the encoder output is now the string of
tokens
0, A 0, B 1, A 2, A 4, A 4, B 6, B
The compressed form (ignoring commas which have been included to aid readability) occupies less space than the
uncompressed form, 14 bytes compared with 16 bytes.
On decoding, the decoder reconstructs the tree data structure so it can decode the token string - Figure 5.6.9.4.
Empty string Empty string Empty string Empty string
0 0 0 0
A A B A B
1 1 2 1 2
A A
Figure 5.6.9.4 LZ78 decoder reconstructs the 3 4
dictionary tree from the token string
A B
0, A 0, B 1, A 2, A 4, A 4, B
5 6
token by token
Dictionary Encoded
Empty string output
Our examples have used an alphabet of two symbols 0 0 Empty
string
A and B but what if the alphabet of symbols was A, A B E C 1 A <0, A>
B, C, D, E? 1 2 4 5
B
2 B <0, B>
The tree data structure dictionary would then A D 7 3 AA <1, A>
3 6
consist of a root for the empty string and then all E 4 E <0, E>
strings that start with the empty string (strings 8
5 C <0, C>
for which the token pointer is zero) are added to 6 BD <2, D>
Figure 5.6.9.5 LZ78 coding
the tree as children of the root. Figure 5.6.9.5 7 CB <5, B>
dictionary tree constructed for
shows the dictionary tree for the input string
ABAAEACBDCBBDE 8 BDE <6, E>
ABAAEACBDCBBDE.
Using the token structure <pointer, symbol> the output of the encoder for this
input string is,
<0, A> <0, B> <1, A> <0, E> <0, A> <0, C> <2, D> <5, B> <6, E>
Questions
15 Explain the principles of dictionary-based lossless compression. Empty string
0
16 This tree data structure was created when encoding a string using a dictionary-based
B A
lossless compression technique.
1 2
(a) What is the dictionary? B B
<0, B>, <0, A>, <1, B>, <2, B>, <0, B>, <1, B> <4, A> <4, B> A B
5 6
17 The output from a dictionary-based encoder is
<0, B> <1, B> <0, A> <1, B> <3, A> <1, D> <1, A> <2, B>
Draw the dictionary tree for this output and then decode the output.
■■ Why images and sound files are often compressed and that other files,
such as text files, can also be compressed
• to reduce the amount of storage space required to store the data
• to reduce the time taken to transmit the data
■■ The differences between lossless and lossy compression and the advantages
and disadvantages of each
• In lossless compression,
the compression algorithm does not remove information from the
original uncompressed data only redundancies
the original uncompressed data is always recoverable.
• In lossy compression,
the compression algorithm may remove information which is
irrelevant from the original uncompressed data
the original uncompressed data cannot be fully restored when the
reverse process is carried out but compression ratios are higher than
for lossless compression.
limited potential for adjustments or changes and every time the
compressed data is uncompressed, edited, compressed again and
saved, more data is lost
■■ The principles behind the techniques for lossless compression:
• run length encoding (RLE)
• dictionary-based methods
■■ Explain why Vernam cipher is Encrypting a message is one way to keep the message’s contents secret from
considered as a cipher with others who are not authorised to view its contents. The encrypted messages
perfect security look just like random gibberish as illustrated in Figure 6.10.1.
What is encryption?
Encryption is the process of obtaining ciphertext from plaintext. Before
Key concept
it is encrypted, the understandable (English) text is normally referred to as
Encryption: plaintext and after encryption it is referred to as ciphertext. The left hand side
Encryption is the process of
of Figure 6.10.1 is an example of ciphertext.
obtaining ciphertext from
plaintext. The encryption process requires two inputs: the plaintext and the key.
The decryption process also requires two inputs, the ciphertext and the key, in
Key concept order to produce as output the plaintext equivalent of the ciphertext.
Decryption: The processes of encryption and decryption are called the cryptosystem or
Decryption is the process cipher. Thus a cryptosystem is a set of rules for converting between plaintext
of obtaining plaintext from and ciphertext.
ciphertext.
The outer wheel in Figure 6.10.3 is set to map the plaintext letter to its
equivalent ciphertext letter, e.g. letter A maps to letter D for the current setting.
The encryption key is the number on the inner wheel corresponding to the
letter A on the outer wheel. For the current wheel setting it is the number 3.
The dot under the letter A in the outer wheel is to remind the user that the
encryption key is the corresponding number on the inner wheel.
W X
V Y
U A B Z
Y Z
T X 24 25 0 1
C
23 2
A
20 2 V W
D
22
S
B
1
E
4
R
U
C
F G H
5 6 7 I
Q
T
R 18 19
D E
S
P
Q 17
O
16
9
F
10 15 J
K 11 12 13 14 P G
N
O
L M
H N M
I L
J K
F R P S X W H U V F L HQ F H L V F R RO
F R P S X W H U V F L HQ F H L V F R RO
COMPUTER S C I E NC E I S COOL
Questions
1 Convert the following plaintext message using the Caesar
cipher and a key of 7:
Mathematical description
To describe the Caesar cipher mathematically, we represent each letter of the
alphabet by an integer between 0 and 25:
0 for A, 1 for B, …, 25 for Z as shown in Figure 6.10.4.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
C O M P U T E R S C I E N C E I S C O O L
2 14 12 15 20 19 4 17 18 2 8 4 13 2 4 8 18 2 14 14 11
+8
10 22 20 23 28 27 12 25 26 1016 12 21 10 12 16 26 10 22 22 19
10 22 20 23 2 1 12 25 0 1016 12 21 1012 16 0 10 22 22 19
KW U X C B M Z A KQM V KM Q A K W W T
Figure 6.10.5 Encrypting with key 8
To decrypt the ciphertext KWUXCBMZ AHQMVKM QA KWWT with key
8, first convert the plaintext letters to their integer equivalent using Figure
6.10.4. Next subtract the key 8 from each integer. If the resulting integer is less
than 0 then add 26 to convert −1 to 25, −2 to 24, −3 to 23, and so on. Finally
convert each integer to its equivalent letter using Figure 6.10.4 again. The steps
of the process are shown in Figure 6.10.6.
KW U X C B M Z A KQM V KM Q A K W W T
10 22 20 23 2 1 12 25 0 10 1612 21 1012 16 0 10 22 22 19
-8
2 14 12 15 -6 -7 4 17 -8 2 8 4 13 2 4 8 -8 2 14 14 11
2 14 12 15 20 19 4 17 18 2 8 4 13 2 4 8 18 2 14 14 11
C O M P U T E R S C I E N CE I S C O O L
Figure 6.10.6 Decrypting with key 8
The number circle
The number circle shown in Figure 6.10.7 is useful for visualising addition and
subtraction performed in the Caesar cipher.
25 0 1
24 2
23
3
22
4
9 20 21
5
7 6
18 1
8
17
10
16
5 11
12 13 14 1
+6
-6 2
22 24
25 0 1
2 2
23
3
22
22
4
9 20 21
5
7 6
18 1
8
17
9
10
16
5 11
12 13 14 1
Modular arithmetic
Key concept Addition using this number circle is called modulo 26 addition. Likewise
subtraction using this number circle is called modulo 26 subtraction. In
Modular arithmetic:
the world of modulo 26, there are exactly 26 numbers, {0, 1, 2, …, 23, 24,
Addition using the number
circle is called modulo n 25}. These are indicated on the number circle as 0 through 25. The number
addition where n is the 26 is called the modulus. This particular arithmetic is called modular
modulus. Likewise subtraction arithmetic.
using the number circle is called
modulo n subtraction. Questions
4 Evaluate the following using modulo 26 addition:
0
11 1
10 2
9 3
8 4
7 5
6
Congruence
This last example illustrates that adding 7 has the same effect as adding 14,
as adding 21 or adding 0 or subtracting 7 and so on. We call this congruence
(remember congruent triangles from maths lessons, it is a similar idea).
Questions
7 Suppose today is Tuesday.
(a) What day of the week will it be in 492 days?
(b) What day of the week was it 210 days ago?
9 Suppose that you are headed due East, your bearing is 90 degrees.
You turn left 130 degrees, what is your new bearing?
Key concept Two integers are said to be congruent with respect to a given modulus if they
Congruence: differ by a multiple of that modulus. For example, if the modulus is 12 then
Two integers are said to be 2, 14, 26, 36 are congruent. Figure 6.10.10 shows some more congruence for
congruent with respect to a modulus 12.
given modulus if they differ by
a multiple of that modulus.
A statement that two expressions are congruent is called a congruence.
For example, if the modulus
is 12 then 2, 14, 26, 38 are
48
congruent. 36
47 24 37
35 12 25
23 13
38
46 0
34 11 1 26
22 14
10 2
45 33 21 9 3 15 27 39
8 4
20 16
32 7 5 28
44 6 40
19 17
31 18 29
30 41
43
42
Congruence modulo 12
Two integers are congruent modulo 12 if they differ by a multiple of 12.
For example 5 is congruent to 17 (which is 5 + 12) and to
29 (which is 5 + 2 • 12).
7 + 5 ≡ 12 (mod 12)
Representative theorem
Every integer is congruent modulo m to exactly one of the integers
0, 1, 2, 3, …, m −1.
For example, modulo 7: every integer is congruent to exactly one of the integers
0, 1, 2, 3, 4, 5, 6.
Questions
10 What are the modular arithmetic representatives of
(a) modulo 5 (b) modulo 9?
Questions
12 Find the remainder r and quotient q if
(a) b = −37 m = 12 (b) b = −38 m = 24 (c) b = −76 m = 60
(d) b = −576 m = 365
Programming Tasks
1 Write a program to encrypt a line of uppercase text using the Caesar
cipher. Represent each character by a number between 0 and 26
(A ↦ 0, B ↦ 1, …, Z ↦ 25, space ↦ 26).
Use only these characters. Allow a user to choose a key within the
range 0…26.
Encrypt each character of the text using the equation
Questions
13 What is it unnecessary to use a key range wider than 0..26 for the
Caesar cipher in this case?
Task
1 Choose a four letter name and a key. Use the key and the Caesar
cipher to encrypt the name. Now give the ciphertext to another
student and ask them to use a brute force attack to discover the
name and the key used.
%
14
12
RELATIVE FREQUENCY
10
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
Ciphertext
12
RELATIVE FREQUENCY
10
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
Figure 6.10.12 shows what the relative frequency distribution would be if the
Caesar cipher was applied with key 3 to plaintext with relative letter frequency
distribution as shown in Figure 6.10.11. It is relatively easy to see that an E
has been shifted to become an H, therefore the key must be 3. The ciphertext is
said to leak information about the plaintext.
Task
2 Try the Caesar frequency analysis exercise at
https://www.khanacademy.org/computing/computer-science/
cryptography/ciphers/e/
Programming Task
1 Write a program that performs a relative frequency analysis of
English text obtained from a text file. You will need access to text
files of appropriate length. Whilst developing and testing your
program you could use any file on your local machine such as a
ReadMe.txt or you could write your own in a text editor. For more
substantial text files you could download an ebook from http://
www.gutenberg.org.
The NLTK toolkit from www.nltk.org written for Python is a very
powerful text processing resource that could be used for this and
other work.
Questions
14 Why is it unnecessary for this cipher to choose numbers greater
than 26?
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
space
Task
3 Watch the Khan Academy polyalphabetic video and try the tool to
see how a non-uniform plaintext distribution can be flattened as
described in this section.
https://www.khanacademy.org/computing/computer-science/
cryptography/crypt/v/polyalphabetic-cipher
Questions
15 Suppose Alice and Bob communicate messages to each other which
have been encrypted using the Caesar cipher and a previously
agreed secret key. Now suppose that Eve intercepts the ciphertext
and that she happens to know or suspect that Alice starts all
her messages to Bob with the characters “DEAR BOB”. The
corresponding ciphertext is GHDUCERE. Alice and Bob make
no secret of the fact that they use the Caesar cipher believing that
keeping the key secret is sufficient to maintain the security of their
messages. The encryption equation that Alice and Bob use is
ciphertext_character = (plaintext_character + key ) mod 27
Explain how Eve using the plaintext and ciphertext could recover
the secret key used by Alice and Bob.
16 Alice and Bob decide to use a new key for each character, generated
randomly, and apply the stream of random keys using the Caesar
cipher to future messages. Explain why this could offer greater
security over the scheme described in Q15 even though Eve
continues to intercept the ciphertext and Alice continues to start
messages with “DEAR BOB”.
To subject this cipher to closer scrutiny we ask how many ways can a particular
plaintext message consisting of only uppercase letters of the alphabet be
encrypted by a shift cipher which chooses key values randomly? Well consider
that for the first character, there are 26 different possible key values, for the
second character, 26 different possible values again and so on. If plaintext
messages of length 32 characters are encrypted then a key will consist of 32 key
values. The total number of keys of length 32 is therefore
26 x 26 x 26 x …….. x 26 x 26 = 2632 ≈ 2 x 1045
Key principle The total number of possible ciphertexts corresponding to any particular
32 character plaintext message is thus 2 x 1045 one for each possible key
One-time pad: consisting of 32 key values.
Plain text of a message is ‘mixed’
with random text taken from a If each of these possible ciphertexts is written on a separate piece of paper
one-time pad resulting in cipher then the entire stack would be
text which is truly random.
2 x 1045 x 5 x 10-5 metres high = 1 x 1040 metres
The same one-time pad is used
to ‘unmix’ the random text from taking the thickness of a piece of paper to be 5 x 10-5 metres.
the cipher text, which results in
the original plain text. By comparison the Milky Way galaxy is estimated to be 9.5 x 1020 metres
across. If the key values were generated randomly then each ciphertext will
One only has to guarantee
that the one-time pad is be equally likely.
safe, that it comprises truly The decryption cipher is
random numbers, that there
are only two copies of it, and pi = (ci - ki) mod 27
that both copies are destroyed
immediately after use to prevent Likewise a particular ciphertext could have come from any one of 2 x 1045
it being used again (the one- possible plaintext messages. The chance of an attacker guessing which
time property), for it to be used
one correctly is therefore vanishingly small. It is impossible therefore for
to send a message safely without
the risk of being deciphered by an attacker or eavesdropper to break this encryption scheme because the
an attacker or eavesdropper. ciphertext yields no possible information about the plaintext (except its
length).
Background This is the strongest possible method of encryption. It is known as the
The “red phone” used one-time pad because when first used the key was written on a sheet of
in the 1980s for secure paper or pad and used only once.
communication between the
Summarising, the one-time pad method is based on the principle that
USA and the USSR was based
on a one-time pad. The random
the plain text of a message is ‘mixed’ with truly random text taken from
key sequences or pads were a one-time pad. Because the resulting cipher text is still truly random it
delivered by courier. can safely be sent without the risk of being deciphered by an attacker or
eavesdropper.
At the receiving end, the same one-time
pad is used to ‘unmix’ the random text
from the cipher text, which results in
the original plain text. One only has
to guarantee that the one-time pad is
safe, that it comprises truly random
numbers, that there are only two copies
of it, and that both copies are destroyed
immediately after use to prevent it being
used again (the one-time property) on
another plaintext message.
Figure 6.10.14 A one-time pad reproduced with kind
permission of Paul Reuvers, Crypto Museum
(www.cryptomuseum.com)
Task Background
4 Watch the Khan Academy one-time pad video Hardware random number
https://www.khanacademy.org/computing/computer-science/ generators have been built into
some processor systems or made
cryptography/crypt/v/one-time-pad
possible in some operating
system. E.g. Raspberry Pi
5 The Venona project was a counter-intelligence program initiated includes a hardware-based
by the United States Army Signal Intelligence Service (a forerunner random number generator that
of the National Security Agency) that lasted from 1943 to 1980. can generate cryptographic
quality random numbers.
The program attempted to decrypt messages sent by Soviet Union
In Unix-like operating systems
intelligence agencies, including its foreign intelligence service and /dev/random is a special
military intelligence services. The project produced some of the file that serves as a blocking
most important breakthroughs for western counter-intelligence in pseudorandom number
this period, including the discovery of the Cambridge spy ring and generator:
.dev>more –f random
the exposure of Soviet espionage targeting the Manhattan Project.
The NSA declassified the program in 1995. It can be read about in a
NSA document at
https://www.nsa.gov/about/_files/cryptologic_heritage/publications/
coldwar/venona_story.pdf
Key principle
Randomness
Randomness means lack of pattern or predictability of events. Randomness Randomness:
abounds in the physical world and in man-made devices such as electrical Randomness means lack of
pattern or predictability of
circuits as fluctuations of an unpredictable nature which we call noise.
events.
Electrical storms, the microwave background left over from the Big Bang and
other events induce random currents of electricity in aerials connected to
radio receivers and televisions that cause the hissing noise that we hear in their
loudspeakers. This atmospheric noise can be captured, sampled and digitised to
provide a source of truly random bits. Such a service is provided by
https://www.random.org. Background
Another source is https://www.fourmilab.ch/hotbits/secure_generate.html RandomX package for Java has
which uses the unpredictable nature of radioactive decay to generate truly an option to get random bits
random bits. from hotbits.
http://www.fourmilab.ch/
A random number is one that is drawn from a set of possible values, each hotbits/source/randomX/
of which is equally probable, i.e., a uniform distribution, e.g. the throw of a randomX.html
six-sided die. When discussing a sequence of random numbers, each number
drawn must be statistically independent of the others, i.e. knowledge of an
arbitrarily long sequence of numbers is of no use whatsoever in predicting the
next number to be generated. Each possible arbitrarily long sequence is thus
equally likely.
Pseudorandom
Pseudorandomness is an important concept in cryptography.
Informally pseudorandom means:
cannot be distinguished from uniform i.e. random.
The cryptographic definition of pseudorandom however is
a distribution is pseudorandom if it passes all efficient statistical tests.
This definition has been arrived at by considering the need to resist an attack
from an adversary who is trying to obtain information from ciphertexts about
the corresponding plaintext messages.
Generating large numbers of truly random numbers is extremely difficult
Key point
so people have turned to the computer and algorithms programmed into
Pseudorandom numbers: the computer to generate pseudorandom numbers from an initial seed.
Pseudorandom numbers are These pseudorandom numbers are generated deterministically and can only
generated deterministically
approximate a truly random distribution because numbers calculated by a
and can only approximate
a truly random distribution computer through a deterministic process, cannot, by definition, be random.
because numbers calculated Given knowledge of the algorithm used to create the numbers and the seed,
by a computer through a it is possible to predict all the numbers returned by subsequent calls to the
deterministic process, cannot,
algorithm, whereas with genuinely random numbers, knowledge of one
by definition, be random.
number or an arbitrarily long sequence of numbers is of no use whatsoever in
predicting the next number to be generated. Therefore, computer-generated
“random” numbers are more properly referred to as pseudorandom numbers,
and pseudorandom sequences of such numbers. Pseudorandom generated
sequences eventually repeat with the periodicity determined by the seed and the
algorithm used. Pseudorandom generated sequences are also reproducible, i.e.
for a given algorithm, starting from the same seed generates the same sequence.
Pseudorandom number generators (PRNGs)
Key point A pseudorandom number generator is an efficient, deterministic algorithm
Pseudorandom number that expands a short, uniform seed into a longer, pseudorandom output in
generators: polynomial time. It is useful whenever
A pseudorandom number
1. It would be difficult to communicate a long sequence of numbers
generator is an efficient,
deterministic algorithm that needed in a symmetric key cipher instead the seed is communicated
expands a short, uniform seed 2. A large number of random numbers are required and access to truly
into a longer, pseudorandom
random numbers is restricted to a much smaller number
output in polynomial time.
The seed may be chosen from The seed may be chosen from the small number of truly random numbers that
the small number of truly are available but doesn’t have to be.
random numbers that are
available but doesn’t have to be.
SEED
OUTPUT
Figure 6.10.15 A pseudorandom number generator, G, producing a longer
stream of “random” bits from a shorter length seed of true random bits Background
Care must be taken when relying on pseudorandom number generators Security experts have long
for cryptographic purposes because they are deterministic. However, a class suspected the National Security
of improved random number generators, termed cryptographically secure Agency (NSA) has been
introducing weaknesses into
pseudorandom number generators (CSPRNG) exist that rely on truly random
CSPRNG standard 800-90 that
seeds external to the software.
they can exploit in ciphers that
use this standard; this being
confirmed for the first time by
one of the top secret documents
leaked to the Guardian by
Edward Snowden.
Task
6 Download the test program at http://www.fourmilab.ch/random/
Use it to test sequences of bytes/bits for their randomness. Run
this program on data generated by a high-quality pseudorandom
sequence generator. You should find it generates data that are
indistinguishable from a sequence of bytes chosen at random.
Indistinguishable, but not genuinely random.
Background
7 Watch the video on pseudorandom number generators at https://
The Vernam cipher was www.khanacademy.org/computing/computer-science/cryptography/
exploited in the design of a
crypt/v/random-vs-pseudorandom-number-generators
high security teleprinter cipher
machine that the Lorenz
company made for the German
Army High Command to The Vernam cipher
enable them to communicate The Vernam Cipher is named after Gilbert Sandford Vernam (1890-1960)
by radio in complete secrecy
who, in 1917, invented the stream cipher and later co-invented the one-time
during WW2.
These transmissions were pad (OTP). His patent US1310719 was filed in 1918 and is, according to the
broken by Bill Tutte. The National Security Agency (NSA), perhaps one of the most important in the
process of decrypting Lorentz history of cryptography.
machine ciphertexts was later
automated using a refinement
At the time of the invention, Vernam was working at AT
suggested by Max Newman & T Bell Labs in the USA. Messages were then sent by
and some clever engineering by telegraph, a system that used pulses of electrical current
Tommy Flowers who designed to encode characters according to the Baudot code. The
and built Colussus, the world’s
characters were entered into and read from the system
first stored program computer
to decrypt Lorentz encrypted using a teleprinter.
messages. Colussus reduced Vernam proposed a teleprinter cipher in which a
the time taken from weeks to
previously prepared key, kept on paper tape, was combined Figure 6.10.17
hours. Colussus came online
just in time to decrypt messages character by character with the plaintext message to Gilbert Vernam
which gave vital information to produce the ciphertext. To decrypt the ciphertext, the
Eisenhower and Montgomery same key would be again combined character by character, producing the
prior to D-Day - plaintext.
http://www.codesandciphers.
org.uk/lorenz/colossus.htm
Working with Joseph Mauborgne, at that time a captain in the US Army
Signal Corps, they proposed that the paper tape key should contain random
information (the key stream). The incorporation of this proposal into Vernam’s
machine implemented an automatic form of the one-time pad.
Task
8 Watch AT & T Labs’ video of the Vernam cipher http://techchannel.att.com/play-video.
cfm/2009/10/12/From-the-Labs:-Encryption1
9 Read about the details of the Vernam cipher machine by visiting the website
http://www.cryptomuseum.com/crypto/vernam.htm
The Vernam cipher relies on the bit-wise eXclusive-OR (XOR) Boolean Key principle
function. This is symbolised by ⊕ and is represented by the following truth
table, Table 6.10.3, where 1 represents true and 0 represents false Vernam cipher:
Encrypts and decrypts a
INPUT OUTPUT message using a one-time pad
A B A⊕B approach in which the plaintext
0 0 0 message M = {0, 1}n is bitwise
0 1 1 XORed with a uniformly
1 1 1 random key k {0,1}n where n is
the number of bits to encrypt
1 0 0
the message.
Table 6.10.3 Exclusive-Or truth table
A very useful property of the exclusive-or operation is that it is possible to Background
recover an input given the output and the other input. For example, if the
The eXclusive-OR function is
inputs A and B are 0 and 1 respectively, then A ⊕ B = 1. If we exclusive-or this an example of an involution.
output 1 with, say, input B which was 1, 1 ⊕ 1 = 0, we recover input A which An involution is a function
was 0. It works for all inputs.
f:X →X
Therefore the same key stream can be used both to encrypt plaintext to
That, when applied twice,
ciphertext and to decrypt ciphertext to yield the original plaintext:
brings one back to the starting
Plaintext ⊕ Key = Ciphertext point
and: f(f(x)) = x
Ciphertext ⊕ Key = Plaintext
If the key stream is truly random, and used only once, this is effectively a
one-time pad. Information
Visit the cryptomuseum at
0
http://www.cryptomuseum.
0
com/crypto/vernam.htm
1
1
1
0
0
0
0
1
1 0 1 1 0 0 1 0 1 1 0 1 1 0 1 1
Encrypt Decrypt
plaintext message M
1
Programming task
3 RC4 is a stream cipher used in WEP. The infinite pseudorandom key stream for RC4 is generated from
a secret key using the following two algorithms. Code these in your preferred language and run the test
below on secret key = AQACS and plaintext = Computer Science
Key-scheduling algorithm
for i from 0 to 255
S[i] := i
endfor
j := 0
for i from 0 to 255
j := (j + S[i] + key[i mod keylength]) mod 256
swap values of S[i] and S[j]
endfor
Task
11 Watch the video on Perfect Secrecy from Khan Academy -
www.khanacademy.org/computing/computer-science/cryptography/crypt/v/perfect-secrecy
Key point
Unconditional or Perfect Security (Perfect secrecy)
Regardless of any prior information the attacker has about the plaintext, the ciphertext leaks no additional
information about the plaintext in a ciphertext-only attack.
Figure 6.10.19(a) Claude The original image can only be recovered by using the same key, i.e. the
Elwood Shannon was an exact sequence of randomly generated bits that produced the ciphertext.
American mathematician,
electronic engineer, and Applying the “wrong” key will result in recovering a different image.
cryptographer famous for
having founded information
theory with a landmark paper
that he published in 1948.
Task
12 Try the reversible XOR demonstration at
https://www.khanacademy.org/computer-programming/reversible-
xor-demo/5580322717564928
Key point
VERY IMPORTANTLY by using XOR and choosing the key randomly, the XOR
operator has a 50% chance of outputting a 0 or a 1.
cn = kn ⊕ pn
The previous section argued that each bit ci of ciphertext tells Eve nothing
about the corresponding bit pi of plaintext and nothing about any of the other
bits of plaintext. From this we can draw the conclusion that the cryptosystem
has perfect secrecy or security.
Questions
17 A 3-symbol message, AQA is encrypted as follows. Each symbol is represented by a number between
0 and 26 (A ↦ 0, B ↦ 1, …, Z ↦ 25, space ↦ 26). Each number is represented by a five-bit binary
sequence (0 ↦ 00000, 1 ↦ 00001, …, 26 ↦ 11010). Finally, the resulting sequence of 15 bits is
encrypted using the key consisting of 15 randomly chosen bits 110000000101110 (obtained from
random.org) and modulo 2 addition.
Computational security
Limitations of the one-time pad
The success of the one-time pad is that it achieves perfect secrecy. However the
following limitations have prevented more widespread use:
■■ The key is as long as the message
■■ Only secure if each key is used to encrypt a single message (i.e. key
Information
not used more than once) The “red phone” used
in the 1980s for secure
This means that the parties wishing to communicate in secret, e.g. Washington,
communication between the
DC and Moscow, Russia via the “red phone” must share keys of total length USA and the USSR was based
equal to the total length of all messages that they might ever send. on a one-time pad. The random
key sequences or pads were
If the same key k is used twice, e.g
delivered by courier.
c1 = k ⊕ m 1
c2 = k ⊕ m2
the attacker can compute
c1 ⊕ c2 = (k ⊕ m1) ⊕ (k ⊕ m2) = m1 ⊕ m2
This leaks information about m1 and m2 because it reveals where these differ:
the characteristics of the ASCII coding scheme can be exploited to identify
some letters and frequency analysis can be brought to bear as well.
Questions
18 Study the ASCII code table and note that letters all begin with 01 and the space character begins with
00. Also note that XOR of two letters gives 00… and XOR of a letter and a space gives 01… It is easy to
identify XOR of letter and space. If the identified XORed letters and spaces are XORed with space’s ASCII
code, the plaintext letter is recovered. The following two ciphertexts were intercepted. It is suspected that
the same random key has been used to produce these. Assuming 8-bit ASCII, can you recover any letters
of the plaintext messages?
c1 = 010100001010110011001000111110111101000011101110
c2 = 010000011010010011011101100100111101110011110011
(The key that was used so that you can check your answer:
000100011111110110001001110110111001001110111101)
Computational secrecy
In practice it is more convenient to allow the leak of information with a tiny
probability to eavesdroppers with bounded computational resources, i.e. not
unlimited. This means relaxing perfect secrecy by
Key principle ■■ Allowing security to fail with tiny probability
Computational security: The meaning of “efficient” attackers is attackers who can try 2112 keys.
An encryption method is
Well, if we choose a key space of 2128, i.e. keys of length 128 bits then we
computational secure if it
safe to assume that no known
should meet the requirement for secrecy.
attack can break it in a practical This kind of secrecy is called computational secrecy. It relies on allowing
amount of time.
1. security to fail but with a probability negligible in n where n is a
measure of the challenge of breaking the system, e.g. factoring a
given integer n
2. restricting attention to attackers running in time polynomial in n
The notion of computational secrecy leads to the classification of an encryption
method as being computational secure if it safe to assume that no known
attack can break it in a practical amount of time.
However, this is very different from a proof of security. Thus in theory, every
Information cryptographic algorithm except for the Vernam cipher (one-time pad) can be
IBM Quantum computer: broken, given enough ciphertext and time.
IBM makes quantum computer
available to members of the Task
public, 4th May 2016 -
www-03.ibm.com/press/us/en/ 13 Research why quantum computing might be a threat to ciphers
pressrelease/49661.wss that rely on computational security and not information-theoretical
See security.
www.research.ibm.com/
quantum/
Task
14 Look at the Kryptos transcript available from
www.elonka.com/kryptos/. Can you decrypt the four ciphertexts?
Don’t worry if you can’t –
see www.wired.com/2013/07/nsa-cracked-kryptos-before-cia
Key concept It needs to be loaded into main memory and it needs to obtain input from
input devices such as keyboards and to write output to output devices such as
System software:
printers and it may need to communicate with other computers.
A layer or layers of software
which enables users to operate Application software may need to store information permanently and to
the computer without having subsequently access stored information. The stored information should be
to be familiar with its internal
backed up so, if necessary, it may be restored from a back-up copy. These
workings.
services are provided by the operating system and utility software without
which it would not be possible to run application software.
Key concept Application software may be classified as
Questions
4 Describe the classification of application software?
Machine code is a language that the hardware of a computer can of which is intermediate-code
generation. It is a much simpler
understand and execute. It consists of executable binary codes.
task to write an interpreter for
Assembler: an assembler translates a program written in assembly a new machine designed with
language into machine code. Assembly language is a symbolic form a different instruction set than
it is to write a compiler. Any
of machine code. The symbolic form consists of mnemonics such as
program in intermediate-code
ADD and SUB that denote the machine operation to be performed. form, including a compiler, can
An assembler simply substitutes the corresponding executable binary be “executed” by interpreting its
code for the mnemonics. intermediate-code form with the
interpreter written for the new
Interpreter: translates and executes a high-level language or machine.
intermediate-code program one statement at a time. It provides Examples of intermediate-code
a way of executing programs not in the machine code of the are p-code and bytecode.
computer.
Questions
6 What are the functions of each of the following software:
(a) operating systems (b) utility programs (c) libraries (d) translators?
Assembly language
Writing programs directly in machine code is challenging. The EDSAC programmers wrote their programs using
letters for the operation to be performed and addresses in decimal using the digit characters '0'..'9'.
The hardware on which they typed these letters and digit characters was wired to punch paper tape with the 5-bit
equivalent of each.
We would call the form of the program shown in Figure 6.2.1.1 which uses letters, an assembly language
program. In assembly language, a (symbolic) name is assigned to each operation/instruction code. The operation/
instruction code name is called a mnemonic or memory jogger. The operation code mnemonic should describe in
some way what the instruction does, e.g. LDR means LoaD a Register, ADD means add - see Table 6.2.1.2. The
address field &1234 is expressed in hexadecimal (& is used to indicate this).
Assembly language Description
LDR means LoaD a Register with content of a
memory location or word, Rd is the symbolic name
LDR Rd, &1234
for the register, &1234 is the memory location’s
address expressed in hexadecimal.
ADD means add content of registers Rn and Rm,
ADD Rd, Rn, Rm
store result in register Rd.
STR means STore the content of the specified
STR Rd, &4321
Register in a memory location or word.
Table 6.2.1.2 Some assembly language instructions
221 Single licence - Abingdon School
6.2.1 Classification of programming languages
much closer to the problem space, leaving the task of translating to machine
code to the computer? This thought led to the development in the 1950s of Key concept
high-level languages, some of which are still used. For example, Fortran (1957) High-level programming
was designed for numerical applications and is still used by mathematicians, language:
scientists and engineers, today. High-level programming
languages are problem-oriented
High-level languages are closer to English than they are to the machine. This and therefore closer to English
means that the mapping from a high-level language statement to machine code than they are to the machine.
will be a one-to-many mapping because each high-level language statement This means that the mapping
from a high-level language
will need to be broken down into several machine code operations. For
statement to machine code will
example, the assignment statement be a one-to-many mapping
x ← y + z because each high-level
when translated could become in the assembly language form of machine code language statement will need
to be broken down into several
LDR R0, &1234 machine code operations.
LDR R1, &1235
ADD R2, R0, R1
Information
High-level language
STR R2, &1236 classification:
Imperative:
Questions • Procedural
• Object-oriented
7 What is meant by the term high-level programming language?
Declarative:
8 What is the mapping between high-level language statements and • Logic
machine code? • Functional
Imperative
Description
program
y := 6; assign 6 to y
z := 7; assign 7 to z
x := y + z; add z to y and store result in x
Questions
9 Explain the term imperative high-level language.
Assembly language and machine code programming allow direct access Key fact
to registers and low-level operating system routines which is not generally
Adv. of programming in
possible with most high-level language programming languages.
machine code and assembly
language:
Questions
Hand-coded assembly language
when assembled can
10 State three advantages of programming in assembly language compared
• achieve a smaller memory
with programming in a high-level language.
footprint in machine code
than compiled high-level
language code
Disadvantages of programming in machine code and assembly • achieve better code
language compared with HLL programming optimisation than compiled
Code written in assembly language or machine code is less readable than code high-level language code and
written in a high-level language and therefore more difficult to understand therefore code that will run
and maintain, debug and write without making errors. Code written in faster
• directly access registers
assembly language or machine code uses the instruction set of a particular
and low-level operating
processor (processor family). It is therefore machine dependent and will system routines which is not
only execute on processors that use this instruction set. High-level languages possible with most high-level
are machine independent. An HLL program is expressed in an English- programming languages.
■■ Classification of programming languages into low- and high-level Code written in assembly
languages language or machine code is
machine dependent making it
■■ Low-level languages classified as: difficult to port to a different
• machine code instruction set processor
compared with code written
• assembly language
using high-level languages which
■■ Imperative high-level language is a type of high-level language do port readily because they are
■■ Machine-code language and assembly language not machine-oriented.
Key principle A compiler only translates a high level language program (the whole of the
program), it does not execute it.
Compiler:
A compiler translates a high The process that the compiler engages in is called compiling.
level programming language
source code program into a
A compiler consists of several stages:
separate and independently • Lexical analysis – splits the source into user-defined “words”,
executable object code target
e.g. variable identifiers and language-defined “words”, e.g. While
language program. Object code
is typically machine code. • Syntax analysis – checks that statements are grammatically correct
• Semantic analysis – e.g. type checking, "A" + 3.142 is incorrect as you
A compiler translates one high
level language statement into can’t add a real to a string
several machine code or target • Intermediate code generation
language statements.
• Code optimising
• Code generation
Key principle Role of an interpreter
Interpreter: An interpreter is a program that executes a high level programming language
An interpreter is a program
program, statement by statement, by recognising the statement type of a
that executes a high level
programming language statement, e.g. X = X + 1, and then calling a pre-written procedure/function
program, statement by for the statement type, to execute the statement. Therefore, an interpreter does
statement, by recognising the not, unlike a compiler, produce an independently executable target language
statement type of a statement equivalent of the source language program. The application of interpreter to a
and then calling a pre-written
source code program is called interpreting.
procedure/function for the
statement type, to execute the The differences between compilation and interpretation
statement. The major differences between the compilation and interpretation are:
• An interpreter both “translates” and executes whereas a compiler only
translates.
• A compiler produces a separate independently executable form of the
source code program whereas an interpreter does not.
Key principle
• A compiler is not needed when target form of source program is
Interpreter vs compiler:
executed whereas in the case of the interpreter, execution requires the
An interpreter both “translates”
and executes whereas a source code form of the program together with the interpreter, i.e.
compiler only translates. the interpreter needs to be available on the machine where the program
is being run.
• If an interpreter is used then only the source code form of program is
Key principle needed to execute the program whereas, if a compiler is used then the
Interpreter vs compiler: object code form of program is needed in order to execute the program.
A compiler produces a separate • Interpreters are usually easier to write than compilers.
independently executable form
of the source code program • With the compiler approach, if an error is discovered while the
whereas an interpreter does not. program is executing the source form of program must be located. An
editor and the source form of the program must be loaded. The error
must be pin-pointed which is not always easy and then corrected.
The compiler must be loaded and a compilation carried out. The new
target form of program must then be loaded and executed. With an
interpreter, the execution is halted at the point where the error occurs.
The interpreter gives precise details of location of error. The error is
corrected with an editor which may be co-located with interpreter. If
it isn’t, an editor will have to be loaded. However, no time-consuming
compilation is involved and execution can resume immediately.
Situations in which assemblers, compilers and interpreters
would be appropriate
Key principle
Assemblers
Interpreter vs compiler:
For time-critical sections of code where execution speed is important, e.g.
Where speed of execution and/
interrupt service routines, assembly language still has a role to play because in or direct access to hardware is
the hands of a skilled programmer, assembly language code can be written that required, use assembly language
is highly optimised for speed. As an assembler simply translates one assembly and an assembler.
language statement into one machine code statement, that optimisation is
preserved. Compilers can optimise code but the binaries produced cannot be
guaranteed to be fully optimised for the given hardware. In the pecking order
of speed, interpreters come after compilers.
Key principle
Interpreter vs compiler:
Assembly language is still used where direct access to hardware is required e.g. Compiled code which has been
processor registers or I/O controller registers. This is the case when writing compiled into machine code of
device drivers, e.g. a screen driver. In this instance an assembler would be the computer will execute a lot
required to translate the assembly language program into machine code. faster than its interpreted source
code equivalent (i.e. interpreter
Compilers and interpreters + the source code equivalent of
It is considerably more productive to write programs in high-level languages the compiled code).
than in assembly language. There are relatively few programmers who are
skilled in writing assembly language programs compared with the number
of programmers skilled in writing in one or more high-level programming
languages. Key principle
Interpreter vs compiler:
Compiled code which has been compiled into machine code of the computer
Where rapid debugging and
will execute a lot faster than its interpreted source code equivalent (i.e. immediate feedback on errors is
interpreter + the source code equivalent of the compiled code). required including pinpointing
the location of both syntax
The immediate feedback and ease of locating errors in source code give
and runtime errors, use an
interpreters an advantage over compilers when developing programs. This interpreter.
advantage is particularly beneficial for novice programmers or when programs
are being prototyped and the write, compile, debug, edit cycle can be too time
consuming. Key principle
Compiling has an advantage over interpreting because it produces a separate Interpreter vs compiler:
Where a separate executable
executable which means that the source code program does not have to be
that can execute independently
distributed. There are plenty of situations where this is desirable such as when of its source code equivalent is
producing commercial software or where there is a requirement is to protect the required, use a compiler.
algorithm or coding technique used.
multiply /multiply the top two items on the stack,replace with result
add /add the top two items on the stack,replace with result
pop x /remove top item from stack and store in local variable
The stack operations push and pop are covered in Chapter 2.3.1 of the
Unit 1 textbook and evaluating expressions using a stack covered in
Chapter 3.3.1 of the same textbook. A compiler for this simple stack
machine would output byte-long numeric codes, called opcodes, for the
operations push, pop, multiply and add. These opcodes are known as
bytecodes because they are one byte long and they form the instruction
set of the stack machine.
The bytecode stream issued by the compiler for this example might be as
follows 1a 1b 68 1c 60 3b. This bytestream example uses bytecodes
for the Java Virtual Machine which is a stack-based machine that is
able to interpret Java bytecodes. Table 6.3.1.1 shows the corresponding
interpretation of these bytecodes.
Bytecode Operation
1a push first local variable onto stack
1b push second local variable onto stack
68 pop top two items on stack, multiply them together, push result on stack
1c push third local variable onto stack
60 pop top two items on stack, add them together, push result on stack
3b pop top item on stack, store in first local variable
Table 6.3.1.1 Java bytecodes and their interpretation
To execute bytecode on a virtual machine requires that it be interpreted by the underlying real machine, i.e. the
bytecode is executed in software running on the underlying real machine. This software is called an interpreter.
Writing a software interpreter to interpret bytecode is an easier task than writing an interpreter to interpret high-
level language source code. All that a bytecode interpeter has to do is parse (identify) and directly execute the
bytecodes, one at a time. This also makes the bytecode interpreter very portable, i.e. very easy to move onto a new
machine with a different instruction set, and very compact.
Interpreting bytecode programs is also much faster than interpreting their high-level language source code program
equivalents because the interpreter written to interpret bytecode has to perform much less work and is therefore
simpler.
Bytecode targets a virtual machine not a real machine and so can run on any machine or operating system for
which a bytecode interpreter has been written.
This means that the same object code can run on different platforms by simply creating an interpreter for the
platform. A compiler that outputs bytecode thus produces object code that is portable.
However, bytecode may be further compiled into machine code for better performance. Some systems, called
dynamic translators, or “just-in-time” (JIT) compilers, translate bytecode into machine language as necessary at
runtime.
Questions
1 Explain the role of each of the following:
(a) assembler
(b) compiler
(c) interpreter
2 State three differences between compilation and interpretation.
3 (a) Give two reasons why programs are still written in assembly language
(b) What is the relationship between
(i) assembly language statement and machine code
(ii) high level programming language statement and machine
code?
5 A particular computer has two compilers for a high level language HLL. The compilers are called
HLL1 and HLL2. HLL1 compiles a program written in HLL into the machine code of this
computer, whereas HLL2 compiles an HLL program into intermediate code, which can then be
executed by an interpreter running on this computer, if one exists.
On purchase, compiler HLL2 was supplied in intermediate code form without an interpreter, the
same intermediate code that is produced by HLL2, and HLL1 in source code program form.
(a) With only a means to write assembly language programs and to run an assembler on the
computer at this stage, explain carefully what could be done to enable HLL2 to compile
HLL programs for this computer.
(b) Explain carefully how HLL1 can now be executed.
(c) Explain carefully how the machine code form of HLL1 can now be produced on this
computer.
X Meaning
0 It is not sunny today
1 It is sunny today
Table 6.4.1.3 Boolean variable representation of truth statements using
0 in place of False and 1 in place of True
It is then a small step to use Boolean variables to represent the state of
components such as switches and indicator lamps as follows:
• a switch can be either closed (1) or open (0) and
• an indicator lamp can be either on (1) or off (0).
Y Meaning
0 Switch is not closed
1 Switch is closed
Table 6.4.1.4 Boolean variable representation for state of a switch Y
Z Meaning
0 Lamp is not on
1 Lamp is on
Table 6.4.1.5 Boolean variable representation for state of an indicator
lamp Z
Logical OR operation
Things become interesting when switches and lamps are combined together in
circuits. Figure 6.4.1.1 shows a simple circuit consisting of two switches wired
Switch in parallel, one indicator lamp and one battery.
X Y Q X Y Q
X Q Open Open Off 0 0 0
Open Closed On 0 1 1
Lamp 1 0 1
Closed Open On
Y Closed Closed On 1 1 1
Battery
Figure 6.4.1.1 OR logical operation: switch arrangement,
switch state combinations and corresponding lamp state
The lamp is on if switch X is closed OR if switch Y is closed OR if both are
closed, otherwise the lamp is off. The state of the switches can be expressed in
the two Boolean variables, X and Y, as open or closed or using 0 for open and
1 for closed. The state of the lamp can also be expressed in a Boolean variable,
Q, because the state has two possible values, off or on, which can be coded as 0
and 1, respectively.
Just as we can write the number equation for the total number of goats G, a
farmer possesses,
G=X+Y
where X is the number in the first goat pen and Y is the number in the second,
X
so we can write for the lamp circuit the Boolean equation
Y OR Q
Q=X+Y
Figure 6.4.1.2 Logical OR
The operator "+" denotes the logical OR operation that behaves according to operation: block diagram
the tables in Figure 6.4.1.1, e.g. if X = 1 and Y = 1 then Q = 1, i.e. the lamp is showing inputs X and Y and
on. output Q
Logical OR Truth Table
If the logical operator "+" is represented by a rectangle labelled OR (Figure X Y Q
6.4.1.2) then Boolean variables X and Y become its inputs and Q becomes its 0 0 0
output. The inputs X and Y are transformed by the logical OR operation into 0 1 1
1 0 1
Q.
1 1 1
In fact, the logical OR operation defines a Boolean function OR because it
Figure 6.4.1.3 Logical OR
operates on binary inputs and returns a single binary output (Figure 6.4.1.4).
truth table
Figure 6.4.1.2 is called a block diagram. A single block in a block diagram is
sometimes called a black box even though it is not coloured black. Logical OR
The black box approach is a convenient way of representing the logical OR Inputs: X, Y
operation with the details of how it is implemented abstracted away. We now Output: Q
define the logical OR operation by its truth table (Figure 6.4.1.3) not by the Function: OR = X + Y
particular details of its implementation which could be, for example, electronic, Figure 6.4.1.4 Logical OR
magnetic, optical, biological, hydraulic, or pneumatic. function
X Y Q The operator "." denotes the logical AND operation that acts according to the
0 0 0 truth table in Figure 6.4.1.6, e.g. if X = 1 and Y = 1 then Q = 1, i.e. the lamp is
0 1 0 on.
1 0 0 Figure 6.4.1.7 shows the block diagram representation of the logical AND
1 1 1
operation with inputs X and Y transformed into output Q.
Figure 6.4.1.6 AND truth
table
Questions
1 Draw the arrangement of switches that produce output Q
X
Y AND Q where Q = X.Y + X.Z
It has its own symbol ⊕ so the Boolean equation is written as follows Figure 6.4.1.14 XOR truth
table
Q = X ⊕Y
X X Y
AND Key point
Y NOT Y
OR Q Logic gate:
A logic gate is a physical device
NOT X
that implements a Boolean
AND function.
X Y
Figure 6.4.1.15 XOR logical operation: block diagram constructed from
NOTs, ANDs and an OR
Questions X Y Z Q
0 0 0
2 Q is only 1 if both X and Y are 1 and Z is 0 or if both Y and Z are 1 and X is 0
0 0 1
or if X, Y and Z are 1. Complete the truth table.
0 1 0
3 Draw the block diagram using AND and OR operations that produces output Q 0 1 1
where Q=X.Y+X.Z 1 0 0
1 0 1
4 Draw the block diagram for logical operations that produce output Q where 1 1 0
Q=X.Y+X.Z 1 1 1
Questions A
B E
5 What is the output of this logic gate circuit
when its input is (a) 0 (b) 1? Q
C F
6 What is the output of this logic gate circuit when its D
input is (a) 0 (b) 1? Figure 6.4.1.16 Three gates
connected to perform a logical
operation
Questions
C
7 What is the output, Q, of this logic gate circuit when its A
inputs A and B are (a) both 0 (b) both 1 Q
(c) different from each other?
B D
8 What is the output of this logic circuit when A1 A0
(a) A0 = B0 and A1 = B1?
(b) A0 ≠ B0 and A1 ≠ B1?
(c) A0 ≠ B0 and A1 = B1? Q
(d) A0 = B0 and A1 ≠ B1?
B1 B0
Questions A B C D Q C
0 0 A
10 Complete the truth table for this logic gate Q
0 1
circuit using Boolean variables A, B, C, D
1 0 B
and Q. 1 1
D
11 Draw the truth table for this logic gate circuit using A
Boolean variables A, B and Q. B
Q
12 Draw the truth table for this logic gate circuit using
Boolean variables
A
A, B, C, D and Q. B
(Note: the table will
have 16 rows for the Q
values of
the Boolean variables). C
D
Questions A 0 Q
A 0 0
13 Complete the truth table for this logic circuit. Q
0 1 0
A A 1 Q
14 Complete the truth table for this logic circuit. 1
Q
0 1
1 1
15 Complete the truth table for this logic gate circuit.
A B D
A B D C3 C2 C1 C0
C3 0 0 1 See Figure 6.4.1.32 for
0 1 1 an example of how this
1 0 1 circuit could be used.
C2
1 1 1
0 0 0
C1 0 1 0
1 0 0
1 1 0
C0
B D
Q=C+D
Figure 6.4.1.19 Three gates
connected to perform a logical therefore Q=A+B
operation
A + B is the Boolean expression equivalent of the logic gate circuit shown in
Figure 6.4.1.19. If we examine Figure 6.4.1.6 carefully we see that the output
from this logic gate circuit for inputs A and B is that of the truth table for an
AND logic gate. Therefore, another equivalent Boolean expression is A . B.
Thus for the two Boolean expressions
A . B is equivalent to A + B
This means that we could replace the logic circuit in Figure 6.4.1.19 by an
AND gate with inputs A and B.
A.B
To convert this into an equivalent logic gate circuit we must take each term in
the expression, starting with the innermost, and apply each operation in turn.
The innermost terms are
A and B
applying the NOT operation to each
A and B
turning these into equivalent logic gates
A A B B
A A
A B
B B
A A
A B
A B
B B
The number of gates can be reduced to three by replacing the AND-NOT combination by its logic gate equivalent
NAND. The logic gate circuit becomes
A A
A B
B B
For more complicated Boolean expressions the same approach is used to arrive at the equivalent logic gate circuit,
e.g.
A.B + A.B
(A + B) . (A + B)
(A + B) is converted to its equivalent logic gate circuit W
Questions
A
16 Write an equivalent Boolean expression in terms of B
A, B, C and D for this logic gate circuit. Q
C
17 Write an equivalent Boolean expression in D
terms of A, B for this logic gate circuit.
A
B
Q
Half-adder
To make a system to add together two binary digits A and B to produce a Sum
and a Carry digit requires the following understanding of binary arithmetic
Questions
18 Trace the operation of the half-adder logic gate circuit shown in
Figure 6.4.1.20 for inputs A = 1 and B = 1.
Full-adder
0 0 0 1 Carry 1 1 1 1
A pair of binary numbers, A and B, can be added
0 1 0 1 A 0 1 1 1
digit by digit from right to left, by the simple rules
+ 1 0 0 1 B + 1 0 1 1
of arithmetic. First, the two right-most digits, also
0 1 1 1 0 A+B 1 0 0 1 0
called the least significant bits (LSB) of the two binary
numbers, are added. no overflow overflow
Figure 6.4.1.23 Adding two binary numbers with carry,
Next, the resulting carry bit (which is either 0 or 1)
one case with overflow and one without
is added to the sum of the next pair of bits up the
significance ladder. The process is continued until Inputs Outputs
the two most significant bits (MSB) are added. If the A B C Carry Sum
last bit-wise addition generates a carry of 1, overflow 0 0 0 0 0
0 1 0 0 1
has occurred, otherwise, the addition completes
1 0 0 0 1
successfully. Figure 6.4.1.23 shows two different cases,
1 1 0 1 0
one with overflow and one without.
0 0 1 0 1
Computer hardware for binary addition of two n-bit 0 1 1 1 0
numbers can be built from logic gates designed to 1 0 1 1 0
calculate the sum of three bits (pair of bits plus carry 1 1 1 1 1
C = 0/1
A=1
0 0
B=1
0
1 Carry
Carry
2nd HALF-ADDER
1st HALF-ADDER
C=1
A =1 B = 0
Figure 6.4.1.26 Conditions that generate
A =0 B = 1 1
carry = 1: two in the upper diagram, Sum = 1
two in the lower Carry = 0 1
1st HALF-ADDER Carry
2nd HALF-ADDER
Note that the first half-adder carry output is the inverse of the second half-
adder output: 1/0 and 0/1 respectively. Therefore if the two carry outs, one
from the first half-adder and one from the second are made the first and second
inputs of an OR-gate then its output will be the final carry. Figure 6.4.1.27
shows the final circuit. C becomes the CARRY IN from the previous digit
column. A and B are the current digit column’s bits to be added and CARRY
OUT is the final carry from the current column.
CARRY IN
A SUM
B
PARTIAL SUM 2nd HALF-ADDER
Questions
19 Trace the operation of the full-adder logic gate circuit shown in Figure 6.4.1.27 for inputs A = 1 and
B = 1 and CARRY IN = 1.
The logic gates and logic gate circuits that we have considered so far are known
as combinational logic circuits. Combinational logic circuits compute functions
that depend solely on combinations of input values. Combinational logic
circuits are used in the construction of the Arithmetic and Logic Unit (ALU)
which lies at the heart of the central processing unit (CPU) – see Chapter
7.3.1.
Edge-triggered D-type flip-flop Key fact
Although combinational logic circuits provide many important processing The D-type flip-flop is the building
functions, they cannot maintain state. In addition to computing values such as block for all the hardware devices
5 + 6, computers must also be able to store and recall values, i.e. they must be that computers use to maintain
state, from single-bit cells to reg-
equipped with memory elements that preserve data over time. These memory
isters to arbitrarily large random
elements are built from sequential logic gate circuits called flip-flops.
access memory (RAM) units.
Flip-flops are the elementary building blocks of all memory devices used in
typical modern computers.
A flip-flop sequential logic circuit is one in which data is captured and
“committed to memory” at a specific moment in time. Time in a computer is
provided by a master clock that delivers a continuous train of alternating binary
signals. The master clock is an oscillator that alternates between two phases
labelled 0 and 1, or low and high with the transition between the two phases
called an edge (Figure 6.4.1.28).
ADVANCING TIME
HIGH 1
Rising edge Falling edge
LOW 0
Figure 6.4.1.28 Clock signal of alternating binary signals
part of its low-high cycle or it is disabled. The state of the device changes on the
rising edge of the clock signal only.
D Clock Q
0 ↓ 0
1 ↓ 1
Q is
0 or 1 not falling
D Q unchanged
Table 6.4.1.12 States of an edge-triggered D-type flip-flop triggered by a
falling edge
CLK
Table 6.4.1.12 describes the behaviour of an edge-triggered D-type flip-flop for
Figure 6.4.1.30 Edge-triggered the case when triggering takes place on a falling edge. Figure 6.4.1.30 shows
D-type flip-flop triggered by a the circuit symbol for an edge-triggered D-type flip-flop triggered by a falling
falling edge edge (○).
Figure 6.4.1.31 shows a typical circuit for using a D-type flip-flop to
“memorise” a datum. While the datum bit is being set up on the D input, the
clock signal is prevented from reaching the clock input, CLK, of the D-type
flip-flop by setting the Enable Clock Signal to the AND gate to 0.
The output of the AND gate is then guaranteed to be zero. Next, the Enable
Clock Signal is set to 1, enabling the AND gate to pass the clock signal through
to the CLK input of D-type flip-flop. On the rising edge of the clock signal,
the output Q assumes the same value as applied to input D. The Enable Clock
Signal is then set to 0, disabling the AND gate from
D Q passing the clock signal to input CLK.
Enable
Clock Signal The input to D may now be changed without
1/0 affecting output Q. The D-type flip-flop remembers
CLK
the value applied to input D at the time the clock
signal last reached CLK.
Clock Signal
Master
Clock
Questions
20 The output Q of an edge-triggered D-type flip-flop is currently 0. Explain how this output could be
updated to Q = 1 using the circuit shown in Figure 6.4.1.31. Pay particular attention to the order of
events.
D1
Data in
D0
Address Write
A1 A0 D Q D Q
Row 1
CLK CLK
D Q D Q
Row 2
CLK CLK
D Q D Q
Row 3
CLK CLK
D Q D Q
Row 4
CLK CLK
Figure 6.4.1.32 Logic gate diagram for a 4 x 2 memory. Each row is one of the four 2-bit words. A row is
selected by setting the value of A0 A1. Data is sent along the Data in wires D0 D1 and written to the selected
D-type flip-flops by the Write signal.
• NOR
■■ Drawing and interpreting logic gate circuit diagrams involving one or
more of the above graphs
■■ Completing a truth table for a given logic gate circuit
■■ Writing a Boolean expression for a given logic gate circuit
■■ Drawing an equivalent logic gate circuit for a given Boolean expression
■■ Recognising and tracing the logic of the circuits of a half-adder and a full-
adder
■■ The use of the edge-triggered D-type flip-flop as a memory unit
Learning objectives:
■■Be familiar with the use of
■■ 6.5.1 Using Boolean algebra
Boolean algebra
Boolean identities and
In Boolean algebra as in the algebra you have studied in Maths, variables are
De Morgan’s laws to manipulate
combined into expressions with Boolean operators that obey certain laws
and simplify Boolean expressions
(rules).
Boolean variables
A B A•B The variables that we have used so far are known as Boolean variables because
0 0 0 they are two-state variables whose states have the values 0 and 1. These are not
0 1 0 the 0 and 1 of arithmetic but represent True and False.
0 1 0 Boolean operators
1 1 1
We need only consider three operators because all other operators can be
Table 6.5.1.1 Truth table for expressed in terms of these. They are the
AND function + operator denoting Boolean addition
OR
A
OR
B
Figure 6.5.1.1 Pairs of switches connected in parallel
Single licence - Abingdon School 247
6 Fundamentals of computer systems
Table 6.5.1.3 shows the truth table for the combination of switches in Figure
6.5.1.1. The two switches labelled A are ganged together as shown by a dotted
line, as are the two switches labelled B. This means that when the first switch A
is open so is the second switch A and when the first switch A is closed so is the
second A. Likewise for the two switches B.
Note that the A + B column of Table 6.5.1.3 is exactly the same as the
A • B + (A + B) column. Therefore, we can say that A • B + (A + B) is
equivalent to A + B.
We draw the conclusion that the Boolean expression A • B + (A + B) can be
simplified to A + B.
0 A 0•A 1 A 1•A
0•A=0 0 0 0 1•A=A 1 0 0
0 0 0 1 1 1
A A A•A A A A•A
A•A=A 0 0 0 A•A=0 0 1 0
1 1 1 1 0 0
A A A
Figure 6.5.1.2 Boolean identities and
their truth tables
A=A 0 1 0
1 0 1
Commutative law
A+B=B+A
Does it matter in which order inputs A and B are presented A B
A + B ≡ B + A
to an OR-gate or an AND-gate? The answer is no. B A
Associative law
Does it matter whether the operator
is applied to B and C first or to A + (B + C) = (A + B) + C
A and B first? The answer is no as
A A
A + (B + C) A +B
long as it is the same operator. B
That is what is meant by saying that ≡
B A
B • C
C C A + C
However, A + (B • C) = A + B • C because the brackets are redundant since • has higher precedence than +.
Similarly, (A + B) • C is not the same as A + B • C because B • C is evaluated first in A + B • C whereas A + B is
evaluated first in (A + B) • C and (A + B) • C = A • C + B • C.
Examples
0+A•B=A•B using 0 + X = X
Show that (A + B) • (A + B) = A • B + B • A
(A + B) • (A + B) = A • A + A • B + B • A + B • B using distribution law
Questions
Information
Using Boolean algebra show that
Writing A • B as AB:
1 (A + B) • (A + B) = A • B + B • A We can omit the • operator and
write the Boolean variables one
2 A(A + B)(B + C) = ABC after another, e.g. A • B as AB.
Examples
3 Use a truth table for each question above to verify that the identities are true.
B + A • B = (B + A) • (B + B) using distributive law
Distributive law for Boolean
(B + A) • (B + B) = (B + A) • 1 using B + B = 1
variables X, Y, Z:
X + (Y • Z) = (X + Y) • (X + Z) (B + A) • 1 = B + A using X • 1 = X
B + A = A + B using commutative law
5 A+A•B= A+B
6 A+A•B+B•C= A+B+C
7 A + A • C + B + D • (B • C + A • C) = A + B + C + D
(HINT: Use the result proved in question 5)
Using Boolean algebra simplify the following
8 ABC+ABC
9 A • (A + B) + A • B
10 (A • B + A • B) • A • B + A • B
11 Show that (A + B) • (A + B) = A • B + B • A
A + B = (A • B) A B A+B A B (A • B) (A • B)
Using + and • in place of ⋁ and
0 0 0 1 1 1 0 _
⋀ respectively, and in place of
0 1 1 1 0 0 1 ¬, De Morgan’s laws become
1 0 1 0 1 0 1
1 1 1 0 0 0 1 1. (A + B) = (A • B)
Table 6.5.1.4 Truth table for A + B and (A • B)
2. (A • B) = (A + B)
A • B = (A + B) A B A•B A B (A + B) (A + B)
0 0 0 1 1 1 0
Key point
0 1 0 1 0 1 0
1 0 0 0 1 1 0 Cancelling NOTs:
1 1 1 0 0 0 1 Care should be exercised when
cancelling NOTs. The following
Table 6.5.1.5 Truth table for A • B and (A + B) examples illustrate when you may
Examples cancel and when you may not:
(A + B) = (A + B) cancellation
possible
Show using Boolean algebra and De Morgan’s laws that A • B = A + B
(A + B) = (A + B) cancellation
A•B = A+B Using De Morgan’s law X • Y = (X + Y) possible
(A • B) = (A • B) cancellation
A+B=A+B Using the Boolean identity X=X not possible
(A • B) = (A • B) cancellation
possible
Show using Boolean algebra and De Morgan’s laws that A + B = A • B
= (B • A ) • (A • B) By commutative law
= (A • B ) • (A • B) By commutative law
A A•B
B (A • B) • (A • B)
The original Boolean expression has
A been transformed into one that can be
implemented just with NAND gates as
A•B shown in Figure 6.5.1.3.
B
(A • B) • (A • B)
Figure 6.5.1.3 NAND gate implementation
Questions
Using Boolean algebra and De Morgan’s laws show for questions 12 to 15 that
12 A • B + A • B = (A + B) + (A + B)
13 (A + B) + (A + B) = A • B + B • A
14 A•B+A=1
15 A•B•A•B=B
16 Simplify the following:
A + B + (A • B)
(a) (b) (c) A • (A + B) A•B•C+A•B
(d) (e)
A + (A • B) (A • B) + (A • B)
17 An electronic control circuit is used to switch off an industrial process when certain parameters, indicated
by two-state electronic signals W, X, Y and Z, reach critical values. The process must be stopped if either
W and X or W, Y and Z become critical at the same time. Write a Boolean expression for these parameters
that when evaluated will output 1 to switch off the process and 0 otherwise.
19 For a process to proceed the following Boolean expression must be true W • (X + Y • Z).
(a) Convert this expression into a form that could be implemented with NAND logic gates each with two
inputs.
Draw the NAND logic gate circuit for this expression.
(b) Convert this expression into a form that could be implemented with NOR gates each with two
inputs.
Draw the NOR logic gate circuit for this expression.
20 Given that
(1) Alice never gossips
(2) Bob gossips if anyone else is present
(3) Dick gossips under all conditions even when alone
(4) Soria gossips if and only if Alice is present
Determine the conditions when there is no gossip in the room.
Questions
21 A security light outside a house is controlled by two switches, which
can be turned on or off from inside the house, and a light level sensor.
The switches are named A and B. The light level sensor is named C.
The security light is labelled L.
If the light level is low (i.e. it is night time) the output of the sensor is
on otherwise it is off.
Information • If both switches A and B are off then the light L is always off.
• If switch A is on the light L is always on.
Dual-in-line package
• If switch B is on and switch A is off then:
(DIP) Digital Integrated
Circuits(ICs or chips): ºº the light L turns on if the light level is low
Logic gates are available as ºº the light L turns off if the light level is not low.
integrated circuits. An integrated Write a Boolean expression to represent the logic of the security light
circuit (IC) containing four
system.
NAND logic gates each with two
inputs is shown below in both 22 A second sensor is added to the system in Q21. This sensor is a
schematic form and as an actual movement detector. This second sensor is named M. The output of M
IC.
is on if it senses movement otherwise it is off.
14 13 12 11 10 9 8
Vcc
Information = (A • B)
A=A
■■ Distribution laws
A • (B + C) = A • B + A • C A + (B • C) = (A + B) • (A + C)
■
address
Be able to explain the buses
Main memory
difference between von or
immediate access store
Neumann and Harvard (RAM)
architectures and describe
where each is typically used
■ Understand the concept of Keyboard
Keyboard
input
VDU
output
Visual
Display
Unit
addressable memory controller controller
(VDU)
Bus width:
The number of wires in a bus is
referred to as the width of the
bus.
Visual Magnetic
Keyboard Processor Main
display disk
controller (CPU) memory
Information controller controller
Bus masters:
In the traditional shared bus
system, one device takes charge of
the system bus at a time, e.g. the Address bus
Questions Information
2 Distinguish between a parallel bus and a serial bus. Serial bus:
Parallel buses suffer timing skew
3 What is a parallel bus used for in the traditional von Neumann which limits their operating speed.
and they only operate one-way at
computer system and what name is given to this bus?
a time (half-duplex).
4 The system bus is subdivided into three functional groups of wires in A serial bus does not suffer from
timing skew and therefore can
the traditional von Neumann computer system. Name these groups.
operate at a much higher rate
than is possible with parallel
5 What is meant by saying that this von Neumann system bus is a shared
buses. They can also operate in
transmission medium? both directions simultaneously
just by adding another serial
channel. PCI Express, Serial ATA
(SATA) and USB are all examples
Modern bus systems that support serial bus operation.
The traditional computer system of von Neumann’s design required that both Address, data and control lines
processor (CPU) and main memory operate together at the same speed. are replaced by address, data
and control phases of the serial
Modern general purpose processors operate at speeds much faster than main
communication. which takes place
memory can operate. Thus the main processor or CPU in such a system will
in packets of bits.
be held up waiting for a requested data word to be fetched from memory
The serial bus is now considered
unless the CPU is decoupled from main memory in a way that allows it to do
to be a point-to-point connection
other tasks. Figure 7.1.3 shows how the bus architecture is evolving to take with one or more channels (pair
account of this and differences in speed of operation of other devices that are of wires).
connected to the computer system. Figure 7.1.4 shows a printed circuit board This has become an important
(motherboard) for a relatively modern general purpose computer. In the latest mechanism for communicating
processors and mother boards, the memory controller has been moved into the between multiple cores and
slices of main memory. One such
CPU and two separate buses are used, a memory bus and a packet-based (up to
system resembles the switching
32 bits) serial bus for peripherals. circuits of a telephone exchange
which connects multiple pathways
simultaneously to enable many
Traditional (von Neumann) computer system data bus
independent calls to be made.
The data bus, typically consisting of 8,16, 32, 64 separate lines, provides
a bidirectional path for moving data and instructions between system
components. The width of the data bus is a key factor in determining overall Key concept
system performance. For example, if the data bus is 16 bits wide, and each
Data bus:
instruction is 32 bits long, then the processor must access the main memory
The data bus provides a
twice during each instruction cycle. bidirectional path for moving
data and instructions between
system components.
Questions
6 What is the purpose of the data bus and why is its width a factor in
determining overall system performance?
Main
memory
(RAM)
Cache slots
Memory
Did you know? bus
Memory (64 bit)
The Spinnaker Human Brain controller
hub 100 Gbit/s
project:
CPU
In this project 100000 to
1000000 simple microprocessors
containing ARM processor System bus 16 Gbit/s
50 - 400 Gbit/s Serial ATA
cores and main memory are (64 bit) 0.48 -10 Gbit/s
Bus USB
being connected together in a bridge 0.1 -100 Gbit/s
Ethernet
hexagonal grid to allow networks
of many millions of neurons to be
20-256
simulated in real time. PCI BIOS
HDMI Graphics Gbit/s
The communication between the express (Flash
controller Card ROM) PCI 64-bit/100 MHz
controller
microprocessors is based on an
6.4 Gbit/s
efficient multicast infrastructure
inspired by neurobiology. It uses
PCI bus
a packet-switched network to (32 bit)
emulate the very high connectivity
of biological systems.
PCI slots
CPU beneath
fan and heat sink
PCI slots Memory
controller
hub
Main memory
(RAM)
Bus Quartz crystal
bridge oscillator for
clock timing
BIOS signals
(Flash
ROM) Figure 7.1.4 Motherboard with memory hub controller and bus bridge
Questions
7 Using examples, explain the purpose of the control bus in the
traditional von Neumann computer system.
Questions
8 The address space of a particular computer system is 220. What does
this mean?
Main memory
Main memory consists of a contiguous block of read/write,
randomly accessible storage locations constructed from
semiconductor technology - Figure 7.1.5. It is a store for
Figure 7.1.5 Main memory RAM chips
addressable words, one word per location, with each word
composed of the same number of binary digits - Main Memory
Figure 7.1.6 (Random Access Memory)
(RAM)
Each location is
• capable of “remembering” what was written
to it
• able to change its contents to another bit
pattern when a write request is received if the Currently
addressed 9 Contents of
memory is read/write (selected) 7
6 1 0 11 0 0 1 1 1 0 1 1 1 1 0 1 addressed
memory 5 memory location
• assigned a unique integer address by which location 4
3
it may be located 2
1
0
• capable of providing a copy of its contents
Figure 7.1.6 Main memory
when a read request is received.
The semiconductor technology used in read/write main memory means Information
• that the main memory is volatile, i.e. the contents of each storage von Neumann architecture:
location is lost when the power is removed. Memory contains addressable
words each composed of the
• the contents of main memory are not restored when powered up again same number of binary digits;
but instead each location consists of a random pattern of bits. addresses consist of integers
running consecutively through the
Storage locations may be visited (selected) one after another in any order
memory, 0, 1, 2, .....
noncontiguously, starting from anywhere in the memory. The time taken to
access any particular storage location is the same.
These two facts have led to main memory being labelled Random Access
Memory or RAM.
Questions
10 How is main memory organised?
Information
Memory-mapped peripherals: I/O controller Block of Command
storage locations register
The registers and block of storage (512)
locations of an I/O controller Address
register &DF03 &DF04
can use the same address space
as main memory. In this case,
&DD00
memory addresses that would
address
have been used for main memory in Status Interrupt
are allocated to I/O controllers. address &DD03 register generator
space of
The peripherals attached to these machine &DF05
I/O controllers are then said to
be memory-mapped.
Information
Address Data Control
bus bus bus
Direct Memory Access
System
(DMA): bus
Block transfer devices such as Figure 7.1.7 Block diagram of an I/O controller
magnetic disks can take charge of
the system bus to transfer a disk Questions
block directly to main memory 13 What is an I/O controller?
bypassing the processor entirely.
This is called Direct Memory 14 Why is a processor not connected directly to external devices?
Access.
15 What is an I/O port?
Processor (CPU)
The processor (CPU) executes machine instructions that have been fetched
along the data bus from main memory locations. The processor selects a
memory location by placing the address of the location on the address bus. The
data processed by machine instructions is also fetched along the data bus from
main memory and the results of processing returned the same way. The control
bus is used by the processor to assert actions, e.g. read from memory, write
to memory, and to allow devices such as the keyboard controller to grab the
attention of the processor via the interrupt mechanism when a key is pressed.
Questions
16 What is the major difference between the von Neumann computer
architecture and the Harvard architecture?
Harvard architecture:
Another stored program
Figure 7.2.1.3 Stored program von Neumann computer basic architecture
computer is the Harvard
architecture computer, a rival
architecture to the von Neumann Program code and data can be treated as if they are the same when they occupy
architecture that came from the same memory. The memory is interpreted as an instruction when the next
Princeton university. instruction pointer references it, and as data when an instruction references it.
In the Harvard architecture, the
program resides in one memory Treating program code as data is useful when, for example, a program needs
and data in a separate memory to be downloaded from a remote location because it can be treated as data and
which is also directly accessible downloaded in the same way that an email can be.
by the processor. Nevertheless a
Harvard architecture computer is Programs such as compilers also treat other programs as data when they read
also a stored program computer. them. However, treating programs as data also has a downside. Computer
viruses are programs too which get treated as data when being downloaded but
Key principle as programs when the host computer is tricked into executing them.
Task
1 Explore the university of Manchester’s site on the world’s first stored program
computer at
http://curation.cs.manchester.ac.uk/digital60/www.digital60.org/birth/index.html
Learning objectives:
■ 7.3.1 The processor and its components
■ Explain the role and operation Processor
of a processor and its major A simplified block diagram of a traditional processor is shown in Figure
components: 7.3.1.1.
A typical processor or Central Processing Unit (CPU) consists of the following
• arithmetic and logic unit
components:
• control unit
• Control Unit, which fetches instructions from memory, decodes and
• clock
executes them one at a time
• general purpose registers
• Arithmetic and Logic Unit (ALU) which performs arithmetic and
• dedicated registers, logical operations on data supplied in registers, storing the result in a
including: register. It can perform, for example, addition and subtraction, fixed
• program counter and floating point arithmetic, Boolean logic operations such as AND,
• current instruction OR, XOR and a range of shift operations.
register • Registers: general purpose, e.g. RegisterA, and special purpose or
• memory address register dedicated registers, e.g. Current Instruction Register (CIR), Program
• memory buffer register Counter (PC), Memory Buffer Register (MBR), Memory Address
Register (MAR), Status Register.
• status register
• System clock, which generates a continuous sequence of clock pulses to
step the control unit through its operation.
Register0
Register1
Memory Buffer
RegisterA RegisterB Registern Register
Operand 1 Data
Operand 2
32 32 bus
32 32
Control Unit
Current Instruction
Register
32 Instruction
Operation Instruction
ALU Operand
Program Counter address Address
RegisterC 32 Address of 32 32 bus
Next Instruction
Result
6
32 Memory Address
System Register Control
Status Register Clock bus
Central Processing
Unit
Tri-state gate: 1 or 0 or non-conducting
Figure 7.3.1.1 Simplified internal structure of a
Control
processor/central processing unit
Single licence - Abingdon School 270
7 Fundamentals of computer organisation and architecture
Key fact The processor or central processing unit is connected to main memory by
the system bus.
Processor (CPU):
The processor or central processing Questions
unit (CPU) consists of the following
1 General purpose registers are one major component of a
components:
traditional processor. Name and describe the four other major
• Control unit
• Arithmetic and Logic Unit components.
(ALU)
• General purpose registers
Processor operation with main memory
• Dedicated or special purpose A memory is a set of words, each with an address and a content:
registers
• The addresses are values of a fixed size, the address length
• system clock
• The contents are values of another fixed size, the word length
Key principle
• A load operation is used to obtain the content of a memory
Memory:
A memory is a set of words, each with word
an address and a content: • A store operation changes the content of a memory word.
• The addresses are values of
a fixed size, called the In a von Neuman computer both program and data reside in the same
address length memory. This memory is called main memory.
• The contents are values of
A processor interacts with this memory in 3 ways:
another fixed size, called
the word length • by fetching instructions
• A load operation is used
to obtain the content of a
• by loading a memory word into a processor register
memory word • by changing the content of a memory word by a store operation.
• A store operation changes
the content of a memory The size of the registers in the processor defines the size of the processor,
word. e.g. a 32-bit processor has registers that are 32 bits long. The length of a
register is known as the word length of the processor. This word length is
Key principle also usually the size of the memory word transferred in a load operation.
Processor and memory:
A processor interacts with memory in In Figure 7.3.1.1 the registers have a word length of 32 bits. Each is
3 ways: connected to a bus inside the processor which is also 32 bits wide shown
• by fetching instructions as /32 in the figure.
• by loading a memory word
into a processor register
Questions
• by changing the content of
a memory word by a store 2 State three ways that a processor interacts with memory.
operation.
Key fact
Control Unit
Processor word length:
The size of the registers in the processor
The control unit of the processor shown in Figure 7.3.1.1 controls
defines the size of the processor, e.g. a 32- fetching, loading and storing operations.
bit processor has registers that are 32 bits
It fetches an instruction into the Current Instruction Register via the
long. The length of a register is known as
the word length of the processor.
Memory Buffer Register and the data bus by
This word length is also usually the size • reading the contents of the Program Counter to obtain the
of the memory word transferred in a load
memory address of the memory word containing the instruction
operation.
• placing this memory address in the Memory Address Register Key principle
connected to the address bus so that the addressed memory word can
be selected and transferred across the data bus into the Memory Buffer Control unit:
Controls fetching, loading and
Register
storing operations.
• transferring the instruction fetched from memory from the Memory
Buffer Register into the Current Instruction Register
The control unit also Key principle
• decodes the instruction to determine if it is a load, store, arithmetic Control unit:
operation, or logic operation • Fetches an instruction
from memory and places
• executes the instruction by it in the Current
using the instruction’s operand fields as addresses to use in load or Instruction Register.
• Decodes the instruction
store operations, if required, or
to determine if it is one
loading a memory word into a register, or of load, store, arithmetic
operation, or logic
changing a word of memory in a store operation, or operation
controlling an arithmetic operation, e.g., ADD, or a logical • Executes the instruction.
operation, e.g., AND, in the Arithmetic and Logic Unit (ALU) using
as operands the instruction’s operand fields.
Questions
3 State the purpose of a processor’s control unit.
4 Describe in detail the operation of a processor’s control unit when executing a stored program, instruction
by instruction. You should state the name and describe the role of each register in this process.
System clock
The system clock or clock is a unit inside the processor that provides regular
Key fact
clock pulses that the control unit uses to sequence its operations. System clock:
The system clock is a unit inside
The clock signal is a 1-bit signal that oscillates between a “1” and a “0” with a
the processor that provides
certain frequency as shown in Figure 7.3.1.2. The change from “0” to “1” is regular clock pulses that the
called the positive edge, and the change from “1” to “0” the negative edge. control unit uses to sequence its
operations.
The time taken to go from one positive edge to the next is known as the clock
period, and represents one clock cycle. The number of clock cycles that fit one
second is called the clock frequency or clock speed. one clock cycle
1
1
Clock period =
Clock frequency 0
Figure 7.3.1.2 Clock signal
Table 7.3.1.1 shows some examples of clock speed/frequency for both current
processors/CPUs and a very popular processor from the 1980s.
Questions
5 What is the purpose of a processor’s system clock?
6 What is meant clock speed?
7 With the aid of a diagram, explain how the control unit could use the
system clock when an instruction in memory is executed in two clock
cycles.
8 The clock frequency at which a particular processor is operated is
2GHz. The number of clock cycles per instruction is 2.
How many instructions can be executed per second in this processor?
10 Processor registers and main memory are located in separate areas of Key concept
a traditional computer system. State four other differences between Special purpose or dedicated
processor registers and main memory. registers:
These are registers that are used
General purpose registers by the control unit in a specific
or dedicated way, e.g. Program
General purpose registers are registers that can be used by the programmer to Counter.
store data, as needed. Each register will be capable of storing a memory word
of a fixed size and will have a unique address known to the control unit. For
example, if there are 16 general purpose registers then their addresses will be 0, Key concept
1, 2, 3, ..., 14, 15 (0, 1, 2, ..., E, F in hexadecimal).
Program Counter (PC):
Dedicated or special-purpose registers Points to the next instruction to
Some registers are designed to be used by the control unit in a specific way, be fetched and executed.
e.g. the Program Counter (PC) stores a memory address which is the
address of the next instruction to be fetched and executed. The control unit
sets this address to ensure it points to the next instruction. The control unit Key concept
increments this address during a Fetch. It also changes this address if the
Memory Buffer Register
current instruction is a branch instruction or a subroutine call instruction or an (MBR):
interrupt service routing call. Connected to the data bus and
contains a word to be stored in
The following special-purpose registers are dedicated as follows:
memory, or a word copied from
• Memory Buffer Register (MBR): Connected to the data bus and memory.
contains a word to be stored in memory, or a word copied from This is also called the Memory
Data Register (MDR).
memory. This is also called the Memory Data Register (MDR)
Key concept • Memory Address Register (MAR): Connected to the address bus so
that the memory address it contains can appear on this bus and be used
Memory Address Register at the memory end of this bus to select a particular memory word
(MAR):
Connected to the address bus • Instruction Register (IR) or Current Instruction Register (CIR):
so that the memory address it When an instruction is fetched from memory it is stored in this register
contains can appear on this bus while the control unit decodes and executes it.
and be used at the memory end
of this bus to select a particular • Status register: This register stores single bit condition codes each
memory word. of which indicates the outcome of arithmetic and logical operations
carried out in the ALU - for example, an arithmetic operation may
produce any of the following a positive, negative, zero result, a carry,
Key concept overflow and the corresponding condition codes are set (made 1).
Current Instruction Register Sometimes the name flag is used for a single bit condition code, i.e. a
(CIR): flag is set. These condition codes may subsequently be tested by the
When an instruction is fetched control unit when it is executing a conditional branch operation. The
from memory it is stored in this
possible condition codes are
register while the control unit
decodes and executes it. Sign: Contains the sign bit of the result of the last arithmetic
operation
Zero: Set when the result is zero
Key concept
Carry: Set if an operation resulted in a carry (addition) into or
Status Register: borrow (subtraction) out of a high-order bit. Used for multi-word
This register stores single
arithmetic operations
bit condition codes each of
which indicates the outcome Equal: Set if a logical compare result is equality. (Alternatively the
of arithmetic and logical zero flag may be used)
operations carried out in the
ALU, e.g. Zero bit or flag is Overflow: Used to indicate arithmetic overflow.
set to 1 if the result of the last The status register will also have single bits to control the operation of
arithmetic operation is zero
the control unit:
otherwise it is set to 0.
The status register also has single Interrupt Enable/Disable: Used to enable or disable interrupts
bits to control the operation of
Supervisor: Indicates whether the processor is executing in supervisor
the control unit, e.g. Interrupt
Enable/Disable bit. or user mode. Certain privileged instructions can be executed only
in supervisor mode, e.g. disabling interrupts, and certain areas of
memory can be accessed only in supervisor mode.
Questions
11 What is meant by general purpose register?
Questions
14 Name four condition code flags and two control flags present in a
typical status register.
Key concept
Arithmetic and Logic Unit (ALU)
Arithmetic and Logic Unit
Figure 7.3.1.4 shows the ALU performing the arithmetic operation 3 + (- 5 )
(ALU):
producing the result -2. The Arithmetic and Logic Unit
RegisterA RegisterB
The Negative flag (ALU) performs arithmetic and
3 -5 logical operations on the data.
condition code (N)
in the status register 32 32
is set to 1 because the
result is negative. The
Zero flag (Z), Carry ADD Control
Operation
flag (C), Overflow +
Unit Key concept
ALU
flag (O) are set to 0. Negative flag:
The Interrupt Enable RegisterC 32 The Negative flag condition
code is the Sign condition code.
flag is 1 therefore -2 6
enabling interrupts.
Z NC OI S
The Supervisor mode 32
0 1 0 010
flag (S) is 0, therefore
Status Register
the processor is in User
Figure 7.3.1.4 ALU performing an ADD operation
mode. The Supervisor
flag is set to 1 when the operating system needs to use the processor, otherwise
it is 0 when a user is executing a program in the processor.
The Arithmetic and Logic Unit (ALU) performs arithmetic and logical
operations on data supplied in registers, storing the result in a register as shown
in Figure 7.3.1.4. It can perform, for example, addition and subtraction, fixed
and floating point arithmetic, Boolean logic operations such as AND, OR,
Information
XOR and a range of shift operations.
ASM Tutor:
Tasks
1 Explore the operation of a processor using a simulator such as
ASMTutor or Visual X-Toy.
■■ 7.3.2 The Fetch-Execute cycle and the role of the registers within it
Learning objectives: Fetch-Execute cycle
■■Explain how the Fetch-Execute A machine code program is made up of machine code instructions which are
cycle is used to execute fetched from main memory, one at a time, and executed in the processor/CPU.
machine code programs In Chapter 7.3.1, we learned that a processor executes each machine code
including the stages in the instruction by breaking its execution into a three-step sequence with the
cycle (fetch, decode, execute) execution synchronised by the system clock and controlled by the control unit.
and details of the registers This sequence of three steps is called the Fetch-Execute cycle or instruction
used. cycle.
The first step is a fetch operation, the second a decode operation and the third
Key concept step is execution.
Machine code program: These steps may be further broken down as follows:
A program consisting of
machine code instructions.
(Fetch phase)
1. The address of the next instruction to be executed (held in the PC) is
copied to the MAR which is connected to the address bus.
2. The instruction held at that address is fetched from memory along the
Key principle
data bus and placed in the MBR.
Fetch-Execute cycle:
A processor executes each
3. Simultaneously with step 2, the contents of the PC are incremented by
machine code instruction by 1 to point to the next instruction to be fetched.
breaking its execution into a 4. The contents of the MBR are copied to the CIR. This frees up the
three-step sequence:
MBR for the execute phase.
1. Fetch
2. Decode (Decode phase)
3. Execute
5. The instruction held in the CIR is decoded.
(Execute phase)
6. The instruction is executed. The sequence of micro-operations in the
Key fact
execute phase depends on the particular instruction being executed.
Registers always involved in
In register transfer notation, the Fetch-Execute cycle is described as follows:
the Fetch-Execute cycle:
• Program Counter (PC)
MAR [PC]
• Memory Address Register(MAR)
• Memory Buffer Register (MBR) MBR [Memory]addessed ; PC [PC] + 1
• Current Instruction Register
CIR [MBR]
(CIR)
[CIR] opcode part decoded and executed
where [ ] means contents of and means assign.
This cycle repeats until the execution of the machine code program terminates.
Machine interrupts to the processor, if enabled, are ignored until the current
Information
Fetch-Execute cycle is completed.
The PC is shown as being
Chapter 7.3.3 covers the meaning of the term opcode. Essentially it is the part
incremented by 1 in step 3 on the
previous page. This assumes that
of an instruction which specifies the type of operation to be carried out, e.g.,
every instruction is one memory ADD, SUBTRACT, AND, etc.
word in length and therefore
The operation specified by the opcode is applied to the operands part of the
occupies one memory address. If
this is not the case then the PC
instruction, i.e. the part which isn’t the opcode.
is incremented by the amount The word field is used to mean a part of an instruction, e.g. an operand field.
necessary for the PC to point to
In the case of multiple operands, there is more than one operand field.
the next instruction to be fetched
and executed. The execution step of the Fetch-Execute cycle will know from the opcode if an
operand field is a datum for immediate use or an address of a memory word
containing a datum. If the operand is an address, the execution step will fetch
Information this datum.
Branch instructions and effect
on PC:
Questions
Branch instructions can change
1 Name the four registers that are always used in the Fetch-Execute cycle.
the contents of the PC when
executed, e.g. Branch on zero
<memory address> checks to 2 Using both register transfer notation and prose, explain how the Fetch-
see if the Zero flag in the status Execute cycle is used to execute machine code programs.
register is set. If it is, the contents
of the PC will be replaced by
the value of <memory address>.
This value is the address of an In this chapter you have covered:
instruction which the PC will now ■■ How the Fetch-Execute cycle executes machine code programs, instruction by
point to.
instruction, in a repeating cycle consisting of three steps:
• Fetch
Information
• Decode
Role of ALU: • Execute
The execution of an arithmetic
or logical instruction will involve
■■ The registers that are always involved are
the ALU. The status register is • Program Counter (PC)
updated during the execution
• Memory Address Register (MAR)
to reflect the outcome of the
arithmetic or logical operation. • Memory Buffer Register (MBR)
• Current Instruction Register (CIR)
■■ These registers are
Information used together with
MAR [PC]
Role of other registers: main memory in the
The execution of an instruction Fetch-Execute MBR [Memory]addressed ; PC [PC] + 1
may involve one or more general cycle as shown here: CIR [MBR]
purpose registers as well as the
status register. [CIR] opcode part decoded and executed
where [ ] means contents of and means assign.
279 Single licence - Abingdon School
7 Fundamentals of computer organisation and architecture
Learning objectives:
■■ 7.3.3 The processor instruction set
■■Understand the term ‘processor Processor instruction set
instruction set’ and know that Format of instructions
an instruction set is processor The language of instruction for a digital computer is machine code; instructions
specific consisting of sequences of binary digits which a machine can recognise and
interpret. Machine code instructions are interpreted (executed) in a digital
■■Know that instructions consist computer’s processor (CPU) which must be designed so that it can understand
of an opcode and one or more and execute valid instructions. To understand why, consider instead a processor
operands (value, memory designed to understand certain three letter instruction words.
address or register)
Table 7.3.3.1 shows examples of possible valid and invalid instructions formed
from letters of the alphabet.
Information
Valid instruction Invalid instruction
Risc simulator:
ADD DAD
A Risc simulator designed by
SUB BUS
Peter Higginson is available from
MUL ULM
www.peterhigginson.co.uk/RISC/
DIV VID
Table 7.3.3.1 3-letter valid and invalid English instruction words
Note that the instructions are of the same fixed length and only some particular
combinations of letters are valid.
If these examples of valid combinations of letters correspond to the arithmetic
operations, ADD, SUBTRACT, MULTIPLY, and DIVIDE then we need to
include operand fields, R, B and C in each instruction so that an instruction
such as ADD has something to add and somewhere to store the result. The
instruction ADD R 3 4 adds together the values 3 and 4 and stores the result
in R. We call ADD the operation and, R, B and C the operands. Table 7.3.3.2
shows the new structure of two valid instructions
Valid instruction Action
Add C to B store
ADD R B C
result in R
Subtract C from B
SUB R B C
store result in R
Table 7.3.3.2 Operation and two operands
Questions
1 What will be stored in R if the instruction is
(a) ADD R 3 4 (b) SUB R 4 3 (c) MUL R 4 3
Similar design constraints apply when designing the set of instructions (instruction set) that a processor (CPU) is
capable of recognising as valid and then executing, i.e. instructions belonging to its instruction set. The processor
or CPU will have access to registers, to memory, to an Arithmetic and Logic Unit and will also be able to make
transfers of data to I/O devices such as magnetic disks.
The basic machine operations that a processor executes can be categorised as follows:
• Data processing: Arithmetic, logic and shift instructions
• Data transfers: Register and memory instructions
• I/O transfers: I/O instructions
• Control: Test, branch and halt instructions
Data processing instructions
The processor might support the instructions shown in column 1, Table 7.3.3.3, which also shows their abbreviated
form in brackets, e.g. SUB.
Data transfer instructions Data processing Data transfers Control
The processor might support the ADD
COMPARE (CMP)
instructions shown in column 2, SUBTRACT (SUB) LOAD (LDR)
Table 7.3.3.3. Bitwise logical AND
Unconditional branch (B)
Control instructions Bitwise logical OR (ORR)
STORE (STR)
The processor might support the Bitwise logical EOR (EOR)
Conditional branch (B)
instructions shown in column 3, Bitwise logical NOT(MVN)
Logical Shift Left (LSL) MOVE (MOV)
Table 7.3.3.3. HALT (HALT)
Logical Shift Right (LSR)
I/O transfer instructions
Table 7.3.3.3 Some examples of basic machine operations
Instructions for this are not shown
but I/O transfers could be done using the given data transfer instructions if the registers and data locations in the
I/O controllers for each peripheral are mapped into the addressable memory space by allocating main memory
addresses to these in the same way as locations in RAM are mapped into the addressable memory space.
Operands
The next choice is how many operands? ARM processors are very popular and successful processors. ARM is the
market leader (2015) for processors in smartphones and tablets and an ARM processor is the main processor in the
Raspberry Pi and in the Parallela platform. ARM is a three-register architecture, meaning that a single machine code
instruction can reference up to three registers. For example, the ADD instruction can specify two registers from
which to read the values to be added and a third register, the destination register to store the calculated sum.
Figure 7.3.3.1 shows that the
Machine code instruction
structure of a machine code
Opcode Operand field (3 operands in this example) instruction is divided into an opcode
Basic Address
field and an operand field. Register
machine mode Rd is the destination register, register
operation
Rn contains the first input to the
operation and Operand2, the second.
1 0 1 1 0 0 Rd Rn Operand2
Operand2 could be an actual value,
6, or it could be another register, Rm,
Figure 7.3.3.1 Structure of a machine code instruction with containing the value to be used. The
an example Address mode bit is used to select
one for each 6-bit pattern. Table 7.3.3.4 Table 7.3.3.4 No of possible opcodes for a given no of opcode field bits
shows how the number of possible opcodes
varies with the number of bits reserved for the opcode field of a machine code instruction (remember: opcode field
size is fixed at the design stage of a processor).
Machine code
Opcode Description
instruction format
000000 LDR Rd, <memory ref> Load the value stored in the memory location specified by <memory ref> into register Rd.
000010 STR Rd, <memory ref> Store the value that is in register Rd into the memory location specified by <memory ref>.
Add the value specified in <operand2> to the value in register Rn and store the result in register Rd.
000100
ADD Rd, Rn, <operand2> 000100 for when operand2 is a value, 000101 for when operand2 is another register, Rm. The same
000101
interpretation applies to the other two opcode instructions.
000110 Subtract the value specified by <operand2> from the value in register Rn and store the result in register
SUB Rd, Rn, <operand2>
000111 Rd. For interpretation of why two opcodes, see ADD for why two opcodes.
001000 Copy the value specified by <operand2> into register Rd. For interpretation see ADD.
MOV Rd, <operand2>
001001
001010 Compare the value stored in register Rn with the value specified by <operand2>. See ADD
CMP Rn, <operand2>
001011
001100 B <label> Always branch to the instruction at position <label> in the program.
011101 Conditionally branch to the instruction at position <label> in the program if the last comparison met
011111 the criteria specified by the <condition>. Possible values for <condition> and their meaning are:
B <condition> <label>
011101 EQ: Equal to NE: Not equal to GT: Greater than LT: Less than.
011111
010010 Perform a bitwise logical AND operation between the value in register Rn and the value specified by
AND Rd, Rn, <operand2>
010011 <operand2> and store the result in register Rd. See ADD for why two opcodes.
010100 Perform a bitwise logical OR operation between the value in register Rn and the value specified by
ORR Rd, Rn, <operand2>
010101 <operand2> and store the result in register Rd. See ADD for why two opcodes.
010110 Perform a bitwise logical eXclusive OR (XOR) operation between the value in register Rn and the
EOR Rd, Rn, <operand2>
010111 value specified by <operand2> and store the result in register Rd. See ADD for why two opcodes.
011000 Perform a bitwise logical NOT operation on the value specified by <operand2> and store the result in
MVN Rd, <operand2>
011001 register Rd. See ADD for why there are two opcodes.
011010 Logically shift left the value stored in register Rn by the number of bits specified by <operand2> and
LSL Rd, Rn, <operand2>
011011 store the result in register Rd. See ADD for why there are two opcodes.
111100 Logically shift right the value stored in register Rn by the number of bits specified by <operand2> and
LSR Rd, Rn, <operand2>
111101 store the result in register Rd. See ADD for why there are two opcodes.
111110 HALT Stops the execution of the program.
Table 7.3.3.5 6-bit opcodes mapped to machine operations. Reproduced with permission of AQA. Currently only in specimen
papers and has therefore not been through the complete rigorous question paper process and is liable to change. Please consult
AQA’s website for the most recent version of the specification.
Table 7.3.3.5 shows a possible mapping of some 6-bit opcodes (5 bits for Key concept
basic machine operation, 1 bit for address mode) to machine operations for an
imaginary processor. Processor instruction set:
The set of bit patterns for which
Instruction set machine operations have been
The simple operations referenced in Table 7.3.3.5 may be combined together in defined.
sequences to perform quite complicated tasks.
Information
The set of 28 bit patterns (strings of bits) shown in the opcode column of Table
7.3.3.5 represent these operations for a given processor and are known as the ARM:
ARM processor technology is
processor instruction set. Note that if Table 7.3.3.5 shows all the operations that
controlled by ARM Holdings.
a processor has been designed to understand and interpret then four bit patterns ARM stands for Advanced
do not correspond to any defined machine operations because for a 6-bit opcode RISC Machine and RISC stands
field, there are 32 possible bit patterns but only 28 are used to define opcodes. We for Reduced Instruction Set
Computer. RISC is a design
therefore use the following definition of instruction set:
philosophy in which the
The set of bit patterns for which machine operations have been defined. instruction set is deliberately
designed as a small set of very
An instruction set is processor specific
simple instructions.
An instruction set is specific to a particular processor for the following reasons: It does mean that a machine
• The machine operations that a processor is designed to perform varies code program will consist of a
large number of these simple
in number and type from processor to processor, e.g. from the ARM®
instructions but execution of
Cortex®-A7 CPU used in the Raspberry Pi 2 to the Intel® Core™ i7 simple instructions is very fast.
CPU used in laptops and PCs.
The simplicity of the instructions
• The number of bits allocated to the opcode field can also vary from simplifies the design of the control
processor to processor as well as how they are mapped to the operations unit. In particular, the instruction
that the processor supports. Therefore, machine code programs written for decoder circuits. This turns out
the ARM Cortex-A7 CPU will not run on an Intel Core i7 CPU. to have the advantage of lowering
processor power consumption.
• The number of possible operands, their type and the number of bits
Low power consumption is
reserved for each may also vary from processor to processor. particularly desirable in portable
• The design of a processor's control unit instruction decoder circuits devices such as smartphones
reflects the structure of the machine code instructions and will not and other embedded systems.
therefore be able to decode instructions designed for a different processor. This accounts for why ARM
processors dominate the mobile,
Structure of machine code instructions
tablet, System On a Chip (SOC)
We have learned that a machine code instruction is divided into an opcode part and embedded systems markets.
and an operand part as shown again in Figure 7.3.3.2. Considerably more ARM CPUs
than Intel CPUs are sold.
Opcode Operand(s)
Questions
3 What is meant by processor instruction set?
Task
1 You are required to design an instruction set for a processor based on a
two register architecture in which the destination register is always the
register called the accumulator. The processor must be able to add and
subtract. The only instructions allowed to interact directly with main
memory are load and store. The first input to an arithmetic operation
is always read from the accumulator. The second input is read from
one of fifteen other registers. Values may be set up in all sixteen
registers either by a load instruction or by a move instruction. The
second operand of a move instruction is always a value. The processor's
instruction set must support four control instructions:
1. branch on zero
2. branch on negative
3. branch unconditionally
4. Halt
Branch instruction 1 tests the zero flag and branch instruction 2,
the negative flag of the status register. All three branches use a single
operand which is a value (positive or negative) to add to the current
value of the program counter. The status register and program counter
are separate from the sixteen registers also used by the instruction set.
Registers have a word length of 16 bits as does main memory.
Main memory consists of 256 memory words.
Some instructions may not use all 16 bits. The processor will use those
bits which define the instruction.
Key concept
Direct addressing
Direct addressing:
The operand is the address of
When the addressing mode is direct addressing the operand is the address in
the datum. memory where the datum can be found.
For example, the LDR operation loads the value stored in the memory location
specified by <memory ref> into register Rd.
LDR Rd,<memory ref>
If Rd is register R0 and <memory ref> is in hexadecimal 0xFCC0 (64704
in decimal) then the assembly language instruction is as follows
LDR R0,0xFCC0
Note that in this instruction there is no #. The absence of the # symbol
indicates that this is direct addressing.
The operand 0xFCC0 is the main memory address of a memory location containing the datum to be used when
this instruction is executed. The contents of register R0 will therefore be 0x4D (we can omit the leading 00) after
this instruction is executed because memory location with address 0xFCC0 contains the datum 0x4D - Figure
7.3.4.2.
Main Memory
Address
R0
0xFFFF 0xFCC2
before ?
0x0000 0xFCC1
0x004D 0xFCC0
after 0x4D
0x014E5 0xFCBF
Figure 7.3.4.2 register R0 before
and after execution of instruction 0x3628 0xFCBE
LDR R0,0xFCC0
Questions
2 Register R0 contains the datum 4310, main memory contents are as shown in Figure 7.3.4.2. What will
register R0 contain after the following assembly language instruction has been executed?
Express your answer in decimal.
LDR R0,0xFCC0
3 In the assembly language instruction STR Rd,<memory ref>, the STR operation stores the value
that is in register Rd in a memory location specified by <memory ref>. Register R0 contains the
datum 4310, register R1 the datum 5610, main memory contents are as shown in Figure 7.3.4.2. What
will the memory locations 0xFCC1 and 0xFCC2 contain after the following two instructions have been
executed? Express your answers in decimal.
STR R0,0xFCC1
STR R1,0xFCC2
4 Main memory contents are as shown in Figure 7.3.4.2 What will be stored in register R1 after the
following instructions are executed? Express your answer in decimal.
LDR R0,0xFCC0
ADD R1,R0,#0xFCC0
Learning objectives:
■ 7.3.5 Machine-code and assembly language operations
■ Understand and apply the basic Load-Store architecture
machine-code operations of In a load-store architecture the only instructions that work directly with
• load memory are load and store instructions or their equivalent. A value in memory
that needs to be processed
• add
must be loaded into the
• subtract processor (core) first,
• store processed and then stored
• branching (conditional back in memory.
and unconditional) Load
• compare A load register operation
is used to transfer a
• logical bitwise operators
copy of a datum from a
(AND, OR, NOT, XOR)
specified location, e.g. main Figure 7.3.5.1 ASMTutor after executing the
• logical memory location 102, to a machine code equivalent of MOVE 102, R0
shift right symbolically named register,
shift left e.g. R0.
• halt For example using direct addressing, LDR R0, 102 transfers the contents of
■ Use the basic machine-code memory location with address 102 into register R0. In some instruction sets,
operations above when machine- the mnemonic MOV or MOVE is used instead of the mnemonic LDR, and
code instructions are expressed the order of the operands can be reversed. The simulator ASMTutor shown in
in mnemonic form- assembly Figure 7.3.5.1 is one that uses MOVE instead of LDR for a load operation. It
language, using immediate and also reverses the order of the operands.
direct addressing Figure 7.3.5.1 shows an example of an assembly language program which
transfers a copy of the datum 32 from memory location 102 to register R0. The
Key concept assembly language program ends with RTS (ReTurn from Subroutine) because
Load-store architecture: ASMTutor expects the last
No direct manipulation of instruction to be RTS in
memory contents. A value
order to work correctly.
in memory that needs to be
Note that memory location
processed must be loaded
into the processor (core) first, with address 102 in this
processed and then stored back example contains 32.
in memory. Figure 7.3.5.2 shows the
register state just before the
Information machine code equivalent of
See Table 7.3.3.5 in Chapter 7.3.3 MOVE 102, R0 is executed. Figure 7.3.5.2 ASMTutor just before executing
for AQA instruction set.
the machine code equivalent of MOVE 102, R0
102 21
103 42
104 84
Figure 7.3.5.4
Questions
1 The assembly language instruction LDR R0, 102 transfers a copy of the contents of memory location 102 to
register R0. Figure 7.3.5.4 shows the contents of memory locations 102, 103 and 104.
What does register R0 contain after the following instructions, expressed in assembly language, are executed
in machine code?
(a) LDR R0, 102 (b) LDR R0, 103 (c) LDR R0, 104
Figure 7.3.5.6 shows the result of executing the instruction LDR R4, [R0].
The ARM μVision simulator
simulates the execution of
instructions for the Cortex™-M
family of ARM microcontrollers.
These microcontrollers implement
the ARMv7 instruction set. In order
for this simulator to function, every
assembly language program must
start with the preamble shown in
Figures 7.3.5.5, 7.3.5.6. The user
chooses the identifier in AREA
FirstASM1, CODE, i.e. FirstASM1,
but the rest of the preamble must
Figure 7.3.5.6 ARM® μVision® V 5.17 Debugger in single-step more
conform to that given.
STORE
A store operation, STR, transfers a copy of the contents of a register to a specified memory location,
e.g. STR R4, 0x20000000
If R4 contains 0x00000065 then
execution of the machine code
equivalent of this instruction will
change the contents of memory
location, address 0x20000000, to
the value 0x65.
Figure 7.3.5.7 shows register R4
preset with value 0x00000065,
register R0 preset with value
0x20000000, and memory Figure 7.3.5.7 ARM® μVision® V 5.17 Debugger in single-step more
location 0x2000000 initialised to
0x00000000. The next instruction
to be executed is STR R4, [R0]
which is equivalent to
STR R4, #0x20000000
Questions
2 The assembly language instruction STR R0, 102 transfers a copy of the contents of register R0 to memory
location with decimal address 102. What are the contents of memory location 102 after this instruction is
executed in machine code when the value in R0 is decimal 67?
MOVE
A MOVE operation copies a value from source to destination. The source could be a register, an immediate value
(or a memory location but not in the case of ARM processors). The destination could be another register (or a
memory location but not in the case of ARM processors).
For example, the ARM processor instruction set has a MOV instruction (MOVS in ARMv7 to update status register as
well). MOV Rd, <operand2> copies the value specified by <operand2> into register Rd. For example, MOV R2, #36
copies the value 36 into the register R2. # indicates immediate addressing.
MOV R1, R2 copies the value in register R2 into register R1. Figure 7.3.5.9 shows an assembly language program,
MOV.s, prepared in Notepad++ and then loaded into ArmSim# version 1.9.1 for assembling and executing one
instruction at a time. Register contents are shown in the window on the left. Note how registers R2 and R3 are
changed by the two different MOV operations. The hash symbol # before the decimal value 36 indicates immediate
addressing, i.e. the operand is the value to be used. As with ASMTutor, the last instruction must be an instruction
that enables ArmSim# to function correctly. This instruction is SWI 0x11.
Questions
3 The assembly language instruction MOV R0, R1 transfers a copy of the contents of register R1 to register
R0. What does register R0 contain after the following instructions, expressed in assembly language, are
executed in machine code?
(a) MOV R0, #78 (b) MOV R1, #25 followed by MOV R0, R1
ADD
An add operation ADD Rd, Rn, <operand2> is
used to add the value specified in <operand2>
and the value in register Rn, storing the result in
register Rd. <operand2> may be an immediate
value or a register.
For example, ADD R2, R3, #1 when executed in
machine code adds 1 to the contents of register
R3 before storing the result in register R2.
Task
1 Using a text editor such as Notepad++,
create the file ADD32.s with the
following contents:
Task
2 Using a text editor such as Notepad++,
create the file ADD.s with the following
contents:
SUBTRACT
A subtract operation SUB Rd, Rn,
<operand2> is used to subtract the value
specified in <operand2> from the value
in register Rn before storing the result
in register Rd. <operand2> may be an
immediate value or a register.
For example, SUB R2, R3, #1 when
executed in machine code subtracts 1 from
the contents of register R3 before storing
the result in register R2.
Figure 7.3.5.11 shows the before and after
contents of the registers R2 and R3.
Task
3 Using a text editor such as
Notepad++, create the file SUB5.s
with the following contents:
Status register
The status register contains flags called condition codes which are set or reset to reflect the outcome of the last
machine operation, e.g. if the result of an arithmetic operation was zero then the zero flag is set. A flag is a single bit
code that can be set (binary 1) or reset (binary 0). A status register consists of at least four condition codes:
• Zero flag - set if the result of the last machine operation stores zero in the results register
• Negative flag - set if the result of the last machine operation stores a negative value in the results register
• Carry flag - set if the result of an unsigned operation overflows the result register or as a result sometimes of
performing two’s complement signed arithmetic.
• Overflow flag - set if the result of a signed operation overflows the result register.
Moving zero into a register can set the Z(ero) flag as will an arithmetic operation if the result is zero.
Moving a negative value into a register can set the N(egative) flag as will an arithmetic operation if the result is
negative. A carry can be produced when a machine performs two’s complement arithmetic or when it performs
unsigned addition.
Questions
6 The format of the MOVE operation in an
instruction set is MOV Rd, <operand2> which
is interpreted as copy the value specified by
<operand2> into register Rd. Assuming that MOV
can set the condition codes.
Which, if any, status register condition codes are
set when the machine code equivalents of the
following are executed
(a) MOV R0, #-1 (b) MOV R1, #0 Figure 7.3.5.12 Single-stepping an assembly
2 (c) MOV R2, #23? language program in ASMTutor to show the effect
7 The format of the SUBTRACT operation in an on the status register
instruction set is SUB, Rd, Rn, <operand2> which
is interpreted as subtract the value specified in
<operand2> from the value in register Rn and
store the result in register Rd. Register R0 stores
decimal 7. Which status register condition codes
are set, if any, when the machine code equivalents
of the following are executed
(a) SUB R1, R0, #9 (b) SUB R1, R0, #7
(c) SUB R1, R0, #5?
COMPARE
Compare instructions may be used to
compare the contents of two registers or
the contents of a register and an immediate
value. For example, CMP R0, R1 compares
the contents of registers R0 and R1. If
the contents of these registers are equal,
the zero flag in the status register is set. A
compare operation performs a subtraction
and uses the result to determine whether
the two operand’s values are equal or not. If
Figure 7.3.5.14 ARM µVision simulating the execution of CMP R0, #9 unequal then the negative flag will be set if
the subtraction result was negative. Figure
7.3.5.14 shows ARM μVision simulating
the execution of CMP R0, #9 with decimal
7 stored in R0. The negative flag is set
indicating that the operation [R0] - 9 has
been performed by CMP. The notation []
means ‘contents of ’.
Figure 7.3.5.15 shows the result of CMP
R0, #7 with decimal 7 stored in R0.
Note that the zero flag is set. The carry
flag is also set because two’s complement
arithmetic sometimes sets this flag.
Figure 7.3.5.15 ARM µVision simulating the execution of CMP R0, #7
Questions
8 What is the state of each condition code after the following comparison operations are executed in machine
code? R0 contains the value decimal 9. Assume that CMP behaves as shown above.
(a) CMP R0, #15 (b) CMP R0, #7 (c) CMP R0, #9
Conditional branch
There is another kind of branch called a conditional branch. In this type of branch a condition must be true for
branching of program execution to occur.
The instruction immediately before a conditional branch must be a COMPARE instruction. Execution of this
instruction affects the condition code flags which conditional branch instructions examine before deciding whether
or not to branch (SUBTRACT can be used instead of COMPARE, e.g. is an alternative to CMP R0, R1).
Table 7.3.5.4 Condition and condition codes for SUB and CMP
Table 7.3.5.5 shows the four conditional branch instructions, BEQ, BNE, BGT and BLT.
Condition
Instruction Description
codes
BEQ <label> Branch if operands being compared are equal Z=1
BNE <label> Branch if operands being compared are not equal Z=0
Branch if first signed operand is greater than
BGT<label> Z = 0, N = 0
second signed operand
Branch if first signed operand is less than second
BLT <label> Z = 0, N = 1
signed operand
Questions
9 Explain what the following snippet of
assembly language code does when its
machine code equivalent is executed
Questions
10 Explain what the following snippets of assembly language code do when their machine code equivalent is
executed
11 What other conditional branch instruction would result in the code behaving in a similar way if used in
place of BGT and BLT in (a) and (b)?
This means perform a bitwise logical operation between the value in register Rn and the value specified by
<operand2> and store the result in register Rd. The symbolic opcode for the AND operation is AND; for the OR
operation it is ORR and for XOR it is EOR.
The format for ARM processors for the logical operation NOT is
MVN Rd, <operand2>
This means perform a bitwise logical NOT operation on the value specified by <operand2> and store the result in
register Rd.
AND
Figure 7.3.5.18 shows ARM μVision
simulating 11112 AND 00012. The result
is 00012 when AND R0, R0, R1 is executed
in machine code. This instruction ANDs the
contents of registers R0 and R1 and stores the
result in R0, the register specified as the first
operand.
A mask operation is one that isolates bits to
Figure 7.3.5.18 ARM µVision simulating the execution of a
be tested. The logical AND can be used in
program that applies a bitwise AND operation to operands
this role. Suppose that we need to test the
0xF and 0x1 i.e. 11112 AND 00012
three least significant bits of a 32-bit word
Single licence - Abingdon School 298
7 Fundamentals of computer organisation and architecture
The zero flag will be set by CMP if they are. Figure 7.3.5.19 ARM µVision simulating the execution of a
program that applies a bitwise OR operation to operands 0x8 and
0x7 i.e. 10002 OR 01112
OR
Figure 7.3.5.19 shows ARM μVision
simulating 10002 OR 01112. The result is
11112 when ORR R0, R0, R1 is executed
in machine code. This instruction ORs the
contents of registers R0 and R1 and stores
the result in R0, the register specified as the
first operand.
XOR
XOR is the eXclusive-OR operation.
ARM names the operator for this operation
EOR (Exclusive-OR) Figure 7.3.5.20 ARM µVision simulating the execution of a
Figure 7.3.5.20 shows ARM μVision program that applies a bitwise XOR operation to operands 0x8
simulating 10002 XOR 01112. The result is and 0x7 i.e. 10002 XOR 01112
11112 when EOR R0, R0, R1 is executed in
machine code. This instruction Exclusive-
ORs the contents of registers R0 and R1 and
stores the result in R0, the register specified
as the first operand.
NOT
To perform a bitwise logical NOT operation
the instruction
MVN Rd, <operand2>
The result is 111111111111111111111111111111112 when MVN R0, R1 is executed in machine code. The result
expressed in hexadecimal is FFFFFFFF. Note that R1 was assigned 0x0 in a MOV operation first.
Questions
12 What will the contents of register R0 be after the machine code equivalent of the following snippets of
assembly language code are executed
(a) MOV R0, #0xFF (b) MOV R0, #0xFF (c) MOV R0, #0x0
EOR R0, R0, R0 MOV R1, #0x7 MOV R1, #0x7
HALT AND R0, R0, R1 ORR R0, R0, R1
HALT HALT
13 A certain process may begin if bits 1, 3 and 5 of an 8-bit word are set. The state of the other bits may be
ignored. Write the assembly language instructions to determine if the process may begin. You should assume
that bit 1 is the least significant bit and that register R0 contains the 8-bit word.
14 Write an assembly language instruction using ORR to set bit 4 of register R0. Assume bits are numbered
from the right 1...8 with bit 1 the least significant bit.
15 Write an assembly language instruction to isolate bits 1 and 3 of register R0 so that the state of each may be
tested by other instructions. Assume bit numbering as in Q14.
16 “We use the logical OR to make bits become one, and we use the logical AND to make bits become zero.”
Explain using examples the meaning of this statement.
17 Register R0 contains a 32-bit word that represents the state of 32 pixels of a black and white image with
colour depth one bit per pixel. Write a single assembly language instruction to invert the state of each pixel
stored in R0. Write another instruction to restore the stored state.
Questions
18 Rewrite
loop LSL R0, R0, #1
B loop
END
to obtain the bit pattern Figure 7.3.5.23 Logical Shift Left by one bit applied 8 times
0x00000100 from the bit pattern by single-stepping through loop eight times
0x00000001 stored in R0 without
a loop.
Questions
19 The decimal number 4 is stored in register R0. Write an assembly language instruction that multiples this
number by 24.
20 Using Figure 7.3.5.22 as a template, record the state of R0 after the following assembly language program
is executed in machine code. Assume R0 is an 8-bit wide register. Include the carry bit in your answer.
MOV R0, #3
LSL R0, R0, #7
HALT
1 0 1 1 0 1 1 0
Figure 7.3.5.24(a) 8-bit bit pattern before it is shifted right one bit
0 1 0 1 1 0 1 1 0 Carry bit
Figure 7.3.5.24(b) 8-bit bit pattern after it is shifted right one bit
Questions
21 R0 contains 0x00000100. What does it contain after LSR R0, R0, #8 in machine code is executed?
22
23 The decimal number 64 is stored in register R0. Write an assembly language instruction that divides this
number by 24. The result should be stored in register R1.
23 Using Figure 7.3.5.24 as a template, record the state of R0 after the following assembly language program
is executed in machine code. Assume R0 is an 8-bit wide register. Include the carry bit in your answer.
MOV R0, #195
LSR R0, R0, #7
HALT
HALT
When a HALT instruction is encountered in an executing machine code program the execution of the program is
stopped.
Questions
Use AQA’s instruction set from Table 7.3.3.5 in Chapter 7.3.3 to answer these questions.
24 The high level language program statement "Sum := Sum + 100;" assigns to variable Sum the result of
adding decimal number 100 to Sum. The symbol ":=" is the assignment operator. Write the equivalent
assembly language instructions for this statement. Assume that memory location with address 0x1000 is
used to store the current value of variable Sum.
25 Write the equivalent assembly language instructions for high level language statement
If Sum > 5 Then Sum := Sum + 1 Else Sum := Sum - 1;
Assume that memory location with address 0x1000 is used to store the current value of variable Sum.
26 Write the equivalent assembly language instructions for high level language statement
While Sum < 10 Do Sum := Sum + 1;
Assume that memory location with address 0x1000 is used to store the current value of variable Sum.
27 Write the equivalent assembly language instructions for high level language statement
Repeat Sum := Sum - 1 Until Sum = 0;
Assume that memory location with address 0x1000 is used to store the current value of variable Sum which
is decimal 10.
28 Write the equivalent assembly language instructions for high level language statement
Sum := Sum * 8;
Assume that memory location with address 0x1000 is used to store the current value of variable Sum, an
unsigned number.
29 Write the equivalent assembly language instructions for high level language statement
If (SwitchSettings BitWiseAND 4) = 1 Then Sum := 0 ;
Assume that memory location with address 0x1000 is used to store the current value of variable
SwitchSettings, an unsigned number and memory location 0x1004 the current value of variable Sum.
30 Use an assembly language simulator to check your answers.
Sources of interrupt
There are many sources of interrupt. Table 7.3.6.1 shows the main ones and
their priority with 1 being the highest and 4 the lowest.
1. The processor must complete the current fetch-execute cycle for the
current program if begun;
2. The contents of the program counter, which points to the next
instruction of the current program to be executed, must be stored away
safely so it can be restored after servicing the interrupt;
3. The contents of other registers used by the current program are stored
away safely for later restoration;
4. The source of the interrupt is identified;
5. Interrupts of a lower priority are disabled;
6. The program counter is loaded with the start address of the relevant
interrupt service routine;
7. The interrupt service routine is executed.
1. The saved values belonging to the current program for registers other
than the program counter are restored to the processor’s registers;
2. Interrupts are re-enabled;
3. The program counter is restored to point to the next instruction to be
fetched and executed in current program.
Questions
1 What is an interrupt?
• word length while it is executing. Figure 7.3.7.1 shows this basic Figure 7.3.7.1 Basic
model. The fetching, decoding and execution of an computational model
• address bus width
instruction is synchronised with the CPU’s clock.
• data bus width
The number of clock cycles ("ticks") of the CPU’s clock it takes the CPU
Information to execute an instruction varies from instruction to instruction with load
instructions which load data from memory taking the most.
No of clock cycles per
instruction: Suppose, the average number of clock cycles per instruction = 2
The number of clock cycles that 1
then the average number of instructions executed per clock cycle = / = 0.5
an instruction takes to execute is 2
determined by its complexity and If the CPU operates at a clock frequency of 800 MHz then there are 800
the design of the control unit in
million clock cycles per second.
the CPU. For a given Instruction
Set Architecture (ISA), the same Using this, we calculate that
instruction may take longer to
the average number of instructions executed per second is 0.5 x 800
execute on one processor than
another operating at the same = 400 million per second
clock frequency. The difference is
Cycles per instruction (CPI)
in the design of the control unit.
Cycles per instruction (clock cycles per instruction) is one aspect of a
processor’s performance. When evaluating processor performance the average
number of clock cycles per instruction is often used.
Information
Questions
Average no of clock cycles The performance of two processors with the same instruction set
1
per instruction:
architecture but operating at different clock frequencies is assessed
The percentage of instructions
that are load, store, integer by measuring the average number of cycles per instruction (CPI) for
arithmetic, branches varies various programs compiled by the same compiler and executed on each
from program to program so processor. Which processor do you think was the faster at executing
the average no of clock cycles
these programs? Justify your answer.
per instruction will vary from
program to program. Processor 1: Clock frequency 5 GHz CPI = 3
Processor 2: Clock frequency 3 GHz CPI = 1.5
labelled 1, 2, 3 and 4. Each core has its own data cache as 16-64k 16-64k Core
I-Cache D-Cache
well as an instruction cache enabling instructions to be pre- 1 2 3
fetched as well. SIMD operations for handling audio and 4
video processing as well as graphics and gaming processing
SCU L2 Cache
rely on a special control unit called the NEON Data Engine.
128-bit AMBA® ACE Coherent Bus Interface
Floating point operations take considerably longer than fixed
point and integer operations (fixed point data can be treated Figure 7.3.7.2 ARM® Cortex®-A7
and processed as integers). The A9 processor also includes a quad-core processor
dedicated Floating Point Unit specially designed to allow the
CPU to offload floating point operations to this unit.
Figure 7.3.7.3 shows the Parallella computer platform which is an energy efficient, high performance, credit card
sized computer based on the Epiphany multicore chips from Adapteva®. This desktop version cost £150 and is
used for developing and implementing high performance, parallel processing. It uses a Zynq dual-core ARM A9
processor to launch and run programs which use the 16-core Epiphany coprocessor for parts of the program that
can be executed in parallel. The Epiphany coprocessor is also available with 64 cores.The Parallela’s Ethernet
Zynq dual-core ARM A9 processor connection allows multiple
Gigabit
with Field Programmable Gate units to be interconnected to
Ethernet
Array (FPGA) make a cluster.
16-core Epiphany
coprocessor
Questions
3 Explain how a multi-core CPU
can improve the performance
of the CPU when executing a
program.
MESH NODE
16 cores
Questions
7 Explain the effect on processor performance of processor word length.
1. Instruction count
• influenced by the design of the compiler or whether sections of code
have been rewritten in assembly language to use fewer instructions.
2. CPI (cycles per instruction) and clock cycle time
• influenced by the design of the processor and its operation.
CPI can be reduced by
• increasing the word length of the processor
• redesigning its control unit to take less time decoding and executing
instructions
• using multiple cores
• increasing memory bandwidth (number of bits transferred per second)
by increasing the width of the data bus for a given clock rate
by clocking the memory bus at a higher rate for a given width of
data bus
• pre-fetching data and instructions and storing these in fast access
memory (cache) located on processor chip.
Clock cycle time can be reduced by increasing the clock speed and clocking
the processor at a higher rate.
To improve the performance of a processor regarding the number of memory
words it can address, the width of the address bus needs to be increased.
Information around for 40 years and the necessary components of laser diode and decoding
QR code: electronics have benefited from high volume of use which has led to economies
Refers to a single member of of scale in their manufacture.
the family of 2D barcodes 2D
barcodes are machine-readable
Information
codes that use markings forming The GS1 organization (www.gs1.org) maintains the standards related to the Global Trade
a two-dimensional grid. The Item Number (GTIN). There are several symbologies that belong to this family, all of them
example below is a QR code that representing a product code and all using the same kind of barcode symbols:
represents the text Unit 2 CS. UPC-A - The first product barcode (12 digits) now refered to as GTIN-12.
EAN-13 - European barcode (13 digits) now refered to GTIN-13.
Questions
1 What is a barcode?
Digital camera
Light reflected from objects is focussed by the lens of a digital camera onto
Figure 7.4.1.3 CCD SONY a two-dimensional array of light-sensitive cells (photosensors) to form an
ICX493AQA 10.14 Mpixels analogue image. Each cell or site accumulates an electric charge proportional
APS-C 1.8” (23.98 x 16.41mm) to the brightness of the illumination and as the latter varies in a continuous
sensor side manner so does charge accumulation. Both are analogue quantities.
To process the analogue image digitally, the magnitude of the charge in each
Information photosensor is sensed and converted into digital format by an analogue to
digital converter (ADC).
Digital single-lens reflex
cameras: The two-dimensional array (matrix) of photosensors and associated electronics
These use an aspect ratio of 3:2. to perform analogue to digital conversion is one of two types:
Aspect ratio is the ratio of the
width of the image to its height.
• Charge-Coupled Device (CCD)
The Canon EOS 600D (released • Complementary Metal-Oxide Semiconductor (CMOS)
February 2011) uses an APS-C
CMOS sensor consisting of a
Both are fabricated from metal oxide semiconductors with photodiodes used
sensor array of dimensions 5184 x as photosensors. A single chip may contain millions of photodiodes laid out in
3456 pixels. rows and columns forming a matrix.
Figure 7.4.1.4(a) illustrates with a 6 x 6 matrix how coloured images can be Information
sensed in a CCD-type camera sensor. Bayer filter:
Red, green and blue filters cover the A Bayer filter mosaic is a colour
One pixel filter array for arranging RGB
photodiodes. Each pixel is composed
colour filters on a square grid of
of one red, one blue and two green photosensors (U.S. Patent No.
filtered photodiodes reflecting the fact 3,971,065). developed by Kodak
that the human eye is more sensitive employee Bruce Bayer in 1976.
captured which means the site is CCD sensors have greater light
sensitivity and produce images
less sensitive to light. It also leads
which are less noisy. Noise
Figure 7.4.1.4(b) CMOS-type sensor to images with more noise than produces grainy images.
photodiodes overlaid onto a CCD-type with the equivalent CCD-produced CCD sensors have a greater
image. resolution than CMOS sensors
sensor matrix for capturing coloured
because their packing density is
images higher.
Laser printer
A laser printer prints a whole page at a time. It prints high-quality text and
graphics on plain paper.
A page description language usually Key fact
describes the page to be printed as lines, Monochrome laser printer:
arcs and polygons. A processor in a A monochrome laser printer uses
CMY “primaries” are combined a single toner containing a black
monochrome laser printer generates a
at full strength, the resulting powder. It is only able to print text
bitmap of the page in raster memory from
“secondary” mixtures are red, in black.
green, and blue. Mixing all three the page description. A negative charge They are poor at printing greyscale
gives black. However, for a darker is applied to the photosensitive drum images which has to be done by
black the Key (Black) toner is at the heart of the laser printer. One or printing small dots and varying the
used. spacing and arrangement of those
more laser beams are directed onto the
dots. This is called dithering or
rotating drum’s surface (Figure 7.4.1.5).
halftoning.
Key fact The lasers are turned on or off at positions
Colour laser printer: determined by the bitmap data stored in
In a color laser printer, each of the raster memory. This causes the negative charge to be neutralised or reversed
the four CMYK toner layers is at positions corresponding to the black parts
stored as a separate bitmap.
of the page to be printed.
The resulting pattern of charges on the
drum’s surface is an image of the page to be
printed. The charged surface of the drum
is exposed to toner, fine particles of dry
plastic powder mixed with carbon black or
colouring agents. The charged toner particles
are given a negative charge so they attach to
the uncharged or positively charged regions
of the drum and not to the negatively
charged regions. Darker areas are achieved by
depositing thicker layers of toner. A higher
voltage applied to the gap between toner
cartridge and drum surface forces more toner
Figure 7.4.1.5 Schematic of the operation of a laser printer onto the drum. The raster memory stores the
Reproduced with permission from Computer Desktop greyscale data for each area of the page and
Encyclopedia, www.computerlanguage.com this is used to set the appropriate voltage level.
By rolling and pressing the rotating drum over a sheet of paper, the toner Key concept
is transferred onto the paper. Transfer may be assisted by using a positively
charged transfer roller on the back of the paper to pull the toner from the Radio frequency identification
(RFID):
surface of the drum to the paper. The paper is passed through heated rollers
Any method of identifying
that squeeze the paper and fuse the toner to the paper. and tracking items using
radio waves. Typically a reader
Questions (also called an interrogator)
7 Why is a laser printer described as a page printer? communicates with a
transponder, which holds digital
8 Explain the operation of a monochrome laser printer. information in a microchip.
Alternatively, a chipless RFID
9 What toners are used by a colour laser printer and why?
tag is used which just uses
material to reflect back a portion
RFID of the radio waves beamed at
them.
Radio frequency identification (RFID) uses radio frequencies (RF) to transmit
data, a timing signal and radio frequency energy if necessary between a reader
(Figure 7.4.1.6(b)) and an RFID device (transponder) as shown in Figure
Information
7.4.1.6(a). RFID tag:
RFID Device A microchip attached to an
Data antenna which is packaged in a
Clock Contactless way that allows it to be applied
RFID
data carrier
Reader Energy to an object such as an item of
(transponder)
clothing for sale. The tag picks up
signals from and sends signals to
a reader.
Coupling element Tags come in many forms such as
(microwave antenna) smart labels which have a barcode
printed on them and tags simply
Figure 7.4.1.6(a) RFID reader and transponder embedded in plastic ready for
attaching to an object.
RFID devices do not need a physical electrical contact to transfer data. Nor do Smart labels are manufactured to
be fed through an RFID printer
they need visible contact as they use radio waves to transfer data. This makes it
that both prints a bar code on
possible to read many codes simultaneously from afar without the need to open the label and writes to its RFID
boxes. microchip.
opened by cats with a recognised RFID tag, but not by other cats to stop these
entering the house). RFID devices are put in the stomach of cattle and remain
there for life.
Barcode scanning versus RFID scanning
RFID scanning allows many RFID tags to be read at a time even when the tags
are hidden inside boxes or behind panels. Barcode scanning scans one item at a
time and the barcode needs to be visible to the scanner.
When stocktaking it is important to avoid counting an item more than
once. RFID tagging and scanning enables each item to be uniquely tagged
and therefore uniquely identified when scanned. When combined with a
timestamp it is very easy to avoid counting an item more than once in a
stocktaking session. Barcodes do not usually identify an item uniquely but
instead encode the product type of an item. Timestamping would be of little
help in preventing an item being counted twice because items are not uniquely
identified.
Questions
10 What is Radio Frequency Identification (RFID)?
Key concept The platter is coated with an emulsion of iron or cobalt oxide (or a cobalt-based
alloy) particles that act as tiny magnets. Binary data is recorded by aligning
Track: these tiny magnets in one direction to represent a binary 0 and in the opposite
One of the concentric rings on a
direction to represent a binary 1. Binary data is recorded in concentric rings, or
platter of a hard disk.
Sector: tracks, subdivided into sectors that hold a fixed number of bytes, such as 512. A
A subdivision of a track. hard disk can store and retrieve a large volume of data.
Disk block:
To read data stored on the hard disk, the read-write head moves to the
The smallest unit of transfer
between a computer and a disk. desired track and waits for the relevant sector to pass beneath it. When data is
A disk block is one sector of a transferred from the hard disk to the computer and vice versa, a whole sector
track. of a track is read or written each time. A whole sector of a track is often called
a disk block or a block. For this reason, a magnetic hard disk drive is known as
Did you know? a block-oriented storage device. The smallest unit of transfer is a block which is
In 2015, the fastest rotation typically 512 bytes.
speeds of consumer disk drives The top and bottom surfaces of a platter may be used to store data. A block
was 10000 revolutions per
address for a single-platter system is composed of a surface address, a track
minute.
address and a sector address. Typically, the surfaces are numbered 0 and 1, the
Key point tracks 0 to 7,000 and the sectors 0 to 63. Figure 7.4.2.2 shows a schematic for
one surface of a magnetic hard disk.
Disk buffer:
Executing programs do not write Stepper
motor
directly to magnetic hard disks.
Instead, they write to an area Read-write
of main memory (RAM) called head
a disk buffer. Before a program
can write to a file, it has first
Sector Track
to open the file, if it exists, or
create the file if it doesn’t. This
open/create action creates a disk
buffer in main memory (RAM)
which is then associated with the
Disk block
file. The program writes to this (one sector
buffer. When the buffer becomes of a track)
full or the program closes the
corresponding file, the operating
system writes the buffer to disk.
The size of the buffer matches the N- Platter
S
size of a disk block or a multiple S-N
N-S
of this. S-N
To read a file it must first be Part of one
opened. This creates a disk buffer sector
which receives a block at a time Magnetic particle of a track
each encoding a 1 or 0
belonging to the file. The program
that opened the file then reads Figure 7.4.2.2 Hard disk platter showing concentric tracks and sectors
from this buffer. When the buffer
becomes empty, the operating Modern hard disks for a PC system are sealed units, called Winchester disks,
system transfers the next disk
containing several platters mounted on a common spindle. The platters are
block belonging to the file into
this buffer. sealed inside an assembly which allows the disk to operate with minimal risk of
damage from contaminants. The read-write heads are built into the assembly Key concept
with one head per surface. The greater the number of platters, the greater the Access time:
storage capacity. Access time for a magnetic hard
disk is the time interval between
Questions the moment the command is
given to transfer data from disk to
2 Why is a magnetic disk drive known as a block-oriented storage device?
main memory and the moment
3 In the context of a magnetic disk, what is (a) a track, (b) a sector, this transfer is completed.
It is made up of three
(c) a disk block and (d) disk block address?
components:
4 Explain the principle of operation of a magnetic disk drive. 1. seek time
2. rotational delay
5 What effect do you think that having smaller platters and faster 3. data transfer time
rotation speeds will have on the time taken to read a disk block?
Seek time:
The time it takes for the read/
Optical disc write heads to align on the desired
track.
An optical disc is a flat, usually circular disc which encodes binary data (bits)
Rotational delay:
in a special reflective layer. In one form of optical disc, binary data is encoded The time taken for the desired
in the form of pits (binary value of 0 due to lack of reflection when read) and sector to come under the read/
lands (binary value of 1 due to a reflection when read) on a reflective material, write head. On average, this is the
usually metallic, on one of its flat surfaces as shown in Figure 7.4.2.3. time taken for half a revolution of
the disk platters. This average is
CD-ROM called the latency of the disk.
The success of compact discs (CDs) for storing audio led to a new format, CD Data transfer time:
Read-Only Memory (CD-ROM). Introduced early in 1985, this format was The total time taken to read the
disk block and transfer it into
initially used to publish encyclopedias, reference works, professional directories
main memory
and other large databases. CD-ROMs were ideal for this because they had (for
the time) a high storage capacity of
600–700 million bytes, offered fast Magnified view
showing pits in
data access and were portable, rugged reflective metal
layer
and read-only. Today, CD-ROMs are
also used for software distribution.
The data is written on the discs
Label Protective
using disc-mastering machinery that
layer
impresses pits (physical depressions)
into a continuous spiral track. The
Reflective
silvery data surface contains pits in a 1.2 mm metal layer
thick
single track 3.5 miles (5.6 km) long.
The disc spins at 200-500 revolutions
per minute depending on which part
Polycarbonate
of the track is being read. disc
Key fact More laser light is reflected from the unpitted surface than from the
Optical disc: pitted surface. This is detected by a photodiode that outputs an equivalent
An optical disc is a flat, usually circular electrical signal. After some conditioning, the result is a digital signal
disc which encodes binary data (bits) in representing a single data bit. This data bit encodes the amount of
a special reflective layer. In one form of
reflection as 0 or 1.
optical disc, binary data is encoded in the
form of pits (binary value of 0 due to lack Did you know? Questions
of reflection when read) and lands (binary Blue-ray disc:
6 What is an optical disc?
value of 1 due to a reflection when read) A Blu-ray disc (BD) is a high-
on a reflective material, usually metallic, on density optical disc capable of 7 Explain how information is
one of its flat surfaces. storing 23.3 GiB (25 GB) in a recorded and then read from a
single-layer which is consider-
CD-ROM.
ably more than a DVD can
store.
Did you know?
CD-R:
Write Once, Read Many (WORM) times Solid-state disk (SSD)
optical disc. The solid-state disk (SSD) in a solid-state disk drive (Figure 7.4.2.4)
CD-R can record about 650 - 900 MiB operates by trapping electrons in a wafer of semiconducting material.
of data
These electrons and their electric charge remain trapped even when
CD-RW:
CD-ReWritable disc that can be read and electric power is removed, i.e. SSD is non-volatile storage. Binary 0 is
written to over and over again. represented by trapped electrons and binary 1 by absence of trapped
DVD-ROM: electrons.
Digital versatile disc or digital video disc
(DVD) is an optical standard offering The sites
much greater storage capacity than CDs. (floating gate
Storage capacity of a single-layer DVD- transistors)
ROM is 4.3 GiB (4.7 GB). where these
DVD-R:
electrons are
DVD-R is a WORM format similar to
trapped are
CD-R.
DVD-RW: organized in a
The DVD-RW format provides a grid. The entire
rewritable optical disc with a typical grid layout is Figure 7.4.2.4 Solid-state disk drive
capacity of 4.3 GiB (4.7 GB).
referred to as a ©D-Kuru/Wikimedia Commons
DVD+RW:
A competing rewritable format to
block, while the
DVR-RW. individual rows that make up the grid are called a page.
DVD-RAM:
Common page sizes are 2KiB, 4KiB, 8KiB, or 16KiB, with 128 to 256
DVD-RAM is a rewritable format
that has built-in error control and a
pages per block. Block sizes are typically between 256KiB and 4MiB. For
defect management system, so it is example, the Samsung™ SSD 840 EVO has blocks of size 2MiB, and
considered to be better than the other each block contains 256 pages of 8 KiB each. The Samsung SSD 840
DVD technologies for tasks such as data EVO comprises 8 NAND flash chips, each of capacity 64 GiB. Each
storage, backup and archiving.
Samsung flash chip contains 32 blocks.
Blu-ray disc:
A Blu-ray disc (BD) is a high-density Unlike magnetic disk drives, solid-state drives contain no moving parts
optical disc capable of storing 23.3 or spinning disks. The absence of moving parts means that solid state
GiB (25 GB) in a single-layer which is
disk drives can operate at speeds far above those of a typical hard disk
considerably more than a DVD can store.
drive.
Access time for a typical hard drive is on average 10-15 milliseconds whereas access time for an SSD drive is 25-100
microseconds (access time for RAM is typically 40 -100 nanoseconds).
The technology used is NAND flash memory which is a type of EEPROM (Electrically Erasable Programmable
Read Only Memory). A solid-state disk is a block-oriented storage device which has to erase a block first in order
to rewrite it because unlike magnetic hard disk drives, NAND flash memory can’t overwrite existing data. Erasing a
block in the SDD means “untrapping” electrons.
The solid-state disk drive requires an onboard controller which consists of an embedded microprocessor with RAM
buffer to perform reading and writing to the solid state disk (Figure 7.4.2.5). The controller is a very important
factor in determining the speed of the SSD drive.
SSD SSD
Controller PCB
SATA NAND
Interface Flash
Memory
Figure 7.4.2.5 SSD drive printed circuit board (PCB) showing the controller and the NAND flash memory
chips (image reproduced with kind permission of StorageReview.com). Information
To alter the contents of a particular memory location of SSD storage, an entire SSD vs other flash-based
devices:
block must be constructed containing the new information and written to SSD.
SSDs are much faster than any
The controller arranges for this new block to be written to a different area of SSD. of the other flash-based portable
The reason for this is that SSD blocks can be programmed for only a limited drives, e.g. USB memory stick.
amount of time before they become unreliable. This is known as write-endurance.
It is measured in number of program erase (P/E) cycles. To lessen this effect, a
controller uses a technique called wear-levelling, which effectively makes sure that all the drive’s memory chips are
used, cell by cell, before the first cell is written on again.
SSD secondary storage is increasingly being used in laptops, tablets and is an option now for desktop PCs. The
attraction is lower power consumption and faster booting of the operating system.
Questions
8 Explain the operation of a solid-state disk drive.
9 Give four reasons why a solid-state disk drive might be preferred to a magnetic disk drive.
CD-ROM, CD-R, CD-RW 650 - 900 MiB CD-ROM 150 CD-R 1.76 MiB/s (12x)
KiB/s (1x) to CD-RW 1.46 MiB/s
6,750–10,800 KiB/s (10x)
(72x)
DVD-ROM, DVD-R, DVD±RW, single layer: 4.3 GiB 1,353 KiB/s (1x) to 1,353 KiB/s (1x) to
DVD-RAM dual layer: 8.5 GiB 21,648 KiB/s (16x) 21,648 KiB/s (16x)
Blu-ray Disc (BD-ROM) single layer: 23.3 GiB 4.3 MiB/s (1x) to BD-R / BD-RE 50 MiB/s
Blu-ray Disc Recordable (BD-R) dual layer: 46.6 GiB 68.66 MiB/s (16x) (12x)
150,000 IOPS
Table 7.4.2.1 Capacities and speed of access for various storage media
Information
CD drives:
1x means 150 KiB/s
DVD drives:
1x means 1,353 KiB/s
Table 7.4.2.2 Storage media and typical applications to which they can be put
Questions
10 State what applications each of the following might be used for:
(a) magnetic hard disk drive (b) CD-ROM (c) DVD-R
(d) DVD-RAM (e) blu-ray disc.
Learning objectives:
Case study continued
■■Understand that: After their rescue on July 29th and arrest after returning to England
• the issue of scale, for Dudley and Stephens went to trial and Brooks turned state’s witness.
software the whole world Defending killing Parker before his natural death they argued that his
over, creates potential blood would be better preserved for them to drink. They freely confessed
for individual computer to killing and consuming Parker but claimed they had done so out of
scientists and software necessity. Parker was an orphan without dependents whilst Dudley and
engineers to produce great Stephens had families that depended upon them.
good, but with it comes the At the time maritime law was not clear cut on such cases and in such
ability to cause great harm circumstances the custom of the sea applied as practised by the officers
and crew of ships and boats in the open sea.
■■Be able to discuss the
challenges facing legislators in
the digital age
The case of the Crown versus Dudley and Stephens in 1884 was important for
two reasons. Firstly, it established a precedent, throughout the common law
world, that necessity is not a defence to a charge of murder. Secondly and more
importantly, it highlighted that morality is more than a matter of cost-benefit
analysis and calculating consequences but has more to do with the proper way
for human beings to treat one another. Morality implies certain moral duties
and human rights so fundamental that they rise above a matter of simply
calculating consequences.
The Global Information Society and a new Digital Ethics Key principles
Society is shaped by the technology at its disposal:
Protection of personal data:
“The handmill gives you society with the feudal lord; the steam-mill the society EU Charter of Fundamental
with the industrial capitalist.” Karl Marx Rights Article 1 defines the
right of an individual to
The information society that we live in today has been shaped by the protection of their personal data
information and communication technologies of the global networking of the via data protection principles
digital computer. the following principles
-necessity, proportionality,
“The digital computer and networking give you the Global Information fairness, data minimisation,
Society” purpose limitation, consent and
transparency.
Whilst previous societies have evolved relatively slowly, the Global Information
These apply to data processing
Society has burst into existence in a relatively short period of time by in its entirety, to collection as
comparison. Berkeley’s School of Information Management and Systems well as to use.
estimated that humanity had accumulated approximately 12 exabytes of data
in the course of its entire history before the advent of the desktop digital Key concept
computer. Since then there has been an explosion in data. For example, in just Personal data:
Data about any living person.
one year, 2002, more than 5 exabytes of data were produced and recorded.
According to IDC, in 2013 the digital universe contained some 4.4 zettabytes, Task
a thousand-fold increase on 2002, roughly. Whilst bringing enormous benefits Data Protection Act
and opportunities, the information revolution’s enormous growth rate has 1. Look at the eight principles
posed a conceptual, ethical and cultural challenge of how to put in place a of the Data Protection Act
1998 at https://ico.org.uk/
viable philosophy and ethics of information.
for-organisations/guide-to-data-
The European Data Protection Supervisor (EDPS), an independent institution protection/ and see if you can
of the EU, published a report in 2015 entitled match these to the EU Charter
principles.
“Towards a New Digital Ethics”
2. Look at Regulation (EU)
(https://secure.edps.europa.eu/EDPSWEB/webdav/site/mySite/shared/ 2016/679 which applies to all 28
Documents/Consultation/Opinions/2015/15-09-11_Data_Ethics_EN.pdf ) EU countries from 25 May 2018
and Directive (EU) 2016/680
At its core is the protection of human dignity: which requires member states
“An ethical framework needs to underpin the building blocks of this digital to enact legislation by 6 May
2018.
ecosystem. The EDPS considers that better respect for, and the safeguarding
of, human dignity could be the counterweight to the pervasive surveillance and Key concept
asymmetry of power which now confronts the individual. It should be at the Asymmetry of power:
Computers do not only carry
heart of a new digital ethics.”
out actions using information
Asymmetry of power and pervasive surveillance in the form of programmed
Asymmetry of power instructions but they can also
produce information that
Coded algorithms automate operations or actions according to a logic that
coded algorithms can capture
differs little from that which humans have applied for centuries. Replacing the and report. However, whilst
human brain by a digital computer enables more consistency, greater speed you have some control over
of calculation and more control. However, it adds an additional dimension the former in that you have
that is not possible with a human agent: computers do not only carry out initiated the action, e.g. a
search, you seemingly have
actions using information in the form of programmed instructions but they
much less control over the
can also produce information that coded algorithms can capture and report. latter.
Key concept “Information Technology alone has this capacity to both automate and reflect
information (informate)” - Professor Shoshana Zuboff. However, whilst you
Machine learning: have some control over the former in that you have initiated the action, e.g.
Machine learning is the
a search, you seemingly have much less control over the latter (e.g. collecting,
science of getting computers
to act without being explicitly storing and associating with you information about what you are searching for)
programmed. This approach and certainly many people do not perceive that the latter is happening behind
enables patterns in and links the scenes. This is what is meant by asymmetry of power.
between data to be discovered
autonomously once the Visiting a particular website results in your visit being logged, tracked with
machine has been “trained” on a cookies and then combined with information already held about you to send
large enough dataset. Examples you personalised advertisements. You can see what is happening behind the
are speech recognition and
scenes using packet capture software such as Wireshark. For example, visiting
language translation software,
the New York Times’ website results in the communication of information to
and self-driving cars.
the following other sites (as of October 2015):
https://www.doubleclickbygoogle.com/
Key concept http://www.conversantmedia.com/
Information: http://www.facebook.com
Data is how information is
Pervasive surveillance
represented.
The logic in automation is the logic of action, i.e. do this followed by do that,
whereas the logic of information reflection is the logic of accumulation.
Key concept The latter has given rise to huge datasets labelled Big Data (see Chapter 11.1)
and the impetus to analyse large datasets using machine learning techniques.
Knowledge:
Knowledge is usable Contributing to this Big Data phenomena in a major way is the capturing of
information. small data from individuals’ online actions and utterances as they go about
their daily lives. Nothing is too trivial or unimportant for this data harvesting,
from Facebook ‘likes’ and clicks on links, to smartphone location data.
Key concept
Such data (representation of information) is aggregated, analysed, packaged and
Pervasive surveillance: sold.
Capturing of small data from
These data flows have been labelled ‘data exhaust’ because they are the
individuals’ online actions and
utterances as they go about their byproduct of users’ actions as they go about their daily business interacting
daily lives. These data flows have with computer systems. Google and Facebook are among the largest and most
been labelled ‘data exhaust’. successful Big Data companies because they sweep up this data exhaust. In
2015, Google’s search engine was the most visited engine and Facebook the
most visited social media site.
Although Google started out with no intention of offering advertisers space on
their search results’ Web pages, they eventually gave in to the need to generate
revenue via an advertising model rather than a fees-for-service one because the
latter might have impacted on the expansion of their user base. This advertising
approach depended upon the acquisition of personal data as the raw material
that after analysis and application of machine learning would sell and target
advertising through a unique auction model reliant upon the accumulation of
huge quantities of personal data to make it work with increasing precision and
success.
Google’s business is thus the auction business and its customers are advertisers. Information
AdWords, Google’s algorithmic auction method for selling online advertising,
What They Know, is an in-depth
analyses massive amounts of data to determine which advertisers get which one
investigative series by the Wall
of eleven sponsored links on each search results page (http://archive.wired.com/
Street Journal. It found that one
culture/culturereviews/magazine/17-06/nep_googlenomics?currentPage=all). of the fastest growing Internet
business models is of data-
Case study gatherers engaged in intensive
surveillance of people visiting
In 2009 Google published a research paper in the prestigious scientific websites in order to sell data
journal Nature that described a method to track influenza-like illness in about, and predictions of, their
a population by analyzing large numbers of Google search queries. The interests and activities, in real
researchers reported that they could accurately estimate the current level time.
of weekly influenza activity in each region of the United States, with a http://juliaangwin.com/the-what-
reporting lag of about one day. they-know-series/
(http://static.googleusercontent.com/media/research.google.com/en//
archive/papers/detecting-influenza-epidemics.pdf )
“This seems like a really clever way of using data that is created Information
unintentionally by the users of Google to see patterns in the world that
would otherwise be invisible,” said Thomas W. Malone, a professor at the Using Big Data to track
Sloan School of Management at MIT. population movements in an
Shortly after its publication, H1N1, a new strain of a particularly virulent Ebola infected country
strain of flu hit the United States. In order to track and contain the http://www.pri.org/
stories/2014-10-24/how-big-data-
outbreak before it became pandemic, the Centers for Disease Control and
could-help-stop-spread-ebola
Prevention (CDC) requested that doctors inform them of new flu cases.
Unfortunately, this reporting at best involved a two-week lag, too long to
enable effective control of the outbreak.
The pacemaker was specially 5 “Surrendering personal data for perceived benefits is not a square
adapted for Dick Cheney so deal because I cannot control all the ways that my personal data
that it would be resistant to
will be used.”
hacking and disruption. This
must be very reassuring to the
6 “I am powerless to stop my personal data being used for purposes
rest of the population which has
that I am unaware of because the services that I use for free in
to make do with pacemakers
that are vulnerable to hacking. exchange for surrendering my data are an integral part of my life.”
Task
Questions for discussion
Cookie Law:
Find out about Cookie law which
10 Do you think that one country should have the authority to
was adopted by all EU countries
control what content someone in another country can access on
in May 2011 and required an
update in the UK to the Privacy the Internet? Justify your opinion.
and Electronic Communications
Regulations. 11 In respect of the Internet, should the reach of the law for each of
the following apply (i) globally or (ii) locally with each country
deciding what law to apply
Software and their algorithms embed moral and cultural values Background
Any artifact that interacts with human beings and that is able to change the
In Egypt on January 25th, 2011
dynamics of social processes is not value free. a Facebook page was posted
Social processes are the ways in which individuals and groups interact, adjust calling for a day of protest after
an incident in which a 28 year-
and re-adjust and establish relationships and patterns of behaviour.
old man was dragged from an
To be value free means to be without bias or to use criteria that do not reflect Internet cafe and beaten to death
prejudice or cultural attitudes. by Egyptian police officers. When
a large number of people signed
Cultural attitudes reflect the culture of a society and are expressed through the up for the protest on Facebook
ideas, customs, and social behaviour that are held to define the society. British the Egyptian government shut
culture, for example, means to act fairly and justly; respect for the right of free down Egypt’s Internet and cellular
expression; respect for the rule of law and the democratic process; respect for phone service. Three weeks
later, on February 11th, Egyptian
a free press (free from Government control); the right to protest in an orderly
President Hosni Mubarak was
manner; and much more. Other cultures place a different emphasis on free driven from office after 30 years
speech, etc. in power.
Email was the first killer application built on top of the Internet. It changed
a major social process dramatically, i.e. the way that we communicate, but
Background
if you were not digitally connected then you could not use it. Although this The built-environment:
is less true today there are sectors of the world’s population who are unable The environment that consists of
buildings, roads and railways.
to communicate via email because they do not have access to the necessary
resources.
Case study
The architecture of a place functions as a form of regulation; it constrains the behaviour of those who interact
with it, often without them even realizing it. Build a bridge so low that buses cannot pass under it then it is
possible to exclude people from travelling to what lies beyond the bridge, if economic circumstances mean that
buses are their only means of travel. If these circumstances are associated with lack of education and therefore
employment opportunities then it becomes a form of segregation, all the more worse if the lack of education
opportunities and employment are linked to ethnicity.
The architecture of a place is the way it is because of how the built-environment was designed. The design is
not value free if it has been based on criteria which are subjective, i.e. not objective. If the design was objective
then the object of connecting one side of the bridge to the other would have applied equally to both buses and
cars.
13 Generally speaking, securing privacy by controlling access is an inalienable right. After all privacy is a
human right. What is morally wrong then with applying design to public places in a way that affects some
peoples’ access but protects other peoples’ right to privacy?
14 Can you imagine a situation where the designers set out with the intention of creating fair access but are
prevented from doing so by circumstances beyond their control?
From your discussions you may have concluded that the architecture of the
Background
built-environment can be said to embed moral and cultural values, i.e. it is
The algorithms of the Internet not value free.
allow data about people to be
collected in the following ways: This is also true of the Internet, the World Wide Web and other applications
• Volunteered data – created and
built on top of the Internet such as social media because each of these has
explicitly shared by individuals, been designed. Each has a particular architecture shaped by its designers in its
e.g. social network profiles. hardware, software, and its algorithms, none of which can be value free because
• Observed data – captured design decisions are always taken which inevitably embed moral and cultural
by recording the actions of
values. Thus the design and operation of the Internet, the World Wide Web
individuals, e.g. location data
when using cell phones.
and other applications that rely on the Internet must also raise questions of
• Inferred data – data about justice, fairness and democracy.
individuals based on analysis
This matters even more because there are major differences between the built
of volunteered or observed
information, e.g., credit scores.
environment and the architecture of the Internet, World Wide Web and these
other applications.
• The first is one of scale, the Internet’s reach is global whereas that of
public space is local.
• Secondly, the Internet, World Wide Web and other Internet-based
applications embody not only the logic of action, i.e. automation,
but also the logic of information reflection or accumulation which
facilitates a form of control.
• Thirdly, the distinction between hardware and software is blurred
because cloud computing effectively delivers physical hardware
packaged as software i.e. a virtual machine running the operating
system of your choice and virtual storage such as DropBox. Users
can gain access in minutes to virtualised, scalable hardware resources
(e.g., Amazon’s Elastic Cloud) which obviates the need to purchase
the equivalent physical hardware. As there is no physical hardware
Information to dismantle, terminating or halting such a provision also takes only
Cloud service providers: minutes.
Amazon Web Services -
Cloud computing also decouples the physical possession of data from their
https://aws.amazon.com
ownership. It means dealing now with the issues of ownership of virtual assets
Google Cloud -
https://cloud.google.com and access to those virtual assets, i.e. the right to ownership and the right
Microsoft Azure: to usage. For example, if your personal data is aggregated and subjected to
https://azure.microsoft.com machine learning algorithms that derive new information about you, who owns
this new information and who has the right to access this information?
Case study
There are a number of large gateways into China through which Internet access for Chinese citizens
is controlled. The Chinese telecom companies that control these gateways are required by the Chinese
government to configure their routers to use DNS servers that screen and filter out content that the Chinese
government objects to because, according to Western opinion, of a fear of losing control to forces other than
the Chinese government. While the primary purpose of routers is to direct or route Internet traffic to its
correct destination, they can also be configured in this way to block content and thereby prevent information
from getting to its destination. Routers can also be configured to block access to websites by their URL, e.g.
Twitter.com and forbidden web page content by inspecting the packets of information, any content that
mentions Tiananmen Square. The router, DNS server, and the software they use become a censor in these
circumstances.
Background communicate more safely with whistleblowers and dissidents. It has proved
to be an effective tool to circumvent censorship of the Internet in countries
Deep Web DVD: such as China and Iran and as a building block for other software designed to
Now available to purchase from
protect privacy.
www.amazon.com.
URL: http://www.amazon.com/ However, the Tor software has also been used to hide criminal activity and
Deep-Web-Directors-Keanu- other undesirable activities. A question mark also hangs over whether Tor is
Reeves/dp/B017WUEJ52
secure against monitoring by governments as it was originally developed, built
and financed by the US military which released it for general use in 2004,
ostensibly to improve the cover of its spies overseas by masking their Internet
activities amongst the activities of a diverse group of people to whom the Tor
Information software was now available. If it is not secure against government monitoring,
Banning use of free or shared it might be difficult to know because the authorities would be careful about
WiFi and Tor: revealing their hand.
In December 2015 in the wake of
the Paris attacks, it was reported
that France’s law enforcement
Case study
authorities were proposing new
The Dark Web is the World Wide Web content that exists on
legislation to forbid the use of
free or shared WiFi during a
underground networks which use the public Internet but which require
state of emergency. They were specific software such as Tor for access. The Dark Web forms part of
also proposing that anonymous the Deep Web, the part of the Web not indexed by search engines. In
browsers like Tor should be the Deep Web there are several online shopping sites that specialise in
blocked in general.
connecting sellers of illicit goods with willing buyers.
The online black market Silk Road launched in February 2011 used
the anonymising tool Tor to protect the identities of buyers, sellers
and the site’s administrators. Payment was made in Bitcoin, allowing
buyers a relatively high amount of protection. Ross William Ulbricht, its
Background
creator, was arrested in October 2013. On May 29, 2015, Ulbricht was
Bitcoin: handed five sentences to be served concurrently, including two for life
Bitcoin is a decentralized digital
imprisonment, without the possibility of parole. He was also ordered to
currency or virtual currency. It
uses peer-to-peer technology to forfeit $183 million obtained from his criminal activities that he held in
operate with no central authority Bitcoins, beyond the reach of the authorities until he was forced to hand
or banks. over the encryption keys. In a letter to Judge Forrest before his sentencing,
Ulbricht stated that his actions through Silk Road were committed
through libertarian idealism and that “Silk Road was supposed to be about
giving people the freedom to make their own choices” and admitted that
he made a “terrible mistake” that “ruined his life”.
Watch the film Deep Web by Alex Winter, if you can, which explores how
the designers of the Deep Web and Bitcoin are at the centre of a battle
for control of how the Internet may be used and its effect on our digital
rights.
19 In 2014 Google launched a service in response to a European Court ruling to allow Europeans to ask
for personal data to be removed from online search results.
(a) Why might removal of personal data from search engine results not be sufficient to comply with
the right to be forgotten?
(b) If on the grounds of freedom of speech it is a human right to be able to access information,
why might forcing people to conduct searches with engines that are able to access the Deep Web be
undesirable?
20 “Software such as Tor should be banned because although it can be used for morally sound purposes
it can also be used for criminal and immoral acts.” Do you agree or disagree with this statement and
why?
Questions
4 Social media and search engine companies collect personal data from users to make their services more
useful to users. Explain why it can be beneficial for the user to give up personal data in this way and
why it might not.
5 Software can produce great good but with it comes the ability to cause great harm. Using at least one
example, explain the meaning of this statement.
6 Software and their algorithms embed moral and cultural values. Using at least one example, explain
the meaning of this statement.
7 Why do legislators face difficulty enacting legislation where the Internet is concerned?
8 Developments in computer science and digital technology have dramatically altered the shape of
information flows in society. The labels asymmetry of power and pervasive surveillance have been applied
to these flows. What is the meaning of each of these labels?
9 Processes involving humans communicating with machines have been replaced by machine processes,
i.e. machine to machine communication with machines making decisions and taking actions
according to the code (automated algorithms) that they are programmed with. Machines in these cases
could be embedded computers or nodes in a network such as the Internet, for example. Describe two
examples of applications where machine to machine communication is relied on substantially.
Software engineers and computer scientists have responsibilities in the algorithms that they devise and
the code that they deploy. What precautions should be observed by computer scientists and software
engineers in the development of the two applications that you have described?
9.1 Communication
Learning objectives:
■■Define serial and parallel
transmission methods and
■■ 9.1.1 Communication methods
discuss the advantages of serial Serial data transmission
over parallel transmission In serial data transmission, single bits (binary digits) are sent one after
another along a single wire by varying the voltage on the wire. Figure 9.1.1.1
■■Define and compare shows a simple electrical circuit for sending single bits coded as 0 volts and 5
synchronous and asynchronous
volts. When the switch is in position A the lamp bulb is connected to 5 volts.
data transmission
When the switch is in position B the lamp bulb is connected to 0 volts. We
■■Describe the purpose of start need to decide what the signal lamp on and the signal lamp off represent.
and stop bits in asynchronous
A
data transmission Signal wire
Key principle 5V
Switch X Lamp bulb
Questions
1 What is serial data transmission?
0 bit 0 0
1 bit 1 1
2 bit 2 2
3 bit 3 3
Sending Receiving
parallel 4 bit 4 4 parallel
interface 5 bit 5 5 interface
6 bit 6 6
7 bit 7 7
CLK CLK
GND GND
Figure 9.1.1.3 Parallel data transmission along an 8-bit data bus, controlled
by a clock pulse to signal the arrival of 8 data bits
Questions
2 What is parallel data transmission?
the more data bits on each wire can get out of step with each other and the
clock signal.
Crosstalk is induced signals in adjacent wires of a parallel bus caused when a
signal on one or more wires varies rapidly. The longer a wire and the more rapid
the variation in voltage in adjacent wires the greater the effect.
Serial data transmission doesn’t suffer from skew because it doesn’t use a
separate clock signal and crosstalk is minimised because there are fewer wires
in close proximity and techniques can be applied relatively cheaply to guard
against crosstalk. It is also considerably cheaper over long distances than parallel
would be, simply because fewer wires are used.
A parallel interface is simpler to design than a serial one. The parallel interface
just requires a buffer of the same width as the data to be transmitted or
received. A serial interface must perform a parallel to serial conversion and
vice versa and so is a little more complicated to design than a parallel interface.
However, a parallel interface requires more pins than a serial one.
Questions
3 State two disadvantages of parallel data transmission when compared
with serial data transmission.
Figure 9.1.1.4 shows the three pin and three wire interface for I2C. The data
bits D7 to D0 are individually clocked into the receiving interface by 8 clock
pulses. The receiving I2C interface places an acknowledgement bit onto the
data wire on receipt of the 9th clock pulse.
Whatever the type of synchronous serial interface, a clock signal is associated
with its data
Serial Serial
line(s) and this
interface A interface B
clock signal is RX RX
used by all the
Information
devices connected TX TX
Serial ports:
Desktop PCs used to have at least
to the serial bus Serial
one serial port called the COM to synchronise all bus
port but the advent of USB meant data transfers. GND GND
manufacturers stopped supplying
PCs with serial ports. They are
Another example
absent from tablets and smart of a synchronous Figure 9.1.1.5 Asynchronous serial data transmission
phones where the space for ports interface used between two interfaces supporting transmit (TX) and
is limited. However, the demise of by embedded receive (RX) in both directions simultaneously
serial ports is greatly exaggerated.
microcontroller
Serial ports are everywhere,
in everything from industrial
systems is the Serial Peripheral Interface bus (SPI).
automation systems to scientific Synchronous serial communication is used in telecommunication systems
instrumentation and in Internet of
which the Internet relies upon. These are time-division systems which
Things (IoT) devices. The Arduino
has at least one TTL serial port
continuously send frames of bits between nodes. The nodes are kept in sync
for transmitting and receiving by synchronisation frames which are sent periodically and which distribute the
serial data. common clock signal derived from an atomic clock. This enables very high data
transfer rates not achievable with asynchronous data transfer.
Asynchronous means that data is transferred without support from an external
clock signal. No clock wire is required so reducing the number of wires by one
and the number of connecting pins in each interface by one as well.
Figure 9.1.1.5 shows a serial interface A connected to another serial interface
Key concept B by a serial bus consisting of three wires, one to transmit from interface A to
interface B and one to transmit from interface B to interface A. Both interfaces
Asynchronous data
transmission: share a common ground wire.
Asynchronous means that data Asynchronous serial data transmission sends 7 or 8 data bits at a time and an
is transferred without support optional parity bit framed by start and stop bits in the RS232/RS422 protocol
from an external clock signal.
as shown in Figure 9.1.1.6. These 7 or 8 bits often represent character data
No clock wire is required.
from the ASCII character data set.
Parallel
Data bus RX
Serial
D4
clocking it out onto the serial TX wire framed in start and stop TX
D5
bits. The UART performs serial to parallel conversion and vice D6
D7
versa. If the parity bit system is being used, it also generates a
Read/Write
parity bit as well as checking the received data for parity errors. Control bus Clock
Interrupt request
Questions
Figure 9.1.1.7 UART
4 Define the following modes of data transmission
(a) synchronous
Information
(b) asynchronous
Ethernet and USB:
These are serial data transmission
protocols that both encode timing
Comparison of synchronous and asynchronous data information in the communication
transmission symbols, i.e. the bit patterns or
codes that they use. This coding
RS232/RS422 asynchronous serial data transmission is relatively cheap because
therefore delivers both a clock
it requires less hardware than synchronous serial data transmission and is
signal and data, so both can be
appropriate in situations where messages are generated at irregular intervals, for classified as synchronous.
example from embedded systems used in scientific instruments as initiation of
a transfer is relatively quick. However, two in nine or ten bits are control bits
thus a significant proportion of the data transmission conveys no information.
Questions
5 Compare synchronous and asynchronous data transmission.
Key concept the time of transmission. This allows data to be transmitted intermittently such
as when typing characters on a keyboard.
Start and stop bits:
The start bit signals the arrival of Start bit
data at the receiver. This enables The arrival of data at the receiver is signalled by a special bit called a start bit.
the receiver to sample the data
As the arrival of data cannot be predicted by the receiver, the transmission is
correctly by generating clock
pulses synchronised to the bits
called asynchronous.
in the received data. Effectively, The start bit is used to wake up the receiver. The receiver’s clock is set ticking
transmitter and receiver clocks
by the start bit. The clock is just a circuit that is designed to generate a preset
are synchronised by the arrival
number of pulses at fixed intervals which are used to sample the data bits as
of a start bit.
they arrive one after another. The signal changes in the serial data transmission
Both transmitter clock and take place at regular time intervals so the receiver must operate a timing device
receiver clock must have been set at the same rate as the transmitter, so the received bits can be read at the
set up previously to “tick” at the
same regular time intervals.
same rate when running.
Other parameters must agree in The transmitter also operates a timing device, a clock, that is set at a rate
both transmitter and receiver determined by the baud rate (see Chapter 9.1.2). Again this can be as simple a
interface such as the number
circuit to generate a preset number of pulses at fixed intervals of time.
of data bits that will be sent,
the number of stop bits and It is important that the receiver reads each bit during the time that it is not
whether parity is used or not changing, i.e. in the time interval between when changes can take place. This
and if so, what parity, even or
is why the receiver’s timing device needs to be brought in step or synchronism
odd. These parameters are set
with the transmission’s timing.
up once before either interface
is used. Figure 9.1.1.8 shows two computers, A and B, with a TTL serial connection
between their serial ports. TTL is Transistor-Transistor Logic and is a logic that
operates between 0 volts and 5 or 3.3 volts. TTL relies on circuits built from
bipolar transistors to achieve switching and maintain logic states.
For the link from computer A to computer B, the data wire is kept at the
voltage level corresponding to a binary digit 1 when not sending – the
idle state. A data
transmission is started
Computer A Pulse train travelling along single wire Computer B
between A and B by changing the
voltage level
5 volts Idle Stop Parity LSB Idle state
the most significant data bit (MSB) last. If parity is enabled, the last data bit is
followed by a parity bit.
Stop bit
Finally, the transmitter attaches a stop bit. The voltage level chosen for the stop
bit is the level for binary digit 1. The time interval for the stop bit allows the
receiver (RX) to deal with the received bits, i.e. transfer them into the RAM of
the computer, before receiving and processing the next serial frame as shown in
Figure 9.1.1.9. Two stop bits are used if the receiver needs more time to deal
with the received bits.
0 Time
1101011001010011100111
t t
le dle tar it 0 it 1 it 2 it 3 it 4 it 5 it 6 it 7 top tar it 0 it 1 it 2 it 3 it 4 it 5 it 6 it 7 top
Id I S b b b b b b b b S S b b b b b b b b S
Questions
9.1 Communication
Learning objectives:
■ Define: ■ 9.1.2 Communication basics
• baud rate
Baud rate
• bit rate The baud rate sets the maximum frequency at which signals may change.
To understand the meaning of baud rate consider the following simplified
• bandwidth
switching system shown in Figure 9.1.2.1. Switch A operates at the baud rate.
• latency For example if the baud rate is 1 baud then the switch remains connected to
one of the four electrical voltages for one second. At the end of each second,
• protocol
the switch can switch the connection to any one of the four possible voltages.
■ Differentiate between baud The changeover happens in a time period very much smaller than one second
rate and bit rate (theoretically, it happens instantaneously). Thus, the output signal that appears
■ Understand the relationship on the signal wire may change every one second but no quicker. If the baud
between bit rate and rate is 10 baud then the switch can change position every tenth of a second.
bandwidth Switch A
7.5 7.5
5.0 5.0
Key concept 2.5 2.5 volts
Table 9.1.2.1 Relationship between baud rate and rate of signal changes
Questions
1 How often may switch A change if the baud rate is 100 baud?
2 How many times a second can the switch change if the baud rate is 1000 baud?
Table 9.1.2.2 Linking signal levels and number of bits they encode
Figure 9.1.2.2 shows how two bits of data can be encoded per time slot on a 1
baud line, giving a bit rate of 2 bits per second.
11 00 10 01 10 11 01 10
7.5
Voltage/V
5.0
2.5
0.0 Time/s
0 1 2 3 4 5 6 7 8
Figure 9.1.2.2 Sending data along a 1 baud line using
four levels of voltage to encode bits
From this we may conclude that the relationship between but rate and baud
rate is
bit rate = baud rate x the no of bits per signal (voltage)
Questions
3 What is meant by (a) baud rate (b) bit rate?
4 Explain the difference between baud rate and bit rate.
6 For the voltages given in question 5 write down one possible set of
corresponding bit patterns (an example of a bit pattern is 01).
7 If the baud rate of the line is 900 baud what is the bit rate for the
voltage levels given in question 5?
Bandwidth
Key concept
Bandwidth is a measure of how fast the data may be transmitted over the
transmission medium. The greater the bandwidth, the greater the rate at which Bandwidth:
data can be sent. The bandwidth of a
transmission medium, e.g.
The bandwidth of a transmission medium, e.g. copper wire, is the range of copper wire, is the range of
signal frequencies that it may transmit from one end of the wire to the other signal frequencies that it may
without significant reduction in strength. Bandwidth is measured in hertz (Hz), transmit from one end of
the communication link to
e.g. 500Hz. The hertz is a unit of frequency equal to one cycle per second.
the other without significant
Figures 9.1.2.3 and 9.1.2.4 show the effect of the transmission medium on reduction in strength of the
two different frequencies. signal.
A B
A B
Although a given signal may contain frequencies over a very broad range,
any medium used to transmit the signal will be able to accommodate only a
limited band of frequencies. This limits the bit rate that can be carried on the
transmission medium. Figure 9.1.2.5 shows the effect of a 500 Hz bandwidth
signal channel, e.g. a copper wire, on a transmission with bit rate of 2000 bits
per second.
Bits 0 1 0 0 0 0 1 0
Pulses before
transmission.
Bit rate 2000 bps
Pulses after
transmission
become the
solid line.
Bandwidth 500 Hz
Figure 9.1.2.5 A 2000 bps transmission over a 500 Hz signal channel
Single licence - Abingdon School 353
9 Fundamentals of communication and networking
Questions
Latency
Key concept
Latency is the time delay that can occur between the moment something
Latency: (an action) is initiated and the moment its first effect begins. In a wide area
Latency is the time delay that network involving satellites, significant time delay occurs because of the
can occur between the moment
physical distance between the ground stations and the geostationary satellite.
something (an action) is initi-
ated and the moment its first Requesting and receiving a web page can involve a considerable time delay,
effect begins. even though the bit rate of the uplink and downlink to the satellite is high, i.e.
the bandwidth is large. The speed of microwaves is 3 × 108 m/s. With a round-
trip distance of over 143,200 km, the propagation time delay is approximately
0.4 s.
Questions
9 What is latency in the context of communications?
waits for a response which indicates that the receiving device is ready to receive. On receipt of this signal, the
sending device coordinates the sending of the data and informs the receiver that it is sending the data. The sender
then waits for the receiver to become ready to receive more data.
Questions
10 What is a communications protocol?
■■ Definitions for
• baud rate
• bit rate
• bandwidth
• latency
• protocol
■■ The difference between baud rate and bit rate
■■ The relationship between bit rate and bandwidth
9.2 Networking
Learning objectives:
■ Understand ■ 9.2.1 Network topology
• physical star topology
Topology
• logical bus network The way computers are cabled together or linked to form a network is very
topology important. The term topology is used to describe the layout of a network.
When the computers that are linked are in close proximity, e.g. in a single
■ Differentiate between both
building, the network is called a local area network.
■ Explain their operation The topology in Figure 9.2.1.1(a) is a mesh and that in Figure 9.2.1.1(b) is a
■ Compare each (advantages bus. Computer Computer
and disadvantages)
Bus
Key concept
Network topology:
The shape, layout,
(a) (b)
configuration or structure of the
connections that connect Figure 9.2.1.1 Two different network layouts
devices to the network.
Star topology
The most common physical network topology is the star which is shown in
outline in Figure 9.2.1.2. The centre of the star is either a network switch or a
central computer.
Central switch
or central computer
Key fact A computer communicates on the network through a network interface card or
network adapter. A network adapter plugs into the motherboard of a computer
MAC address: and into a network cable. Network adapters perform all the functions required
A MAC address is a 48-bit address
to communicate on a network. They convert data between the form stored
expressed in hexadecimal and
separated into 6 bytes, in the computer and the form transmitted or received on the cable (Figure
e.g. 00-02-22-C9-54-13. 9.2.1.3).
It is the physical or hardware
Serial data flows from the network
address of the network adapter adapter card on to the network
and is designed to be unique.
Network
adapter
card
Did you know?
Ethernet uses CSMA/CD Parallel data flows from the
(Carrier Sense Multiple Access/ computer’s motherboard
Collision Detection). Ethernet to the network adapter card
Did you know? The network adapter then transmits the frame one bit at a time onto the
network cable. The address information is sent first, followed by the data and
The MAC address is written
to EEPROM (Electrically
then the checksum. In the Ethernet protocol, each network card is assigned
Erasable Programmable Read a unique address called its MAC address. MAC stands for Media Access
Only Memory) on the network Control. A MAC address is a 48-bit address normally expressed in hexadecimal
adapter. This means that it is and separated into 6 bytes, e.g. 00-02-22-C9-54-13. Part of the MAC address
possible with the right software
identifies the manufacturer. Each network card manufacturer has been allocated
to alter a card’s MAC address.
Why might a person want to a block of MAC addresses to assign to their cards.
do this? Figure 9.2.1.5 shows how three computers connected via T-pieces to a bus can
send Ethernet frames to each other.
357 Single licence - Abingdon School
9.2.1 Network topology
Blue
Orange + White
Orange
Blue + White
Figure 9.2.1.10 shows a 10/100 Mbps Ethernet network adapter that connects to the PCI bus of a computer.
Cat5 network
cable plugs
into RJ45
socket RJ45 socket
Figure 9.2.1.10 10/100 Mbps Ethernet network adapter that plugs connects to the PCI bus of a computer
Questions
1 In the context of networking, what is a topology?
4 Explain how a network wired in star topology can behave logically as a bus network.
Questions
9 Discuss the advantages and disadvantages of operating a logical bus network topology wired as a physical
star topology. You may wish to make reference in your answer to a physical bus topology and a traditional
star topology.
9.2 Networking
Learning objectives:
■ Explain the following and
describe situations where they
■ 9.2.2 Types of networking between hosts
might be used Peer-to-peer networking
In a peer-to-peer (P2P) network there is minimal or no reliance on dedicated
• peer-to-peer networking
servers.
• client-server networking All computers are equal hence the name peer.
Each computer can communicate with any other computer on the network to
which it has been granted access rights.
Key concept
Each peer computer acts as a client when initiating requests to another
Peer-to-peer network: computer(s) for resources and as a server when satisfying requests from another
A network in which there is computer(s).
minimal or no reliance on
dedicated servers. All computers Figure 9.2.2.1 shows a wired peer-to-peer local area network and the possible
are equal, and are called peers. communication paths between peers.
Each peer may act as both a
client and server.
.
Peer-to-peer Switch
network
Peers act as
both clients and
servers
A B C
Peer Peer Peer
on the computer, e.g. a directory, but there is no central login process where a
user’s access level is protected by a single password. A user logged in at one peer
computer is able to use resources on any other peer computer if the resources
are unprotected by passwords or if the user knows the relevant password.
Peer-to-peer networks can be as small as two computers or as large as thousands
of computers.
Questions
1 Explain peer-to-peer networking.
Questions
2 Describe the circumstances when peer-to-peer is an appropriate choice for local area networks.
Questions
3 Explain how a server-based local area network differs from a peer-
to-peer local area network and why it is considered a client-server
network.
from browsers running on client hosts. When a web server receives a request from a client host for a web page, it
responds by sending the requested web page to the client host.
In the client-server architecture, clients do not directly communicate with each other. For example, two web
browsers do not directly communicate. This is very different from the peer-to-peer architecture where peers can
communicate with each other because peers can act as both client and server (Figure 9.2.2.1).
Another characteristic of the client-server architecture is that a server has a fixed, well-known address in TCP/IP
networks, called an IP address. A client can always contact the server by sending a packet to the server’s IP address
and get a response because the server is always on.
Search engines such as Google and Bing employ more than one server (hundreds of thousands, in fact) in order to
meet demand. However, to the client these servers appear as a single machine, a virtual server.
Questions
A
4 What is meant by client-server architecture
H B and describe a situation where it is used?
Figure 9.2.2.4 shows an FTP client downloading a file from an FTP server
Increasing the number of clients to twenty reduces the download rate because
the server’s upload rate of 5000 KiB/s is now shared by 20 clients. This means a
maximum download rate of 5000/ KiB/s per client = 250 KiB/s. This is less
20
than the client’s maximum of 500 KiB/s.
One solution is to add another server so that the demand from twenty clients is
shared now between two servers. This should restore the download rate to 500
KiB/s assuming that each client is able to operate at a download speed of 500
KiB/s.
Peer-to-peer networking architecture
The client-server model works well for many applications. However, if many Key concept
clients are downloading very large files from a server, then download speeds Peer-to-peer architecture:
diminish unless more servers are added. It would be better if uploads could be In a peer-to-peer architecture,
shared amongst both servers and clients. This would lighten the load on servers client hosts not only download
and reduce the amount of server bandwidth that clients have to pay for. In fact, a file, but can upload what they
have obtained to others as well.
web hosting could be eliminated altogether. Client hosts not only download
Client hosts are capable of doing
a file, but can upload what they have obtained to others as well. Client hosts both at the same time. They are
are perfectly capable of doing both at the same time. They are then called then called peers rather than
peers rather than clients. Figure 9.2.2.5 shows how separate parts of a file are clients.
uploaded to three peers which in turn upload what they have to their peers.
Peer distribution can start whilst each peer is still downloading their part of the
file from the server. The server doesn’t have to be a dedicated server but can be
another peer that happens to have the file that other peers want to download.
S
Part 1
⅓ Part 3
⅓
⅓ Part 2
Part 3
Part 2 Part 2
A B C
Part 1 Part 3
Peer Peer Peer
Part 1
Figure 9.2.2.5 shows peers uploading and downloading files using FTP
BitTorrent came up with a protocol for distributing large files in this way.
BitTorrent doesn’t overload servers that provide the download since it relies on
peers contributing upload capacity. The result is that large files can be received
Did you know? faster than would be the case in a client-server architecture where downloads
Jaan Tallinn was one of the co- rely on just a central server.
founders and authors of Skype. Decentralized P2P networks have several advantages over traditional client-
Read an interview with Jaan at
server networks. P2P networks scale well because they don’t rely on costly
http://affairstoday.co.uk/interview-
jaan-tallinn-skype/
centralized resources. Scaling doesn’t lead to a deterioration in download speed
because P2P networks use the processing and networking power of the end-
users’ machines which grows in direct proportion to the network itself.
The costs associated with a large, centralized infrastructure are virtually
eliminated because the processing power and bandwidth reside in the peers
within the network.
Investigation
1 Peer-to-peer (P2P) architecture became widely used and popularized
by file-sharing applications such as Napster and Kazaa. Research the
history of Napster and Kazaa.
Questions
5 Explain what is meant by peer-to-peer architecture and describe a
situation where it is used?
■■ peer-to-peer networking
■■ client-server networking
■■ situations where they might be used
9.2 Networking
Learning objectives:
■ Explain the purpose of WiFi ■ 9.2.3 Wireless networking
■ Be familiar with the The purpose of WiFi
components required for
WiFi was invented to provide a wireless connection between computing devices
wireless networking
and to enable these devices to connect
■ Be familiar with how wireless
Internet
to the Internet via a bridge between a
networks are secured wireless LAN and a wired LAN known Fibre/DSL
Modem
as an access point.
■ Explain the wireless protocol
Router
Questions
Key concept
1 What is the purpose of WiFi?
Frame/packet:
2 A frame or packet is a unit
In a wired network, data travels between wired devices by electrical means.
of transfer consisting of
What is the equivalent means by which data travels in a wireless network?
data, addresses and control
information.
The originating station transfers the frame to the access point (first hop) which
then relays it to the destination station (second hop). This lifts the restriction
that stations must be in range of each other. They only have to be in range of an
access point belonging to the BSS. Secondly, access points can buffer frames so
that stations which are battery-powered can be powered down until they need to
transmit and receive frames, e.g. wireless sensors.
The stations must have the capability to send and receive over a WiFi connection.
Figure 9.2.3.3 shows an EDIMAX® wireless network adapter that plugs into a
USB port of a computer. This particular adapter has MAC address 001F1FCD5D7A.
All adapters, wireless or wired have a unique MAC address so that they can be
identified.
The tall slim tubular structure is its antenna through which it radiates and receives
radio waves on specific frequencies designated by the IEEE.
A wireless network adapter is also known as a wireless interface.
Desktop computers can have a wireless network adapter installed on their
motherboard in a PCI slot and tablets and smartphones use a built-in wireless
network adapter.
Within each BSS, stations communicate directly with an Access Point (AP),
similar to a mobile phone network base station. The Access Point acts as a bridge
between a wireless and a wired local area network. When a device searches for WiFi
Figure 9.2.3.3 EDIMAX® connectivity, it sends messages to discover which APs are in its transmission range.
USB port wireless adapter
802.11b/g
Key concept
SSID:
A user-friendly network name,
e.g. educational-computing,
commonly known as a Service
Set Identity/Identifier/
Identification (SSID). It is used
to identify a Basic Service Set
(LAN) to users of a wireless
network.
Information Figure 9.2.3.5 Wireless radio frequency options for wireless network
with SSID educational-computing
Beacon frames:
In an infrastructure network,
Figure 9.2.3.5 shows the settings for an AP with SSID educational-computing.
the access point is responsible The AP automatically chooses either 2.4 GHz or 5 GHz depending on which
for transmitting beacon frames currently provides the better transmission, and then within the chosen band a
at regular intervals. Wireless particular channel1.
stations in range receive these
beacon frames and use them to Questions
find and identify a network. The
reception area for beacon frames 3 Name and describe four major physical components that may be
defines the basic service area. found in a wireless network.
All communication in an
infrastructure network is done 4 What is the purpose of an SSID?
through an access point, so
stations on the network must be
close enough to the access point Carrier Sense Multiple Access with Collision Avoidance
to receive beacon frames. (CSMA/CA)
Channel selection
Each access point (AP) operates on a given frequency channel, e.g. channel 36.
Both the 2.4 GHz and the 5 GHz frequency bands are divided into a number
of such channels, each with a predetermined width. Figure 9.2.3.6 shows that
the channels for the 5 GHz band are 20 MHz wide. For example, channel 36 is
centred on the frequency 5180 GHz and it covers the range 5170 to 5190 GHz.
5170 5330 5490 5710
MHz MHz MHz MHz
100
104
108
112
116
120
124
128
132
136
140
IEEE channel #
36
40
44
48
52
56
60
64
20 MHz bands
Figure 9.2.3.6 Operating channel bands and frequencies for 5 GHz wireless
transmission in Europe
In order for a station to communicate with its access point, it selects the
channel that its AP is using. This means that all the stations in the same basic
service set, e.g. educational-computing, use the same channel. Therein lies a Did you know?
problem, as more stations join a basic service, the transmission rate usually The late Steve Jobs, former
goes down which is one reason why the maximum transmission rate is rarely CEO of Apple had difficulty
achieved. The AP will actually tell the stations to use a lower transmission rate connecting via WiFi at the
if there is too much traffic from too many devices. conference in 2010 that
launched the iPhone 4
Questions smartphone. Too many people
in the auditorium were already
5 What is meant by channel selection in wireless networking?
using the WiFi channel Steve
needed for his demonstration.
Interference Steve had to ask all the delegates
WiFi is prone to interference from other sources as well as from stations using a to disconnect their WiFi to
enable him to connect.
particular basic service because
• the frequency bands used by WiFi are unlicensed and so WiFi is Information
not the sole user of the spectrum. The spectrum is used by lots of Signal to Interference +
other equipment over which the basic service set has no control, e.g. Noise Ratio (SINR):
microwave ovens used to heat and cook food in the kitchen Radio wave communication
relies on differentiating a signal
• its maximum transmit power is restricted to a very low level because it from background noise and
is unlicensed interference. If the signal power
• it is typically used indoors where there are lots of objects to block the level falls below the noise +
interference power level, it
signal or reflect and echo a transmitted signal into the path of another
becomes more difficult to
Noise extract the signal. The ratio of
The circuits used in WiFi generate unwanted electrical noise (random electrical signal power level to noise +
fluctuations) which can mask signals of too low power. interference power level is a
measure of how easy or difficult it
Collisions is to extract a signal
When two stations, A and B are transmitting at similar times their frames will Signal power
SINR =
collide if they are within interference range of each other which is the case Interference + Noise power
when they both use the same access point (Figure 9.2.3.7). The outcome is
determined by the signal-to-interference + noise ratio (SINR) of
each. Access Point
Figure 9.2.3.8 shows the overlap of two frames one from A and one
from B. The greater the overlap with another frame, the higher the
chance that neither frame will be properly decoded at the receiver.
Signal Frame A
Strength sent by A Frame
sent by B B
A
B
Figure 9.2.3.7 shows the energy of
Overlap of two frames spreading through the
frames air and crossing at the access point
Time
Figure 9.2.3.8 Two frames overlapping in time. A is a stronger signal
than B
Single licence - Abingdon School 372
9 Fundamentals of communication and networking
Wait &
listen Wait &
If a station that wants to send senses listen
Frame to B
that the channel is busy at any time C
during the wait and listen period, then Wait &
listen
the station does not transmit. C starts Frame to E
its wait & listen period just after A’s. It D
waited and found the channel idle before sending. Another station might have begun sending at the same time
because it began its waiting period at the same time and also concluded that the channel was idle. This is shown
in Figure 9.2.3.14 where both C and D start sending at the same time and cause a collision. However, neither
receives an acknowledgement No ACK
signal within the expected
window of time and so both Randomly
pick
conclude that an error has Station Wait & slot
a
occurred. listen 0 15
Frame to B
If both resend at the same time C
the same result ensues.
Wait &
The solution in this listen 0 15
Frame to E
circumstance is for stations C D
and D to employ a contention Randomly
Time
window of say size 15. Such pick
a slot
a window is divided into 15
Figure 9.2.3.14 Shows stations C and D in addition to a wait & listen
equal-sized time slots. Stations
period backing off a randomly determined number of time slots between
C and D then each choose one
0 and 15 after concluding by receiving no ACK that that a collision has
of the time slots at random,
occurred
e.g. C might choose slot 3 and
D slot 7 (there remains a small
chance that they could choose
the same slot). C will now listen Station
C
The RTS silences stations within its range and the CTS silences stations within
its range. In this way collisions that result from the hidden station problem
shown in Figure 9.2.3.12 are avoided. Key concept
In Figure 9.2.3.15 station A is able to reach station B but not station C whilst
Security:
station B is able to reach both. Station A sends an RTS to station B, B responds Security means protecting
with a CTS which reaches both A and C. C now avoids sending until it receives against unauthorised access,
an indication that the transaction between A and B is over. This is the ACK alteration or deletion.
signal. Meanwhile A receives the CTS signal and proceeds to send the data
frame. A finishes sending and then waits for the ACK signal from B.
Key concept
Questions
Authentication:
7 What is meant by multiple access in wireless networking? Proving that the user is who
they say they are.
8 Explain the CSMA/CA protocol used in wireless networking.
Questions
12 What are WPA and WPA2?
13 State and explain three reasons why wireless networks need to be secured.
Install Temporal
Message 1: Key
Information
Message 2 includes a MIC. This is a message digest that has used the EAPoL Key Confirmation Key (KCK)
from the supplicant’s copy of the PTK. The authenticator now uses its copy of KCK and the received message
to calculate the corresponding
MIC. The two MICs, one received
and the other calculated, are
compared.
Message 3:
ANonce
If the MICs agree, the
authenticator sends an
acknowledgment message to the
Replay Sequence Counter
supplicant confirming that it has
Message Integrity Code
been authenticated and is now to prove data origin
authenticity
allowed to join the network and
Encrypted GTK encrypted
to install the PTK data encryption using KEK key
key (Figure 9.2.3.20). The Figure 9.2.3.20 Message 3 frame captured with Wireshark
authenticator awaits confirmation
from the supplicant that it has installed
the data encryption key (temporal key)
before it installs its copy. The same
GTK is used for all stations.
Message 4:
The supplicant responds with an
SNonce
acknowledgement message (Figure
9.2.3.21) confirming to the authenticator
that it has installed the temporal key
(data encryption key) that should be
used from now on to encrypt data
transmissions as well as to generate the
Figure 9.2.3.21 Message 4 frame captured with Wireshark
MIC to protect the integrity of the data
as well as authenticate its origin. The authenticator now installs its copy of the temporal key.
Task
1 Download and install Wireshark from www.wireshark.org on a computer with a wireless interface. In
Wireshark, select the wireless interface and enable monitor mode. Start capturing wireless frames whilst
at the same time connecting to a wireless network access point. Stop the capture once you are connected.
Set the filter in the main window filter to EAPoL so that you can see four messages similar to those above.
Expand the Authentication part of the frame and examine messages 1 to 4 in turn as above. What you
will see will depend on the wireless protocol that you have chosen and the cipher suite supported by your
wireless interface. Clear the EAPoL filter and search for Beacon, Probe Request and Probe Response frames.
Information
Figure 9.2.3.23 shows two networked connected stations each with a different
Man-in-the middle attack:
paired temporal key (data encryption and MIC key).
In cryptography and computer
security, a man-in-the-middle
Station 1 Station 3 attack is an attack where the
attacker secretly relays and
pairwise possibly alters the communication
between two parties who believe
temporal they are directly communicating
key 1
temporal
key 1
with each other.
pairwise
Access temporal
point key 2
Information
temporal Replay attack:
key 2
In network security, a replay
attack is a form of network attack
Station 2 Station 4
in which a valid data transmission
A temporal key is used to encrypt data transmissions and to create MICs
to protect and authenticate the data is maliciously or fraudulently
repeated or delayed.
Figure 9.2.3.23 Station 1 and the access point are connected
One way that replay attacks are
with a pair of keys, temporal key 1; station 4 and the access point defeated is to use the sequence
are connected with a different pair of keys, temporal key 2. number of packets. If stations and
access points record the highest
Questions received sequence number then
14 Why is relying on PMK, source and destination addresses alone as they can reject packets with lower
input to the PTK computation not as secure as the method which sequence numbers which occur
with replayed packets.
includes two nonces?
Key principle Media Access Control (MAC) address white list filtering
A wireless network could not use any form of encryption for its packets but
Media Access Control (MAC) instead rely on filtering of packets. MAC address white list filtering is one
address white list filtering:
such form of filtering. MAC addresses are 48-bit addresses uniquely assigned
In MAC address white list
filtering, the access point has an to each wireless network interface card. In MAC address white list filtering,
internal table of MAC addresses the access point has an internal table of MAC addresses which it consults to
which it consults to decide decide whether to permit access to the network or not. If the supplicant’s MAC
whether to permit access to the
address is on this list then it may join the wireless network controlled by this
network or not.
access point. If its MAC address is not on the list then the access point will
reject any attempt that the supplicant makes to join the network. Whilst MAC
address white list filtering gives a wireless network some additional protection,
Key principle MAC filtering can be defeated by a spoofer who learns the MAC address of
a valid wireless network interface card, i.e. one on the white list, by scanning
SSID broadcast disabled
protection: wireless traffic and then replacing a validated one with their own MAC. Task 1
Wireless stations require a with Wireshark should have revealed that MAC addresses do not get encrypted
knowledge of the SSID in when travelling over the air between computer and wireless access point. A
order to join the network. If MAC address is “glued” into a network card, but it is possible to command the
broadcast, the SSID appears in
operating system to change information about the MAC address in every data
the network settings window of
stations within range. packet it sends out to the network. In this way a spoofer could gain access to
In this form of protection, the white list protected network.
an access point disables SSID broadcast disabled protection
broadcasting its SSID to wireless
Access points have the option to disable broadcasting their SSID. This
stations. Thus, only clients
who already know the pre- means that the SSID will not appear in the client’s network settings window
configured SSID can establish a (see Figure 9.2.3.4). Clients who already know the pre-configured SSID
connection, others will not be can establish a connection, others will not be able to (without a bit of extra
able to (without a bit of extra
effort). Unfortunately, clients who already know the SSID cause the SSID
effort).
to be revealed to snoopers when establishing a connection with the access
point. Before the authentication stage begins, the client sends a Probe Request
message and receives a Probe Response from the access point in return as
shown in Figure 9.2.3.24. The (unencrypted) SSID is present in these packets,
therefore reducing the effectiveness of disabling broadcasting of the SSID.
To discover the SSID, a snooper might first send a deauthentication message
to the stations that are connected to force them to disconnect and reconnect.
Reconnecting should cause Probe and Response Request messages to be
broadcast which reveal the SSID.
Figure 9.2.3.24 Probe and Response Request frame/packets captured with Wireshark to reveal the SSID
Questions
15 What is (a) MAC address white list filtering? (b) SSID broadcast disabled protection?
16 Explain why both MAC address white list filtering and SSID broadcast disabled protection are insufficient
alone to protect a wireless network.
Learning objectives:
■ 9.3.1 The Internet and how it works
■ Understand the structure of The structure of the Internet
the Internet The term ‘internet’ is a combination of ‘inter’ and ‘net’. The ‘net’ refers to a
■ Understand the role of packet computer network and ‘inter’ refers to interconnections between two or more
switching and routers
computer networks and computers or devices with computing capability on
■ Know the main components of
a packet these networks. This is how the Internet began back in the 1970s. The problem
■ Define: was how to connect packet-switched networks located in North America and
• router Europe. It was solved by Robert Kahn, Vint Cerf and others, and the Internet
• gateway was born. Any network of computer networks is an internet or internetwork.
■ Explain how routing is NHS workers use a secure private internet to access patient records. The
achieved across the Internet network used by the general public for e-mail and web page access is a
■ Describe the term ‘uniform special internet called the Internet. It consists of a network of interconnected
resource locator’ (URL) in the computer networks and computers using a globally unique address space
context of internetworking
(IP addresses) based on Internet Protocol (IP), and Transmission Control
■ Explain the terms ‘domain Protocol (TCP) to support public access to e-mail and web pages, among
name’ and ‘IP address’
other things. Figure 9.3.1.1 shows how computer networks and computers are
■ Describe how domain names
connected by the Internet.
are organised
Global internetwork or the Internet
■ Understand the purpose
and function of the University
Domain Name System campus North Central and
(DNS) and its reliance America South America
on the DNS server
system
■ Explain the service
provided by Internet Home
registries and why they user
International links
are needed. or core backbone
Continental and
national backbones
Key term Europe, Africa, the Middle East, North America, Central and South America,
the Far East and the Pacific are linked by very high-speed connections which
Internet: A network of form the core backbone of the Internet. Each continent has a backbone of very
computer networks, computers
high-speed links which interconnect routers located in each country. Routers
and devices with computing
capability using globally unique are special packet switches that receive incoming packets of data along one link
IP addresses and TCP/IP. and send them as outgoing packets on another link.
Open architecture networking
The Internet uses open architecture networking. Designers are free to design
Key term
networks however they want, but all these different networks can be connected
Packet switching: to and communicate over the Internet because of the way that the Internet has
Messages to be sent are split
been designed. Each network is connected to the Internet through a router (a
into a number of segments
called packets. The packets
special router called a gateway router).
of a message are allowed to Until recently, the end-system devices in these networks that connected to the
travel along independent paths Internet were desktop computers and powerful servers but now a wider range of
through a network of routers.
devices are being connected to the Internet. These end-systems are referred to as
Routers use a packet’s
destination IP address to route hosts because they host (i.e. run) application programs such as a Web browser
the packet, taking account of program, a Web server program, an email client program, etc. Each host has a
how congested particular routes user-friendly memorable hostname, e.g. www.aqa.org.uk.
are.
This network resembles a fishnet Questions
of switching nodes called routers 1 Explain the term internet.
connected by links in a way
that allows multiple pathways 2 What is the Internet?
through the network between
endpoints. The role of packet switching and routers
The role of packet switching is to support end-to-end communication of a
message between two hosts located in different parts of an internet. Messages
split into smaller segments called packets travel along independent paths
Key term
through a network of packet switches called routers. Each packet sent by a host
Role of a router: contains the IP (Internet Protocol) address of the destination host. Each router
Routers are special packet uses the destination IP address to choose from among its outgoing links one
switches that receive incoming
along which the packet can reach its destination.
packets of data along one link
and send them as outgoing Within national boundaries, networks belonging to large businesses and
packets on another link. organisations such as universities are connected directly to the national
backbone. Smaller organisations and home users of the Internet connect to
an Internet Service Provider (ISP), which connects to the national backbone.
Figure 9.3.1.2 shows three networks connected by routers.
The design of the Internet is based on the Catenet (internet) concept of a
network of networks. This concept describes data packets flowing essentially
unaltered throughout an internet with their source and destination addresses
(IP addresses) that of the endpoint systems (now referred to as end-systems)
sending and receiving the packets, respectively.
Circuit switching connects two endpoints with a complete end-to-end electrical circuit for communication by
exclusively allocating all the switches and their links along a single pathway for the duration of the communication.
No other endpoints can use the same pathway if it is already in use.
By contrast, packet switching allows packets from different messages to use the same
nodes and links at the same time. This makes better use of the network’s bandwidth,
Question
defined as its maximum capacity for sending information. 3 Explain the role of
packet switching
The built-in redundancy of packet switching means better congestion handling, e.g. if a
and routers in a
particular route is busy then packets may be rerouted along a different, less busy route.
packet switched
It also means that the network can withstand switching nodes going down. This means
network.
that cheaper, less reliable nodes may be used.
The Internet uses packet switching.
Packet transmission
The packet is the unit of communication in the Internet.
In its simplest form, a packet consists of three parts: Source address Destination Application
address data
When computer X, connected to the Internet, wishes
to send a message or a document to computer Y, also connected to the Internet, computer X splits the message or
document into chunks; Figure 9.3.1.5 shows five chunks: A, B, C, D and E. Computer X then generates as many
packets as there are chunks, placing each chunk in the application data part of the next available packet. The unique
address of the sending computer is placed in the source address part of each packet and the unique address of the
receiving computer is placed in the destination address part of each packet.
Each packet is then dispatched to the Internet through a router. The packets are sent independently through a
series of interconnected routers until they reach their destination. Each router examines the destination address of a
packet it receives to determine what to do with it. Computer Y could reply to computer X by a similar process.
Original
message
EDCBA Network
Message
packets
E D C B A D C A D A
1 3 5
D
Computer
X E B C
Computer
A Y
E E C B D A E C B
2 4 6
EDCBA
Router node Re-assembled
message
Figure 9.3.1.5 Routing of packets A, B, C, D and E through a packet-switched network
decision. In this example the decision is to route packets up the national hierarchy each connected device and the
port it is connected to. It is then
because 202/203 addresses are outside England.
able to map a packet’s destination
Local
Company
/School
/College/etc
networks
Figure 9.3.1.7 Routing hierarchy for one country
Questions
6 Explain why routers are used.
7 What is a router?
8 How are routers organised in the Internet?
Gateway
Gateways are special routers that provide an interface between two or more dissimilar networks. Gateways translate
one network layer protocol into another, translate link layer protocols, and open sessions between application
programs.
Local Area Network Local Area Network
Gateways allow two or more A B
respects from the Wide Area Figure 9.3.1.8 Gateway connections to the Internet
Network (WAN) protocols used
on the Internet. The gateway does the job of translating the LAN frame into its equivalent WAN frame and vice
versa. Sometimes gateways are called gateway routers.
Another common use of gateways is to enable LANs that use TCP/IP and Ethernet to communicate with
mainframes that use other protocols.
Question
9 What is the purpose of a gateway?
to the sender constructed with the received acknowledgement number to indicate that the packet with this number
was received successfully.
Packet transmission errors can occur for a variety of reasons but it must be possible for the receiver to detect when
errors have occurred. A checksum (e.g. CRC) attached to each packet is used for error detection.
Packets travel along transmission media consisting of wires, fibre-optic cable and radio frequency links. To launch
a packet onto or to receive a packet from any of these requires a layer of hardware called a network interface.
Each sending/receiving host’s network interface hardware needs to be assigned a “fixed” unique hardware address
called a link layer address. Similarly, each router will need link layer hardware, uniquely identified by a link layer
hardware address, because routers are also connected to transmission media.
A packet is structured into a series of headers which contain all of the above information and the message chunk or
data - Figure 9.3.1.9. The data D and the headers HT, HN, HL make up each packet (and a checksum appended to
the end of the packet).
Web Web
browser server
Application data D Application HTTP HTTP
Port No & Port No &
TCP/IP
protocol
Segment HT D Transport Error control Error control
IP address
stack Datagram HN HT D Network IP
address
Frame HL HN HT D Link hardware
address
hardware
address
Physical Cable/radio
frequency link
Packet Cable/radio
frequency link
Figure 9.3.1.9 The protocol stack for TCP/IP showing how different headers are added to the payload data
- see Chapter 9.4.1
0 15 16 31 TCP header
16-bit source port number 16-bit destination port number
Figure 9.3.1.10 shows some of the detail in the header
32-bit sequence number labelled HT in Figure 9.3.1.9. HT is called the TCP
32-bit acknowledgement number 20 bytes header. The key fields to note are the port numbers and
4-bit header the sequence number fields.
length
16-bit TCP checksum IP header
Figure 9.3.1.11 shows some of the detail in the header
Figure 9.3.1.10 TCP header, HT labelled HN in Figure 9.3.1.9. HN is called the IP
0 15 16 31 header. The time-to-live field is used in order to stop
4-bit header 8-bit type of 16-bit total length (in bytes) of IP
length service (TOS) datagram a lost packet wandering the Internet forever. The
key fields to note are the source and destination IP
8-bit time to 16-bit header checksum addresses and the time to live fields.
20 bytes
live (TTL)
application data. The length field value is for the entire frame including application data but excluding the CRC
checksum.
48-bit destination address 48-bit source address 16-bit length 16-bit type
field field
Figure 9.3.1.12 Link layer header, HL
The type field identifies the type of data, e.g. ARP request which maps an IP address to a hardware address. The
destination and source addresses are hardware addresses usually assigned to the network interface card by its
manufacturer. In the Ethernet protocol, the hardware addresses are known as MAC addresses or Media Access
Control addresses. The key fields to remember are the hardware destination and source address fields.
Figure 9.3.1.13 shows the main components of a packet.
Hardware Hardware Source IP Destination IP Source Dest. Sequence
Data
destination address source address address address port no port no no
Router R1
Network
End-system host X Link
Application Router R2
Transport Network
Network Link
Link
Home network
Local or
Symbol for a router
regional ISP
centre
Symbol for a switch Network
Link
Network
Link
Router R5
Company network
Servers
Routing refers to the network-wide process that determines the end-to-end paths that packets take from source to
destination. Routing is necessary when the destination is not directly reachable.
A router receives a packet on an input link and transfers the packet to the appropriate output link. This is known as
forwarding. Every router has a forwarding table. A router forwards a packet by using the destination IP address in
the arriving packet’s header as an index to an entry in the router’s forwarding table. This entry indicates the router’s
outgoing link to use to forward the packet.
Using a driving analogy, routing would be equivalent to an ordered list of roads that need to be travelled to get
from place X to place Y. Forwarding would be equivalent to choosing the exit to take at the junctions/roundabouts
connecting each road segment to the next.
Routers play a crucial role in both the process of forwarding and the process of determining the paths to follow. A
router uses algorithms known as routing algorithms to calculate these paths and perform packet switching when
forwarding a packet. The routing algorithm determines the values that are inserted into the forwarding table of
each router. In the decentralised model, each router executes a part of a distributed routing algorithm relevant to its
location in the hierarchy of routers (see Figure 9.3.1.7).
In Figure 9.3.1.14, host X is sending a message to host Y. The message might be an HTTP GET message to an
HTTP server Y (see Chapter 9.3.2). The transport layer of the TCP/IP protocol stack in host X adds a transport
header to the message packet generated by the application layer to produce a transport layer segment. This segment
then passes to the network layer which adds its own header to it to create an IP datagram which then passes to the
link layer. This layer adds its own header and a checksum trailer to create a link frame. This frame is delivered to the
physical medium where it travels to router R1.
R1 strips away the link layer header and trailer to extract the IP datagram which then passes to the network layer.
The network layer consults its forwarding table to determine that the datagram should be sent onto the outgoing
link that connects R1 to router R2. The datagram is passed to the link layer that adds the correct link layer header
and trailer to create a new frame. The link layer then passes the frame to the physical medium which transfers it to
router R2.
The process that occurred at R1 now repeats at router R2. R2 consults its forwarding table to find which outgoing
link to use, i.e. the one joined to router R3. This process continues until the frame arrives at the end-system host Y.
At host Y, the headers are stripped away progressively as the packet is passed up the protocol stack until only the
message remains. The last header, the transport header, is removed by the transport layer before the message is
delivered to an application layer process, e.g. an HTTP server.
Routers require just two layers for this task: the network and link layers of the protocol stack.
Note that the source and destination IP addresses set by host X remain the packet’s source and destination IP
addresses throughout the journey through the routers from host X to host Y. However, the link layer hardware
source and destination hardware addresses are changed for each hop.
Table 9.3.1.1 shows a simplified example of a forwarding Destination address
Link interface
table for a router with four links, numbered 0 to 3. The router prefix
has IP address 146.97.33.2. It connects to hosts with dotted- 10001110 0
10010000 1
decimal IP addresses beginning with 142, 144, 152, 155 via
10011000 2
other routers.
10011011 3
For example, www.southamption.ac.uk has IP address 152.78.118.52. The router with the forwarding table shown
in Table 9.3.1.1 routes packets it receives with destination IP address 152.78.118.52 to the link interface numbered
2.
Questions
11 Explain how routing is achieved across the Internet.
13 Explain why some addresses must change and some must not.
Questions
14 What is a URL and what does it specify?
15 Give an example of a URL, different from the one above. Identify its how, where and what parts.
Questions
16 Why was the Domain Name System invented?
emps ags
exeter bucks
Root
Figure 9.3.1.16 Domain Name System hierarchy
A particular host on this network was mail.ags.bucks.sch.uk. This name (hostname) is an example of a fully
qualified domain name (FQDN). An FQDN uniquely identifies a host. When the host ID of this computer, 124,
is added to the network ID, the IP address becomes 195.112.56.124.
In mail.ags.bucks.sch.uk. the domain uk includes all hosts that use the top-level domain name suffix ‘uk’.
The second-level domain sch includes all the hosts that use ‘sch’, the third-level domain bucks includes all the hosts
in Buckinghamshire, and the fourth-level domain ags includes all the hosts in the organisation AGS with globally
unique IP addresses. Finally, mail is the public name of the host. Table 9.3.1.2 shows the interpretation of some
top-level domain names. Table 9.3.1.3 shows the interpretation of some second-level domain names.
Questions
17 How is the domain name system organised?
18 Give two examples of top level domain names and two examples of second level domain names.
19 Give the meaning associated with each domain name given in Q18.
Purpose and function of the Domain Name System (DNS) and DNS Servers
People prefer to use the user-friendly memorable form of hostname, i.e. its fully qualified domain name, e.g.
university-of-exeter.ja.net.
However, routers prefer fixed-length IP addresses. In order for the needs of each to be met a directory service is used
that translates hostnames to IP addresses e.g. university-of-exeter.ja.net → 146.97.144.42.
This is the main purpose of the Internet’s Domain Name System (DNS).
Domain Name System servers translate FQDNs into IP addresses.
The DNS is a distributed database implemented in a hierarchy of DNS servers, and an application-layer protocol
that allows hosts to query the distributed database to resolve domain names into IP addresses before connecting to
other hosts on the Internet.
No single DNS server has all the mappings for all the hosts in the Internet. Instead the mappings are distributed
across the DNS servers.
DNS also provides a few other services in addition to translating hostnames into IP addresses:
• Host aliasing - A host with a complicated hostname can have one or more alias names. For example,
www.exeter.ac.uk and admin.ex.ac.uk are alias hostnames for a server with hostname webdata02.
ex.ac.uk and IP address 144.173.6.226 that hosts University of Exeter’s website as well as other sites. The
hostname webdata02.ex.ac.uk is said to be the canonical hostname (CNAME).
• Mail server aliasing - user-friendly memorable email addresses such as [email protected] are aliases.
The hostname of the mail server hotmail.com that is used for Fred’s email will almost certainly be more
elaborate than this, e.g. the canonical hostname of the alias hotmail.com is actually
origin.sn145w.snt145.mail.live.com with server IP address 155.55.152.112. DNS can be used by a mail
application, e.g. Microsoft Outlook, to obtain the canonical hostname for a supplied hostname as well
as the IP address of the host. Aliasing also allows a Web server and a mail server to have identical aliased
hostnames: e.g. company.co.uk.
• Load distribution - servers such as Web servers may be replicated if they host busy sites. Each server
runs on a different end system with each having a different IP address. The set of IP addresses for these
server end-systems is associated with just one canonical hostname. The DNS database contains this set
of addresses. A DNS server responds with the entire set of IP addresses when hit with a client request to
translate a hostname into an IP address. Clients normally pick the first IP address in the set they come
across. The DNS server rotates the order of IP addresses in each reply so that the traffic is distributed
amongst the replicated Web servers (the same is true of mail servers).
Questions
20 What is the main purpose of the Domain Name Service?
21 What services are performed by DNS servers?
A very useful tool to use when exploring the DNS system is the Unix DNS tool dig. It is also available to use in
Linux and Mac OS X operating systems. The Raspberry Pi2 computer connected to the Internet may be used to
explore the use of dig but dnsutils will need to be installed first as follows:
sudo aptget install dnsutils
Another useful tool available in both Microsoft Windows and Unix/Linux/Mac OS X systems is nslookup. This
tool maps domain name to IP address.
For example, nslookup www.aqa.org.uk returns the answer 194.34.8.20.
The tool dig may be used to query DNS name servers as shown in Table 9.3.1.4.
dig NS uk. Returns the hostnames of name servers for the domain dns1.nic.uk.
name uk. located in the top level of the domain name nsd.nic.uk.
system. Nominet UK is responsible for nic.uk. and several others.
dig NS ac.uk. Returns the hostnames of name servers in the domain with auth03.ns.uu.net.
domain name ac.uk. and located in the second level of the ns4.ja.net.
domain name system. and several others.
dig NS bris.ac.uk. Returns the hostnames of name servers for the domain irix.bris.ac.uk.
name bris.ac.uk. located in the third level of the domain ncs.bris.ac.uk.
name system. ns3.ja.net.
dig A irix.bris.ac.uk. Returns the IP address of the DNS server irix.bris.ac.uk.
137.222.8.143
A means address.
dig A snowy.cs.bris.ac.uk. Returns the IP address of the host with hostname snowy.
137.222.103.3
cs.bris.ac.uk.
dig +trace www.aqa.org. This will report all the DNS servers that are consulted in 194.34.8.20
uk @a.root-servers.net. resolving www.aqa.org.uk. NOTE a.root-servers.net (AQA Education
must end with a full stop. 194.34.8.0/24)
Table 9.3.1.4 Using the dig command to query the DNS system
For more examples of the use of the dig command see
www.cyberciti.biz/faq/linux-unix-dig-command-examples-usage-syntax
Tasks
1 Use the URL http://simpledns.com/lookup-dg.aspx to access a DNS delegation trace utility. Use this
utility to trace the DNS server queries for a host snowy.cs.bris.ac.uk. Note that the trace begins with a DNS
root server chosen from the list of possible root servers, and proceeds down the hierarchy of name servers
until the IP address of snowy.cs.bris.ac.uk is obtained.
2 Use the nslookup command to look up the IP address of the following www.exeter.ac.uk, www.
manchester.ac.uk, www.bristol.ac.uk.
3 Use the dig command to find the hostnames of third level DNS servers for the University of Exeter (hint:
use the information in Table 9.3.1.3 to target exeter.ac.uk.). Repeat the exercise for the Computer Science
department, University of Washington in the USA (hint: target cs.washington.edu.). If you do not have
access to the dig command you can use the site referenced in Question 1.
5 Use ipconfig/all in Windows command line to discover your computer’s IP address, the default gateway
and the physical/hardware address (MAC address) of its network card.
6 Install Wireshark - https://www.wireshark.org. Familiarise yourself with Wireshark by capturing a few
packets and examining their contents.
Internet registries
Private companies and organisations called Internet registrars are responsible for registering Internet domains and
therefore domain names to people, businesses and organisations, domain names such as educational-computing.
co.uk. A registrar is an online retailer where domains (domain names) can be bought.
The world is divided into five geographical regions for the purposes of Internet registries: Canada, USA, and some
Caribbean Islands (ARIN), Africa (AFRINIC), Asia/Pacific Region (APNIC), Europe, the Middle East and Central
Asia (RIPE), Latin America and some Caribbean Islands (LACNIC).
Regional Internet Registrars delegate responsibility for registering domain names to their customers, which include
Internet Service Providers and other organisations. The Regional Internet Registry for Europe is RIPE (Réseaux IP
Européens).
RIPE has delegated to Nominet to hold the official registry for all .UK domain names. Nominet is therefore an
Internet registrar. Nominet provides a
WHOIS tool that can be used to find out Task
if a .UK domain name is registered and 7 Use the WHOIS service of Nominet (www.nominet.uk/whois)
if it is, provide details of the registration to look up the registration of the following: (i) co.uk. (ii) ac.uk,
including the registrant. Nominet sets (iii) org.uk. (iv) plc.org. (v) me.uk. (vi) ags.bucks.sch.uk.
the policies and rules that relate to the (vii) commonweal.co.uk. (viii) lordwilliams.oxon.sch.uk.
management of .UK. It does delegate
Question registration to third parties, e.g. RM Education PLC, but third party registrars
22 are required to follow Nominet’s policies and rules in respect of .UK domains.
What is an Internet
registry? Internet registries store registered domain names and the details of the
registrants, e.g. domain name educational-computing.co.uk has been
Key term registered by Educational Computing Services Ltd, the registrant.
Internet registries: The registrant or their ISP will supply the IP address of a DNS server(s) (the
Internet registries store registrant’s or the ISP’s) for this domain name to their Internet registrar. This
registered domain names registrar will then place an entry for this domain name and supplied IP address
together with the details of their at the corresponding level in the Domain Name Server System.
registrants.
Registering a domain name and associating it with a range of IP addresses
Information Suppose that you wanted a domain name for a group of servers, including a
Web server, that you intend managing yourself.
ASN:
The Internet is organised into a You have chosen myowndomain.co.uk as your domain name.
network of autonomous systems
You have obtained Internet connectivity, by contracting with and connecting
each of which manages their own
internal routing. Autonomous to, a local ISP (Internet Service Provider). You have purchased a gateway router
systems are identified by a unique which will be connected via DSL (Digital Subscriber Line) to a router in your
autonomous number (ASN that is local ISP.
stored in the databases maintained
by the Internet address registries. Your local ISP has granted you a block of IP addresses, one of which you will
The web page assign to the Web server (www.myowndomain.co.uk), one to the gateway
https://ipinfo.io/countries/gb router (gateway.myowndomain.co.uk) and another to the DNS server (dns.
shows the ASNs for the UK and
myowndomain.co.uk).
to whom they are assigned, e.g.
AS2856 is assigned to BT public You will need to check with an Internet registrar that your domain name,
Internet Service and covers a myowndomain.co.uk, has not been registered already. If it hasn’t, it may now
block of 11,215,616 IP addresses. be registered with your Internet registrar.
AS204160 is assigned to
SAMKNOWS-LTD and covers a You will also need to provide the IP address of your DNS server to your
block of 16 IP addresses. Internet registrar. Your registrar will then place an entry for your DNS server
(domain name and corresponding IP address) in the .co.uk second level
domain servers. After this is done, the IP address for your domain name and
therefore your DNS server can be obtained via the DNS system on request.
You must provide entries in your DNS server that map the hostname of your Web server, e.g. www.myowndomain.
co.uk to its IP address. You will need entries also for all your other publicly available servers.
Suppose that your ISP provided you with a block of 8 IP addresses expressed as 144.173.6.176/29 (a prefix). This
is interpreted as 29 bits for the network ID and 3 bits for the host ID (0..7) starting with host ID 144.173.6.176.
Therefore, the range of the block of IP addresses is 144.173.6.176 to 144.173.6.183.
Routers need to become aware directly or indirectly of your network ID and range of IP addresses so that packets
can be routed to your network. This is achieved by your ISP which will inform the ISPs it is connected to by
sending them 144.173.6.176/29 and they in turn will propagate your network ID and range of IP addresses to
others. Eventually all Internet routers will know a subset of your network ID and therefore will be able to forward
packets destined for your Web server, etc. Figure 9.3.1.17 shows a hierarchy of routers and their simplified routing
tables. Router 1 routes using the most significant byte of an IP address, Router 2 the next most significant and
so on. Router 5 is the gateway router for domain myowndomain.co.uk. It has the fully qualified domain name
gateway.myowndomain.co.uk.
Router 1
Router 2
144 2 Router 3
173 1 myowndomain.co.uk
Router gateway.myowndomain.co.uk
Router 4
port 144.173.6.176
no 6 3 10110 0 001 1 144.173.6.177
Part of network
ID to match 010 2 144.173.6.178
011 3 144.173.6.179
Routing path to network with prefix 144.173.6.176/29
100 4 144.173.6.180
and network ID in binary (29 bits)
10010000101011010000011010110 101 5 144.173.6.181
110 6 144.173.6.182
Router 5
Tasks
8 Visit the web page https://ipinfo.io/AS786 and discover how many IP addresses with the prefix
129.12.0.0/16 i.e. 16-bit NET ID 129.12 are allocated to the University of of Kent.
9 Visit the web page https://ipinfo.io/AS786 and locate the prefix 192.150.184.0/24 assigned to the University
of Manchester. Explain why for this prefix only 256 IP addresses can be allocated to this University. What is
the network ID for this block of IP addresses?
10 The web page https://ipinfo.io/AS786 shows that 512 IP addresses are allocated to the University of
Northumbria at Newcastle at prefix 192.173.2.0/23. How many bits are allocated for the host ID and how
many for the network ID? What is the network ID?
Learning objectives:
■ 9.3.2 Internet security
Firewalls
■ Understand how a firewall A firewall is a combination of hardware and software that isolates an
works (packet filtering, proxy
organisation’s internal network from the Internet at large, allowing some
server, stateful inspection)
packets to pass and blocking others. Figure 9.3.2.1 shows a firewall located
■ Explain symmetric and between an organisation’s local area network and the router that connects,
asymmetric (private/public via an ISP, the organisation’s network to the
key) encryption and key Internet. With all network traffic entering Key term
exchange and leaving the organisation’s network passing Firewall:
through the firewall, the firewall is able
■ Explain how digital certificates A firewall is a combination of
hardware and software that
to allow authorised traffic through whilst
and digital signatures are isolates an organisation’s internal
blocking unauthorised traffic. Two ways by
obtained and used network from the Internet at
which a firewall can control traffic are large, allowing some packets to
■ Discuss worms, Trojans and • traditional packet filtering pass and blocking others.
viruses, and the vulnerabilites
that they exploit • stateful inspection of packets
Information For example, a datagram arriving at the firewall from the Internet with IP
destination address of the Web server shown in Figure 9.3.2.1 and destination
TCP three-way handshake to
port number 80 will be allowed through the firewall interface connected to
establish a TCP connection:
the Web server. However, if the destination IP address was that of server D
Client Server connected to the firewall’s second interface then the datagram could be blocked
Connect Listening
request
SYN for all ports except those related to an FTP connection.
Accept
ACK A datagram arriving at the firewall from the Internet with IP destination
Connected ACK
address of the Web server shown in Figure 9.3.2.1 and destination port
Connected
number 23 might be blocked by the firewall from passing through the firewall
interface connected to the Web server.
Figure 9.3.2.2 TCP 3-way
handshake Information
ICMP message datagrams have a type and a code field, and contain the header and the first
8 bytes of the IP datagram that caused the ICMP message to be generated.
Information Ping is a software utility used to test the reachability of a host on an Internet Protocol (IP)
network and to measure the round-trip time for messages sent from the originating host to
ICMP message types:
a destination computer and back. Suppose ping sends an ICMP type 8 code 0 message (see
Table 9.3.2.1 - echo request) to the Web server in Figure 9.3.2.1. The ping ICMP type 8
ICMP
Code Description code 0 message is a request to the Web server to reply to the sender with ICMP type 0
type code 0 - echo reply. The reply will contain the originator’s IP address, i.e. the Web server’s
0 0 echo reply IP address. That’s fine because the ping echo request was addressed in the first place to this
destination Web server. However an attacker could do more by guessing the IP address range of the
3 1 organisation hosting the Web server and proceeding to carry out a ping sweep across this
host
IP address range in the hope that other hosts will be discovered. To prevent this, the firewall
unreachable
can be configured to block all ICMP echo request packets.
destination
3 3 port
unreachable Task
echo
8 0 1 Use ping to test if ICMP echo requests are blocked by a firewall:
request
(a) www.ucl.ac.uk (b) www.bristol.ac.uk
TTL
11 0 (c) www.educational-computing.co.uk
expired
Table 9.3.2.1 Some ICMP
Stateless packet filtering
message types
Stateless packet filters do not match return packets with outgoing packet
flow, and therefore ignore whether a connection has been established. Instead
Key term they focus on source and destination IP address, source and destination port
numbers and protocol type. A set of rules are constructed called a ruleset and
Stateless packet filtering:
Information about packets is not then expressed in the syntax of a particular firewall scripting language.
remembered by the firewall. This Suppose we want to allow inbound mail (SMTP, port 25) but only to the
type of firewall can be tricked
Internet gateway shown in Figure 9.3.2.1.
very easily by hackers because
allow/deny decisions are taken on Figure 9.3.2.3 shows how the rule for this could be expressed. Figure
a packet by packet basis and these 9.3.2.3(a) shows the table starting out in the default state which is to block
are not related to the previous
everything (*). Figure 9.3.2.3(b) shows the rule that allows an inbound
allowed/denied packets.
connection to port 25 on the Internet gateway. Port 25 is the Simple Mail
with an external host - see Figure 192.168.1/24 means all hosts on network with NetworkID 192.168.1
9.3.2.2. Figure 9.3.2.4 Firewall ruleset for SMTP
Check
Source Destination Source Destination Flag
Action Protocol connection
address address port port bit
table
External to
Allow 95.144.156/24 TCP >49152 80 Any
95.144.156/24
External to
Allow 95.144.156/24 TCP 80 >49152 ACK X
95.144.156/24
Table 9.3.2.2 Access control list for stateful filter - this is scanned for a match from row 1 downwards
of request they receive from clients behind the firewall by the application protocol that is used, e.g. SSH, HTTP
request.
A content-filtering web proxy server provides control over the content that may be relayed in one or both directions
by filtering by URL (and DNS) or by keywords in the content.
The proxy server may also scan incoming content in real time for viruses and other malware and block such content
from entering the network.
Question
1 Stateless packet filtering, stateful inspection packet filtering and proxy servers are three kinds of firewall
techniques. Explain by examples how they differ from each other.
Encryption
Encryption is the process of obtaining ciphertext from plaintext. Plaintext is understandable (English) text. The
intention of encryption is to render plaintext incomprehensible to all but those granted the means to reverse the
process, i.e. decrypt the ciphertext.
The encryption process requires two inputs: the plaintext and the encryption key.
Key term
The decryption process also requires two inputs: the ciphertext and the decryption
key. Symmetric encryption:
Symmetric encryption uses the
Encryption is covered in Chapter 5.6.10. same secret key to perform both
the encryption and decryption
Symmetric encryption
processes.
Shared private key
In symmetric encryption the communicating parties use the same key for
encryption and decryption. Symmetric key encryption is also known as private key or secret key encryption
because the key used must be private and known only to the communicating parties. If not, then anyone
intercepting encrypted messages can use a knowledge of the key and the encryption algorithm to decrypt the
messages.
For example, if Alice wanted to use symmetric encryption to send an encrypted message to Bob then Alice would
use the private key k, agreed with Bob, to encrypt the plaintext form of the message. Bob would use the same
private key k to decrypt the received encrypted message. Bob could reply with his own encrypted message which he
has also encrypted with private key k (Figure 9.3.2.6).
Using symmetric encryption, Alice and Bob are able to communicate securely through a communication channel.
However, there is a potential problem with use of a single private key. Let’s suppose that Alice gets to choose the
key. She now has to communicate the key to Bob. They could agree to meet so that Alice could, by whispering into
Bob’s ear, communicate the key securely. However, in a networked world, Alice and Bob may never meet and may
never communicate except over a network, e.g. the Internet. However, to communicate the key securely to Bob via
a network connection, Alice needs
to secure the channel through the Message M Private key, k Private key, k
Message M
network.
Encrypted with key k Encrypted message M Decrypted with key k
How can she do this if she hasn’t
yet distributed the private key? Communication channel
She needs the secure channel Figure 9.3.2.6 Secure communication channel using symmetric encryption
to distribute the key. This “chicken and egg” situation is known as the key distribution problem of symmetric
encryption.
Asymmetric encryption (public key/private key encryption)
Key term Secure communication in the world of online transactions
Asymmetric encryption:
The key distribution problem of symmetric encryption becomes an even more
Asymmetric encryption uses two
keys, a public key and a private key.
challenging one in the world of online transactions which the World Wide Web
The public key and the private key has made possible. Many transactions take place online through online stores
are mathematically related but where goods may be bought with a credit or debit card. All these transactions
different. If the public key is used need to take place through secure channels of communication to protect a
for encryption, the private key
purchaser’s payment details. Securing each channel with a separate symmetric
must be used for decryption.
encryption private key shared in advance between seller and customer would be a
Public key encryption: nightmare:
The public key is freely shared • The seller would have to store a large number of private keys, one for each
with any party. Documents/
customer buying goods.
messages destined for the owner
of a public/private key pair are • Each key would have to be distributed to the corresponding customer who
encrypted with the public key. would have to remember the key and for which seller.
This public key is linked A mechanism for secure communication is needed which
mathematically with its paired
private key. The private key is
• reduces the number of keys that need to be remembered
used to decrypt the document/ • allows two parties to exchange information secretly, but with no
message encrypted with the pre-arranged symmetric encryption private key
paired public key. The owner of
the private key must be the only
One solution is public key/private key encryption to secure a communication
one to know this key. channel. This secure channel can then be used to distribute a temporary
If the key pair is chosen well: symmetric encryption key to be used for securing credit/debit card details.
• The private key cannot be This temporary shared key is called a session key because it is only used for the
derived from the public key
duration of the transaction.
• The public key encrypted
messages can only be decrypted Public key encryption
by the private key with which it In public key encryption two mathematically related keys are used:
is paired mathematically.
• a public key p
k
Information • a private or secret key s
k
Key pair:
Amazon EC2 elastic cloud
Public key encryption has two clear advantages over private key encryption:
uses public key cryptography • The online seller has only to store a single private key s rather
k
to encrypt and decrypt login than sharing, storing and managing N different secret symmetric
information. The keys that Amazon
encryption keys (i.e. one for each buyer).
EC2 uses are 2048-bit SSH-2 RSA
keys. • The number and identities of potential buyers need not be known at
the time of key generation.
Public key encryption is an asymmetric encryption scheme, asymmetric because only one of the paired keys, the
private key s is secret. The public key p can be freely shared with any party. Thus an online seller makes their
k k
public key p available to any buyer. A buyer can then use the seller’s public key to establish a secure channel over
k
which they can send their credit card information securely to complete a purchase. The seller uses their private key
s , which only they know, to decrypt the buyer’s credit card information to process the transaction or, if symmetric
k
encryption is used for the latter, to set up a secure means of communicating a shared private session key.
• Bob then encrypts these blocks as Ci = Mie mod N and sends the encrypted blocks to Bob.
Alice decrypts the encrypted message blocks as follows:
• Alice decrypts each Ci to recover Mi using her private key d by calculating Mi = Cid mod N
• Alice then converts the string of integer codes, Mi, back into their equivalent characters to recover the
plaintext form of the message Bob has sent her.
Example
Suppose Alice chooses primes p = 7 and q = 11. So N = 77, (p – 1)(q – 1) = 60 and she chooses e = 7, since 7 and 60
are relatively prime.
Alice then calculates her private key using ed = 1 (mod (p -1)(q - 1) to be d = 43, since 43 x 7 = 301 = 1 mod 60
Hence Alice’s public key is the pair (N, e) = (77, 7) and her private key is d = 43. (Use Windows calculator in
If Bob wants to send the plaintext message M = 4 to Alice he encrypts it as Scientific mode for the mod
ciphertext C = Me mod 77 = 47 mod 77 = 16384 mod 77= 60 (mod 77). operation)
Alice then decrypts C using her private key to recover the message
M = Cd mod 77 = 6043 mod 77 = 4 (mod 77)
Background
Public key/private key encryption schemes rely on the discrete logarithm problem to make it difficult to
discover the private key. A very simple example that illustrates this problem is as follows:
To encrypt integer 19 raise it to the power 43 and then find the remainder after dividing by 77, the result is the
encrypted value 61
1943 mod 77 = 61
To recover the unencrypted value 19, an exponent must be found for which
61d mod 77 = 19
If we try d = 1, 2, 3, 4, 5, 6, 7 we find that 7 does.
Thus, 617 mod 77= (1943)7 mod 77 = 1943x7 mod 77 = 19
by replacing 61 by (1943).
Now suppose, we encrypt 2137 by raising it to the power of 17 and finding the remainder after division by
3233, the result is the encrypted value of 2137
213717 mod 3233 = 166
To recover the unencrypted value 2137, an exponent d must be found for which
166d mod 3233 = 2137
If we try d = 1, 2, 3, ........., 2753 we find that 2753 does.
Now suppose we use exponent 65537 and a 2048-bit modulus such as the one shown in the information panel
on next page labelled Public modulus. We would find that d would be the 2048-bit value labelled Private
exponent in the information panel. Using the public exponent 65537 and the public modulus in the margin
we could encrypt integers up to 256 digits long. To decrypt we would use the private exponent shown in the
margin and the same public modulus. The public key is (public exponent, public modulus) and the private key
is (private exponent).
The discrete logarithm problem is the difficulty with which the private exponent can be discovered given only
the public exponent, the public modulus and the result from encryption. The last example illustrates that this
becomes an exceedingly difficult task when the number of bits used for the public key and the private key is a
large number.
Tasks Information
The discrete logarithm problem: Public exponent:
This is the problem of finding x where 65537
ax ≡ b (mod n) Public modulus:
Watch the Khan Academy video “The discrete logarithm problem” 2945766458199006354518714370
https://www.khanacademy.org/computing/computer-science/cryptography/modern- 789937313459038969836551741
crypt/v/discrete-logarithm-problem 474365312554100524284016689
991569699787388856353829206
RSA encryption: 878996051572086976167536159
Watch the Khan Academy video “RSA encryption part 1” 367501426841421545476777770
https://www.khanacademy.org/computing/computer-science/cryptography/modern- 911830025184723218753394504
crypt/v/intro-to-rsa-encryption 845305877537841256481722403
521947540091647129987550526
501191700526487333570141735
Questions 966271760998390300210716322
6848818840015822496173899627
2 Two computers, X and Y, communicate securely using public/private 2694978348391161716080846065
key encryption. X and Y each has a public key and a private key. 8137438912535222664160206809
X encrypts a message that it sends to Y using Y’s public key. 110088175424068694053002994
332201096356829801571379647
Explain why the message should not be encrypted with:
912703834109764508780629994
(a) X’s private key 906036475520264572478205942
(b) X’s public key 172563985777892068763898076
972147981555129254575283743
398768074422492663494096553
654793297152972338693514772
Task 6813268287153607983,
Explore the Wolfram programming lab: Private Exponent:
The Wolfram programming language is a useful language with which to explore 1867954933393873782460117671
encryption, factoring, modular arithmetic and a lot of other concepts. 258803650956417619184085589
https://lab.wolframcloud.com/app/ 395176368450238942707099281
362736371572765095935920688
763253092317817272007727966
Key exchange 049630350743485307336678984
Diffie-Hellman key exchange 444723319447437745471314964
Diffie-Hellman exponential key agreement or key exchange provides a way 559885586137870316567243231
236783738516085134016244637
for Alice and Bob to agree on a key k while communicating over an insecure
080374821711090843470222168
network channel. The key k is used only for one communication session. It is 596156702985658545495780230
then discarded. Hence the name session key. 0706215013490208463305771899
8128856073063404802108286698
A modulus N is made available publicly for network users to use to secure their
8921193324834578936195915247
communications. 363635418311637187885848492
Alice privately selects a large random number sA, and calculates 541980772633601888977561074
s
pA = 2 A (mod N). 349848200284511774738451327
Alice’s private key is sA. Alice sends pA to Bob. 825749779060874980402587688
552346646502447343908134657
An eavesdropper, Eve, who intercepts pA will find it extremely difficult to 726546395530654563666934925
s
discover Alice’s private key sA from a knowledge of 2 A (mod N). 365767118252566387299259415
Bob also privately chooses a large random number sB and calculates 793638059269399354303032695
s 3854631035757879505
pB = 2 B (mod N). Bob sends pB to Alice.
Alice has her private key sA and Bob’s public key pB. Task
Bob has his private key sA and Alice’s public key pA.
Diffie-Hellman key exchange:
Bob and Alice can now calculate their shared secret key, k. Watch the Khan Academy video
sA sB sA “Public key encryption: What is
Alice uses her private key sA to calculate kA = pB = (2 (mod N)) mod N
it?”
sB sA sB
Bob uses his private key sB to calculate kB = pA = (2 (mod N)) mod N https://www.khanacademy.org/
s s s s computing/computer-science/
But (2 B (mod N)) A = (2 A (mod N)) B, therefore kA = kB = k, the shared secret key.
cryptography/modern-crypt/v/
The shared secret key k may now be used for symmetric encryption of messages diffie-hellman-key-exchange-
between Alice and Bob. part-1
SSH
SSH, or secure shell, is a secure protocol and the most common way of safely
administering remote servers. Symmetric keys or shared secret keys are used by SSH in order to encrypt the entire
connection between client and server.
The secret key is created through a process known as key exchange. This exchange results in the server and client
both arriving at the same key independently by a process similar to that described in the previous section.
The symmetric encryption key created by this procedure is session-based and is the key used to encrypt the data sent
between server and client, e.g. credit card details.
SSH utilizes asymmetric encryption during the initial key exchange process used to set up the symmetrical
encryption key used to encrypt the session between two parties. The two parties exchange public keys (see pA and pB
in previous section) in order to produce the shared secret key k used for symmetrical encryption.
Question
3 Describe two different methods for communicating a shared secret symmetric encryption key k via an
insecure network channel.
Digital certificate
When a website is visited that supports the Secure HyperText
Cannot guarantee authenticity of the domain
Transfer Protocol HTTPS, to which encrypted connection is established
e.g. https://www.google.co.uk, the identity of the website is
“proved” with public key encryption. It is important that the Application: Google Chrome
URL: www.dingleydell.com
authenticity of the website is checked, i.e. that it really is the Reason: Invalid name on certificate. Either
genuine site and not a man-in-the-middle attacker placed the name is not the allowed list, or
was explicitly excluded.
between a visitor and a website, impersonating both.
With a man-in-the-middle attack, the browser thinks it is Disconnect
talking to the web site on an encrypted channel, and the
website thinks it is talking to the browser, but they are both Continue
talking to the attacker who is sitting in the middle. All traffic
passes through this man-in-the-middle, who is able to read and View certificate
modify any of the data.
Figure 9.3.2.8 Digital certificate warning that
Operating systems and browsers typically have a list of
web site authenticity cannot be guaranteed
certificate authorities that they implicitly trust. If a website
presents a certificate that is signed by an untrusted certificate authority, the Key term
browser warns the visitor that something could be amiss - Figure 9.3.2.8. Digital certificate:
Certificates are files containing information about the owner of a website, Digital certificates are files
and the public half of an asymmetric key pair (e.g. RSA). A certificate containing information about the
owner of a website, and the public
authority (CA) digitally signs the certificate to verify that the information in
half of an asymmetric key pair
the certificate is correct. By trusting the certificate, you are trusting that the
(e.g. RSA). A certificate authority
certificate authority has done its due diligence. (CA) digitally signs the certificate
A website that supports HTTPS should have a certificate and a corresponding to verify that the information in
public key. This will enable a connection to be made between a web browser the certificate is correct. By trust-
ing the certificate, you are trusting
and the website using the Transport Layer Security (TLS) protocol. The
that the certificate authority has
browser must also support the TLS protocol. done its due diligence.
When a web browser uses HTTPS to visit a website such as https://www. A website that supports HTTPS
google.co.uk, a TLS connection is established between the web browser and should have a certificate and a
the website. TLS is used to encrypt data sent between both and to prove the corresponding public key. This
will enable a connection to be
identity of the server.
made between a web browser
Question and the website using the
Transport Layer Security (TLS)
4 (a) What is a digital certificate?
protocol.
(b) Explain how it is used to authenticate a website that supports
HTTPS.
Authentication Information
The web browser starts the TLS connection by telling the website which Digital certificate authority:
CAcert.org is a community-driven
ciphersuites it supports i.e. it tells the website which types of encryption it is
Certificate Authority that issues
able to use. The website https://cc.dcsec.uni-hannover.de/ will report what certificates to the public at large
ciphersuites your browser supports, e.g. ECDHE-RSA-AES256-SHA which for free - see www.cacert.org.
means Elliptic Curve Diffie-Hellman Exchange which is used to share a
symmetric key, RSA for authentication, AES256 is the symmetric encryption CAcert’s goal is to promote
awareness and education on
algorithm to use and SHA is the hash function used to create a message digest.
computer security through the
The web browser generates a 128-bit random number called a nonce which it use of encryption, specifically
sends to the website. The website encrypts this nonce using its RSA private key by providing cryptographic
certificates. These certificates
and sends the encrypted nonce to the web browser. The web browser decrypts
can be used to digitally sign
this encrypted nonce using the trusted-certificate-authority-validated public key and encrypt email, authenticate
belonging to the website. A match confirms that the website knew the private and authorize users connecting
key half of the public key/private key pair and therefore is authentic. The same to websites and secure data
nonce is never used twice to prevent a bogus site replaying the encrypted nonce transmission over the Internet.
Any application that supports the
which it had intercepted and recorded on a previous occasion.
Secure Socket Layer Protocol
Digital signature (SSL or TLS) can make use of
A digital signature is a cryptographic technique that can be used in a digital certificates signed by CAcert.
world when you want
• to indicate that you are the owner or creator of a document, or
• to affirm that you agree with a document’s content.
For Bob to know that a document genuinely came from Alice, he needs some way to authenticate the document.
This is what Alice does to make this possible:
• Alice digitally signs the Signed document: Enc(M)
document with her private key Document: M Digital signature
Message digest
Key term
Encryption and decryption are computationally intensive. Message digest:
Digitally signing a document by encrypting the whole document is therefore A message digest is a a fixed-
expensive computationally. length “fingerprint” of a message/
To reduce the computation overhead a hash function H is used. document M of arbitrary length.
It is calculated by applying a hash
This function takes a message/document M, of arbitrary length and computes
function H to message/document
H(M), a fixed-length “fingerprint” of the message/document M called a message M to compute H(M), the
digest. message digest.
Long document: M
Alice now digitally signs the
Alice
message digest rather than the
A VERY LONG REPORT
message/document itself to create a I have so much to tell
you that I am afraid the
digital signature. information in this Fixed-length hash
As H(M) is generally of much report extends to a H(M)
hundred pages. grF54Try73*c£$jqap
Hash function
shorter length than the original ................. Ud4a£z\98m>Dpw2
.................
message/document M, creating the
digital signature consumes much
H(M)
less computational effort.
Alice now sends the document M Send M
digitally signed hash H(M)
Send Digital signature
followed by the digital signature to Digital
signature
Bob as shown in Figure 9.3.2.11. Bob
Hv#(&56rEalpz1+\;9 Encryption
3VxC@=Lqyxa56$^v algorithm
On receipt of M, Bob uses the receives Alice’s private
hashing function H to produce a
M key, sA
Digital signature
hash of document M as shown in Figure 9.3.2.11 Using Alice’s private key to create a digital
Figure 9.3.2.12. signature from the hash of document M
On receipt of the digital Digital
signature
signature, Bob decrypts Bob (Signed hash)
M receives Hv#(&56rEalpz1+\;9 Decryption
it using Alice’s public key Digital from 3VxC@=Lqyxa56$^v algorithm
pA to obtain the message signature Alice Alice’s public
key, pA
digest or hash produced Long document: M
by Alice if she genuinely grF54Try73*c£$jqap Fixed-length hash
Ud4a£z\98m>Dpw2
signed the document. If A VERY LONG REPORT
I have so much to tell
the two hashes match then you that I am afraid the
information in this
Bob can conclude that report extends to a
hundred pages.
Alice was the signer of the .................
.................
document and that the
document has not been
tampered with whilst in Fixed-length hash
Match Therefore
transit. Hash function grF54Try73*c£$jqap Compare document has
Ud4a£z\98m>Dpw2 been genuinely
signed by Alice
Figure 9.3.2.12 Verifying a signed
document on receipt of the document Don’t match Therefore document has been
and its digital signature created from the signed by someone other than
Alice or document tampered with
document’s message digest or hash
Single licence - Abingdon School 412
9 Fundamentals of communication and networking
Question
6 Explain how signing a long electronic document or message with a digital signature might differ from
signing a short document or message.
7 Two computers, X and Y, communicate securely using public/private key encryption. X and Y each have
a public key, a private key, and a hash function H that generates a message digest of a plaintext message
of arbitrary length. Computer X sends a plaintext message and a digital signature in encrypted form to
Computer Y as shown in Figure 9.3.2.13. Computer Y processes the received transmission as shown in
Figure 9.3.2.13.
State what processing takes place at each of the stages indicated by a ringed numeral. Where a key is used,
specify which key.
Computer X
Transmission
path
Worms
Worms are malicious software that can enter a computer from the Internet Key term
without any explicit user interaction by exploiting a vulnerability in a running Worm:
network application program, e.g. the Conficker computer worm that targeted A computer worm is a self-
flaws in Windows OS software to propagate. The worm in the newly infected contained program that attacks
a system and tries to exploit a
computer scans the Internet, searching for other hosts running the same
specific vulnerability in the target.
vulnerable network application. On discovering other vulnerable hosts, it sends a
It replicates itself in order to
copy of itself to those hosts. spread to other computers.
Unlike a computer virus, a worm does not need to attach itself to an existing Unlike a computer virus, it does
program or file because they are self-contained programs. Worms are one way of not need to attach itself to an
existing program or file but
installing a backdoor in the infected computer to allow the creation of a “zombie”
instead it exploits vulnerabilities
computer under control of the worm author. Networks of such machines are often in network application software
referred to as botnets and are very common use is the sending of junk email called running on the infected host.
spam. The aim of a worm is often to
take over a computer for the
Trojans purposes of creating a botnet for
A Trojan horse or Trojan is a type of malicious software that is often disguised as sending spam and Distributed
Denial of Service (DDoS) attacks.
legitimate software. Users are typically tricked into loading and executing Trojans
on their systems. Once activated, Trojans can enable cybercriminals to spy on
you, steal your sensitive data, and gain backdoor access to your system. Unlike
computer viruses and worms, Trojans are not able to self-replicate. Key term
Trojan:
Trojans are classified according to the type of actions that they can perform on the
A Trojan horse or Trojan is a type
infected computer. A selection is shown below. of malicious software that is often
• A Backdoor Trojan enables remote control over the infected computer disguised as legitimate software.
Unlike computer viruses and
by a cybercriminal or hacker to do anything they wish on the infected
worms, Trojans are not able to
computer – including sending, receiving, launching, and deleting files,
self-replicate.
displaying data, and rebooting the computer. Backdoor Trojans are Users are typically tricked into
often used to link a group of victim computers to form a botnet or loading and executing Trojans on
zombie network that can be used for criminal purposes their systems. Once activated,
Trojans can enable cyber-criminals
• Trojan-Banker programs are designed to steal account data for online
to spy on you, steal your sensitive
banking systems, e-payment systems, and credit or debit cards from data, and gain backdoor access to
infected computers your system.
• Trojan DDoS are programs that conduct DDoS (Distributed Denial
of Service) attacks against a targeted web address. They send multiple
requests from your computer and several other infected computers that attack and overwhelm the target
address leading to a denial of service at the target address computer, typically a server
• Trojan-Downloaders can download and install new versions of malicious programs onto your computer
– including Trojans and adware
• Trojan-Ransom can modify data on your computer – so that your computer doesn’t run correctly or you
can no longer use specific data. The criminal will only restore your computer’s performance or unblock
your data, after you have paid them the ransom money that they demand
• Trojan-Mailfinder is a program that can harvest email addresses from your computer.
inadvertently runs the malware on their computer. Once executed, the virus is of user interaction to infect a
user’s computer because the user
able to replicate and then spread by sending an identical email with the same
needs to be tricked into opening
malicious attachment to, for example, every recipient in the user’s address book. the file to which the virus is
Viruses piggy-back on seemingly legitimate files but they require some form of attached. Viruses are not stand-
alone programs but are always
user interaction to infect the user’s computer.
embedded in another program or
file. Once the virus is executed
Question it can replicate and infect other
8 Explain the differences between worm, virus and Trojan malware. computers.
Code quality
One way of securing computers against malicious attacks is to improve the quality of the code, from operating
systems to application
programs, which executes on
computers. Attackers exploit
vulnerabilities in code in order
to get a computer to execute
their malicious software.
For example, the command
shell in the Windows
operating system is a separate
software program, CMD.
exe, that executes programs
and displays their output as
individual characters on the
screen. It is known to have
vulnerabilities.
For instance, one of the
commands it can execute is
the echo command which
is shown in Figure 9.3.2.14
being used once to output
the string “Hello World” and
once to output the name of
the current directory stored
in the environment variable
%CD%. On the second
occasion, a new directory has Figure 9.3.2.14 Exploiting a vulnerability in cmd.exe, the command line
been created beforehand using interpreter in the Windows operating system to run the ping program
415 Single licence - Abingdon School
9.3.2 Internet security
the command md "NewTest&ping 8.8.8.8" and made the current directory. The command shell program CMD.
exe is tricked into executing ping 8.8.8.8 because & is interpreted by the command shell as the separator of multiple
commands on one command line. CMD.exe runs the first command echo c:\test\NewTest1, and then the second
command, ping 8.8.8.8 because NewTest is separated by & from ping 8.8.8.8 in %CD%.
Figure 9.3.2.15 shows the same exploit causing calc.exe to execute. A third example could replace calc.exe with
malware.exe.
CMD.exe can also execute batch files. There are batch files in systems that have been running for years which
contain echo %CD% commands probably with a pipe, e.g. echo %CD% > logfile.txt.
If Microsoft’s recommended fix to change echo %CD% to echo "%CD%" has not been done then running any batch
file that has not been fixed could allow malware to execute.
The contents of environment variable %CD% can be changed with the SET command outside of the batch file
containing echo %CD%,
e.g. SET CD=c:\test\NewTest&malware. If this is the value of %CD% when the batch file is executed the malware
program will be executed.
Buffer overflow
A common vulnerability exploited by worms and viruses is buffer overflow. The Stuxnet worm used buffer overflow
and some other techniques. It was widely suspected of targeting Iran’s nuclear enrichment programme and may
have destroyed 1,000 centrifuges, reduced output and sowed chaos.
Another way that malicious code can be executed is to place this code in the buffer that is being caused to overflow
and to overwrite the return address so it points back into the buffer.
Patching or updating software is usually an effective way to remove vulnerabilities to worms and viruses. Coding to
standards that avoid creating code vulnerabilities in the first place is a better way.
Background
Figure 9.3.2.16 shows a simplified case of how buffer overflow can occur when a call is made to a library routine input(Password) that reads
keyboard input from a user one character at a time. The routine copies each character’s byte-value, in turn, into the memory reserved in the stack
frame for the call to IsPasswordValid() labelled Password (the stack frames for the calls to input and stringCompare are not shown). Seven
bytes have been allocated in this current stack frame to Password according to the declaration "chararray[7] Password;" in the program
source code. These seven bytes constitute a buffer. Figure 9.3.2.16 shows the result of the user typing password "1234567" and then on another
attempt to login, password "123456789ABCDEF".
Notice that the library routine input stores the first character of the password in the byte pointed to by Top of Stack Pointer, the next in the
byte above and so on. Notice also that the longer password overwrites the Previous Stack Frame Pointer and the Return Address areas of
the stack frame. This is because the library routine input (Password) does not apply array bounds checking when the 8th byte and subsequent
bytes are written to Password. Buffer overflow occurs and the return address is corrupted. This return address is used to return to the routine
main but this will not happen because the return address now points to somewhere else in the memory of the computer. It is likely that this will
cause an exception (Figure 9.3.2.17). Now suppose the 15 character password is chosen so that the return address on the stack is overwritten
with the address of the else output ("Access allowed"); line of code. The result will be that the user will be granted access to the system.
In a different scenario, buffer overflow could replace the return address on the stack with the address of the malicious code of a worm or a virus.
Previous Stack Frame Previous Stack Frame Pointer (4 bytes) int main (void)
{
boolean PasswordValid;
output("Enter password: ");
PasswordValid (4 bytes)
PasswordValid <─ IsPasswordValid();
Stack Frame Pointer if (PasswordValid = False)
F then
Return Address (4 bytes) E {output ("Access denied");
Main
D exit(-1);
C
B
}
A else output ("Access allowed");
Previous Stack Frame Pointer (4 bytes)
9 }
Current Stack Frame 8 True or False
7 7 returned boolean IsPasswordValid (void)
6 6
5
{
5
4 4 chararray Password[7];
3 3 input(Password);
Password
2 2 return stringComparison(Password, "LetMeIn");
(7 bytes)
Top of Stack Pointer 1 1 }
Password validator
OK
Questions
9 Worms, viruses and trojans exploit vulnerabilities in computer systems. Describe one vulnerability that is
exploited by a
(a) worm (b) virus (c) Trojan.
10 How might the quality of operating system and application code affect whether a system is vulnerable to
worms, viruses and Trojans?
11 How can monitoring and protection be used to prevent worms, viruses and Trojans from infecting systems?
Networking protocols: Application Telnet, FTP, email(SMTP, POP3), Web browsing (HTTP)
Networking protocols make
possible communication Transport TCP
between processes executing on
Network IP (Internet layer)
different hosts whilst hiding the
complexities of the underlying Link Ethernet, PPP, WiFi, DOCSIS (cable TV)
network from these processes.
TCP/IP protocol suite: Figure 9.4.1.1 The four layers of the TCP/IP protocol suite and stack
The TCP/IP protocol suite
consists of four conceptual
Application programs interact with the software stack via an Application
layers: application, transport, Programming Interface (API). The de facto standard is the socket API.
network or IP, and link - see RFC
The following Python code snippet shows a client application using the socket
1122 https://tools.ietf.org/html/
rfc1122#page-27.
API to set up a socket to send a message to a server:
clientSocket = socket(socket.AF_INET, socket.SOCK_STREAM)
TCP/IP protocol stack:
message = "Hello Server"
Implements the TCP/IP protocol
suite in software. clientSocket.sendTo(message, (serverName, serverPort))
Key term Figure 9.4.1.2 shows a client process and a server process that use the socket
API from the TCP/IP protocol stack to send and receive messages via a TCP/IP
Application layer:
connection pipe established between client and server.
Application layer protocols are
used to exchange data between
programs running on the source Host B
Host A
and destination hosts. It is the
application layer that provides the
interface between these programs
and the underlying network over
which the programs’ messages are Socket API used by both client
Client process and server processes to send Server process
transmitted, e.g. HTTP message
e.g. Web browser
and receive messages e.g. Web server
GET / which fetches the default Application
Web page from a Web server. Messages Messages
Modules
Key term within Client Server
operating socket byte pipe socket
Transmission Control
system or
Protocol (TCP): library of connection pipe
TCP/IP protocol TCP/IP protocol
TCP enables applications routines stack stack
executing on two hosts to
establish a connection and End-system End-system
exchange application-layer
Figure 9.4.1.2 Sending and receiving messages using the socket API
messages through a reliable
byte-stream channel (pipe) for
data flows between the two end Application layer
systems.
A process in one end-system (host) uses the application layer of TCP/IP to
exchange packets of information with a process in another end-system. The
Key term packets of information at the application layer are called messages.
Function of Transport layer: The application layer uses different application-layer protocols for different
The basic function of the
applications. For example, if the application is designed to enable Web pages
transport layer is
• to accept messages/data from
to be fetched from a Web server then the application will use either the HTTP
the layer above it application-layer protocol or the HTTPS application-layer protocol. An
• split these into smaller units application-layer protocol defines the kind of messages to send. In the case of
called segments if necessary HTTP or HTTPS, one such message could be a GET message.
• pass these segments to the
network or IP layer
Transport layer
• ensure that all the segments The transport layer of the protocol stack is a piece of software in each
arrive correctly at the other host. This software implements the transport protocol known by the name
end Transmission Control Protocol (TCP). TCP enables applications executing
• reassemble the received
on two hosts to establish a two-way connection and exchange application-layer
segments, which it gets from
the network layer, in the messages through a reliable byte-stream channel (pipe) for data flows in either
correct order to form the direction between the two end-systems as shown in Figure 9.4.1.2. It also
message/data to pass to the allows the connection to be terminated.
layer above.
TCP breaks long messages into shorter segments which it sends as separate
The transport layer is a true
transport-layer packets known as TCP segments.
end-to-end layer which carries
messages/data all the way from The application at the sending side (e.g. a Web browser using the application-
the source to the destination. layer protocol HTTP) pushes messages (e.g. GET) through a TCP/IP socket.
The transport-layer protocol TCP has the responsibility of getting the messages Key principle
to the socket of the receiving application process, e.g. a Web server listening on
End-to-end principle:
port 80. Port numbers such as port 80 are 16-bit numbers used for application The end-to-end principle of
and service identification on the Internet. the Internet requires that the
two endpoints, the hosts, are
TCP does everything in its control to guarantee delivery of the application-
responsible for establishing,
layer message and also guarantee that the received TCP segments will be supervising and maintaining
reassembled in the correct order to form the message to be passed to the a connection between two
application-layer and then the corresponding application process. communicating processes,
one on each host. This is done
Once the TCP has established a connection: by a piece of software in each
• it monitors the connection for transmission errors and responds host known by the name
Transmission Control Protocol
when an error is detected by retransmitting the segment that suffered
(TCP).
the error
• it detects when a connection is broken Key term
Network or IP layer:
• it performs flow control by speed matching sender and receiver
The network or IP layer of the
• it provides congestion control when the network is congested. TCP/IP protocol stack in hosts
and routers is responsible for
0 15 16 31 moving IP-layer packets from one
16-bit source port number 16-bit destination port number host to another without regard to
whether these hosts are on the
32-bit sequence number
same network or not.
32-bit acknowledgement number 20 bytes It adds source and destination
4-bit header IP addresses to packets on their
length way from the transport layer to
16-bit TCP checksum the link layer, and removes source
and destination IP addresses from
Figure 9.4.1.3 TCP header packets on their way from the link
layer to the transport layer.
Both routers and hosts are assigned IP addresses by which they may be identified. An IP
address (Internet Protocol address) is a logical 32-bit (IPv4) or 128-bit quantity (IPv6). The
IP layer in the sending host attaches the source (sending host) and destination (receiving
host) IP addresses to the packets handed to it from the transport layer. The IP layer then
passes these packets to the layer below, the link layer. The IP layer also receives packets from
the link layer, removes the source and destination IP addresses then passes them to the
transport layer.
The format of an IP packet is universal so that all routers recognise it (see Chapter 9.3.1).
This makes it possible for IP packets to pass through almost every network of networks, e.g.
the Internet.
Both hosts and routers need to use the network or IP layer of the TCP/IP protocol stack
but since the job of a router is dedicated to routing packets, a router only requires use
of the network and link layers of the TCP/IP stack. The IP layer in a router must have
sufficient knowledge of other routers and links in its internet to be able to make routing
decisions for packets that pass through it.
Together TCP and IP hide the differences between the underlying networks through
which packets pass when going from source to destination host.
Link layer
The link layer handles all the physical details of interfacing with the network cable or
wireless connection. It includes the network interface card (network adapter) and a device
driver. TCP/IP protocol supports many different types of link layer, depending on the type
of networking hardware being used. One example is Ethernet.
The link layer adds source and destination hardware addresses (e.g. MAC addresses) to
packets that it receives from the IP layer then dispatches the packets onto the local cable or
wireless connection.
If the packet is destined for a host on another network, the link layer destination address
is the hardware address of the gateway (router) to the internet which the other network is
connected to.
In an Ethernet local area network (LAN) these hardware addresses are Ethernet
Key term card addresses, or MAC addresses - see the following section on MAC
Link layer:
addresses. Figure 9.4.1.4 shows a packet despatched by the link layer of a
The link layer handles all the
physical details of interfacing with host with IP address 174.89.0.54 to a remote host with IP address 210.5.0.67.
the network cable or wireless Figure 9.4.1.4 shows the first, second and last hop of many hops.
connection.
Note that the link layer hardware address changes from hop to hop whilst the
The link layer adds source and
destination hardware addresses source and destination IP addresses remain constant. This is because the link
(e.g. MAC addresses) to packets layer’s role is to stream bytes between directly connected machines, hosts and
that it receives from the IP layer routers. It is the link layer that puts bits onto the network cable or wireless
then despatches the packets
connection. Sending to a remote machine is done in hops where each hop is
onto the local cable or wireless
a direct connection (link) between a host and a router, a router and a host, a
connection.
router and another router, or two directly connected hosts.
IP address IP address
174.89.0.54 IP layer IP layer
210.5.0.67
First hop
identified by its two endpoints.
For example,
Source Link-layer Destination Source Destination
<174.89.0.54:49717, 210.5.0.67:8080>
address Link-layer address IP address IP address
00-03-47-C9-69-52 00-03-47-B6-21-46 174.89.0.54 210.5.0.67
Second hop
Router Network
Figure 9.4.1.4 TCP/IP protocol stack and the role of the link layer in the communicating hosts and
intermediate routers
Questions
1 Describe three tasks performed by the transport layer of the TCP/IP protocol stack.
3 Explain why the source and destination IP addresses of a packet remain the same whilst link layer addresses
need to change when a packet is sent from a host to a server on a different network.
4 Describe the role of the different layers of the TCP/IP stack in each of the host, the server, and intervening
routers when a Web browser running on a host uses the TCP/IP protocol stack to send an HTTP GET
message to a Web server running on a different network.
Application Service
Endpoint Endpoint
Web browser Port no 49717 Communication link Port no 8080 Web server
A procedure called bind, performed on the client side and the server side, tells the operating system which local
IP address/port no pair to associate with the socket before a connection is established (on the client side the bind
procedure may be called as part of the connect operation). On the server side, the bind procedure establishes the IP
address/port no pair that the server listens on to accept client connections.
Figure 9.4.1.6 shows an HTTP server listening on port 8080 for connection requests from client hosts. The client
shown is called localhost. It has IP address 127.0.0.1 (known as the loopback IP address). A TCP socket has been
created and the bind operation applied to assign the socket to the IP address/port no pair 127.0.0.1:49717 (client
port numbers are in the range 49152-65535) . The colon symbol (:) is used to separate the IP address 127.0.0.1
from the port number 49717. The client host sends a TCP connect request through this socket to the listening
socket of the HTTP server. This server has been set up on the same machine so Information
also has localhost as its computer name and the same IP address 127.0.0.1. The
Normally, the two IP addresses in
name loopback IP address derives from the fact that TCP/IP stack packets sent
an end-to-end connection will be
to the network adapter of the machine are looped back to another TCP/IP stack
different, but in this example the
within the same machine. The server has created a listening socket that is bound loopback address is being used
to the IP address/port no pair 127.0.0.1:8080. so that students can try the code
On receipt of a connection request from the client on the listening socket, the out using just their machine (with
their machine acting as both client
server decides whether it will accept this request or not. If it accepts, it creates
and server).
a new TCP socket (connection socket in Figure 9.4.1.6) which it then binds to
127.0.0.1:8080. Figure 9.4.1.7 shows a three-way handshake which takes place
Key term
to establish this TCP connection between server and client.
TCP/IP socket:
The established TCP connection between HTTP client and HTTP server
A socket is one endpoint of a
(connection socket to connection socket) is uniquely identified by its two endpoints: two-way communication link
<127.0.0.1:49717, 127.0.0.1:8080>. Having established this TCP connection, between two programs running
the HTTP server now returns to listening on its listening socket for connection on the network, e.g. a Web
browser and a Web server.
requests.
Figure 9.4.1.6 shows Python 3.4 code for both the HTTP client and the HTTP
server. The HTTP server runs (httpd.serve_forever()) until it is shutdown.
200 OK
Client <html><head><title>Title goes here.</title></head>
<body><p>This is a test.</p>
on </body></html>
localhost
200 OK Connection
IP: 127.0.0.1 ......
socket
Connection Port Port GET
socket 49717 byte pipe 8080
Server
Listening on localhost
GET 200 OK TCP 3-way socket IP: 127.0.0.1
......
handshake
import http.client
http_server = "127.0.0.1:8080"
#create a connection to server listening on port 8080
connection = http.client.HTTPConnection('localhost:8080') import http.server
#request HTTP GET to server class MyHandler(http.server.BaseHTTPRequestHandler):
connection.request("GET", http_server) def do_GET(s):
#get response from server #Respond to a GET request
response = connection.getresponse() s.send_response(200)
#print response from server and data s.send_header("Content-type", "text/html")
print(response.status, response.reason) s.end_headers()
data_received = response.read(4096) s.wfile.write(bytes("<html><head><title>A test</title></head>",
print(data_received.decode("utf-8")) "utf-8"))
connection.close() s.wfile.write(bytes("<body><p>This is a test.</p>", "utf-8"))
s.wfile.write(bytes("</body></html>", "utf-8"))
Key fact TCP connection to the HTTP client. The client-side Python 3.4 code print
Identifying a TCP connection: statements output the response to the client's console.
Every TCP connection can be uniquely Figure 9.4.1.8 shows a web browser using URL localhost:8080 rendering the
identified by its two endpoints. HTML received from localhost:8080 in the browser's window as "This is a
For example,
test". To do this, the web browser has connected to the HTTP server, sent a
<174.89.0.54:49717, 210.5.0.67:8080>
GET request using the established TCP connection, received back 200 OK and
Key fact the HTML with content "This is a test".
Binding port and socket:
A socket is bound to a port
number so that the transport
layer can identify the application
that data is destined for.
Key term
Well-known ports: Figure 9.4.1.8 Shows the same HTML rendered in a browser window
Well-known ports use the port
number range 0-1023 and are
Questions
associated with service names
assigned by the Internet Assigned 5 What is a TCP/IP socket?
Numbers Authority (IANA) such
6 Why is a socket bound to a port number?
as http or www.
7 How is each TCP connection uniquely identified?
Information
8 Describe the role of sockets in the TCP/IP stack when a web browser
System or Well-known ports:
See -
on a host with IP address 195.61 3.4.7 connects with a web server
https://tools.ietf.org/html/rfc6335#page-11 listening on port 80 on a machine with IP address 210.56.78.3 to
download a web page.
Key term
User ports: Well-known ports and client ports
User ports use the port number Port numbers are 16-bit numbers which are also known by their associated
range 1024 - 49151. They are assigned
service names such as "telnet" for port number 23 and "http" (as well as
on request by the Internet Assigned
Numbers Authority (IANA), e.g.
"www") for port number 80. Hosts running services, hosts accessing services
ciscocsdb 43441 is the service on other hosts, and intermediate devices such as firewalls and NATs all need
name and assigned TCP port number to agree on which service corresponds to a particular destination port. Many
of Cisco NetMgmt DB Ports. services have a default port which servers usually listen on. These ports are
For an application form to register a
recorded by the Internet Assigned Numbers Authority (IANA) through the
port number and service, see:
https://www.iana.org/form/ports-services service name and port number registry.
Port numbers are subdivided into three ranges of numbers:
Key term
• the System Ports or Well-known Ports use the range 0-1023
Client ports:
Client ports use the port number • the User Ports or Registered Ports use the range 1024-49151
range 49152-65535 and are assigned
• the Dynamic Ports or Private or Ephemeral Ports use the range
temporarily by the TCP layer so that
it can identify the client application 49152-65535
that data is destined for. The first two ranges are assigned by IANA. The third range is used by clients. A
Port numbers in this range are not
port number from the dynamic ports range (or private port or ephemeral port
controlled or assigned by IANA.
range) is allocated temporarily to a connection socket requested by a client application. Hence, these are also called
client ports. Client port numbers are never allocated permanently.
Table 9.4.1.1 shows some examples of well-known port numbers and their corresponding service names.
Questions
9 Explain what the well-known ports and client ports are used for and the differences between them.
10 The following information was captured by packet capture software monitoring the network adapter of a
host when a Web browser sent an HTTP message to a Web server:
50268 192.168.2.22 64.29.1.45.9 80 HTTP GET /books.html
For this captured transmission state
(a) the source IP address (b) the port no associated with the sending application's connection socket
(c) the destination IP address (d) the port no associated with the connection socket in the destination
(e) the application-layer message.
11 In the same capture session, the following information was captured immediately after books.html was
transferred to the client:
50272 192.168.2.22 64.29.1.45.9 80 HTTP GET /img/AQAUnit2.jpg
50268 192.168.2.22 64.29.1.45.9 80 HTTP GET /img/AQAUnit2.jpg
(a) What is the meaning of each line of this capture?
(b) The first number in the second line is the same as the first number in Q10. What explanation can you
give for the two numbers being the same?
with its own unique assigned MAC address. If it matches then the network
adapter passes the packet to the link layer software of the TCP/IP stack.
Questions
12 What is the role of Media Access Control (MAC) addresses?
13 The following information was captured by packet capture software monitoring the network adapter of a
host when a Web browser sent an HTTP message to a Web server:
50268 192.168.2.22 64.29.1.45.9 80 74:d4:35:94:ad:53 70:73:cb:b2:f7:d0 HTTP GET /books.html
■ Explain the role of a web Figure 9.4.2.1 FTP transfer of file Test.txt from Computer B to Computer A
server in serving up web pages
in text form The client may need to navigate the directory structure of the server, create new
■ Understand the role of a directories, rename files and directories, delete files and directories. These are
web browser in retrieving sent to the server as command requests.
web pages and web page
resources and rendering these
accordingly
Figure 9.4.2.2 FTP client using FileZilla FTP client software connected to an FTP server
Figure 9.4.2.4 FTP client using FileZilla FTP client software connected to a Cerberus FTP server
Questions
1 A file Test.txt stored on a computer on one network is to be copied to another machine on a different
network which is reachable from the first computer.
(a) What type of software running on the first computer could enable this to be done?
(b) What type of software running on the second computer could enable this to be done?
(c) Why might it be necessary to send commands as well as file data across the connection between the
two computers? Give an example of one command.
Response
Index.html
Message
Other data, such as images or audio files, may also be transmitted. TCP establishes a connection between the client
computer and the server computer so that HTTP has a pathway for its request and response messages.
The simplest request message is
GET / <Return key pressed>
<Return key pressed>
This gets the default web page, index.html, for the given site. HTTP finishes with the connection after the response
message is sent; the TCP connection is broken unless specifically requested to stay connected. A web page returned
by an HTTP GET request is a text file containing content to be displayed together with instructions on how to
style and structure this content when displayed.
Here is what a web browser does:
1. It accepts a URL from a user, e.g. www.educational-computing.co.uk/books.html.
2. It extracts the FQDN (Fully Qualified Domain Name - host name + domain name - e.g. www.educational-
computing.co.uk) and uses a DNS server to translate it into an IP address. DNS is another application layer
protocol.
3. It sends a GET request for the web resource specified in the URL; the request is sent to a web server at this
IP address – port 80 unless another port number is specified.
4. It receives the file returned by the web server.
5. It renders this file’s contents in a web browser window; that means it uses the style and structure
instructions to display the content appropriately.
6. If this file contains other URLs, e.g. a reference to a graphic, then the browser should issue a GET to obtain
this resource from the web server, e.g. GET /images/flower.jpg, and, when received, display it according to
the instructions on style and structure.
To obtain a web page other than the default web page, the web browser sends an HTTP GET request message with
the structure
GET <path to resource> <Return key pressed>
<Return key pressed>
For example,
GET /books/books.html <Return key pressed>
<Return key pressed>
Questions
2 Explain how a web browser and a web server interact via the HTTP protocol when the web browser is used
with URL www.educational-computing.co.uk/books.html.
TCP TCP
IP IP
Link Link
SSL provides a simple Application Programmer Interface (API) with sockets similar to TCP's API. When an
application wishes to use SSL, the application includes SSL classes/libraries.
SSL provides encrypted communication but it also embodies support for data integrity, server authentication, and
client authentication. SSL secures TCP. As such it can be used by any application that runs over TCP, e.g. SMTP
and POP3.
Questions
3 Give two reasons why Web browsers have been redesigned to use HTTPS rather than HTTP when
interacting with Web servers.
4 In what way does the use of the TCP/IP stack differ when HTTPS is used instead of HTTP?
SMTP (Simple Mail Transfer Protocol) and POP3 (Post Office Protocol (v3))
Simple Mail Transfer Protocol (SMTP) is used by e-mail clients to send e-mail. It is a relatively simple text-based
protocol. One or more recipients of a message are specified then SMTP is used by the email client to transfer the
message text to a mail server listening on port 25. The mail server takes care of delivering the mail to the ultimate
destination using SMTP. The user who retrieves the message from the destination mail server uses the application-
layer protocol POP3 to retrieve the stored mail. POP3 uses well known port 110. POP3 is Post Office Protocol
version 3. E-mail is stored in a mailbox and a user does not need to be connected for mail to be sent to them. The
server holds incoming mail until the user connects and requests the mail.
There are other email protocols that may be used instead of SMTP and POP3.
The POP3 protocol defines commands that can be used to retrieve a mail message, e.g. the command RETR
no, where no is the position number of the message in the mailbox. The command LIST returns a list of these
numbers. DELE no marks an email for deletion.
For creating and sending email the SMTP protocol supports commands such as
• MAIL FROM: - defines the e-mail address of the sender of the message.
• RCPT TO: - defines the e-mail address of a recipient of the message. Repeating this command once for
each recipient means you can send one piece of mail to many users without having to repeat the entire
process over and over again.
• DATA marks the start of the data portion of the message, essentially everything that you would
consider "content", this includes the "To:", "From:", "CC:" etc. as these are not commands but simple
informational components making up a header which the e-mail client picks out of the content and
displays in a far nicer format. Just as a reminder - anything which is in the content can be faked as it is
content and so consequently cannot be validated.
Questions
5 A TCP connection to port 110 of a POP3 mail server pop3.apm-internet.net was established by an
email client to a user's mailbox stored on the mail server. Give two examples of POP3 commands that
might be sent by the email client over this TCP connection when the user is browsing their emails.
The Windows computer's IP address is 192.168.2.22 and the Apple Macbook Pro computer's IP address is
192.168.2.21.
Figure 9.4.2.7 Using an SSH client on a Windows computer to connect to an SSH server running on a remote
computer so that the remote computer, an Apple Macbook Pro, can be managed
Background
Plink (PuTTY Link) is a command-line connection tool written for Microsoft's Windows operating system and similar to UNIX's SSH
application. Openssh is a fork(version) of SSH which has been released under an open source licence. It is now supported in Microsoft's
Windows PowerShell.
Questions
6 Communication of confidential information such as log in details over an insecure network is a risk. Why
might an SSH client application be used to connect to an SSH server on a remote computer that one
wishes to log in to?
Key fact unique 32-bit number known as the host interface's Internet Protocol address
or IP address or Internet address. The IP version 6 (IPv6) standard specifies
IPv4:
The IPv4 standard specifies that that each host interface is assigned a unique 128-bit number. Users, application
each host/router interface is programs and higher layers of the protocol software stack use these logical
assigned a unique 32-bit number addresses, i.e. IP addresses, to communicate through the Internet with each
known as the host/router
other without regard to whether the sending and receiving hosts are on the
interface's Internet Protocol
address or IP address or Internet
same physical network or a different one and without needing to know physical
address. addresses.
Figure 9.4.3.2 shows IPv4 addresses assigned to host and router interfaces.
Local Area
Key fact Network A
IPv6: 217.1.1.129
The IPv6 standard specifies
that each host/router interface
217.1.1.68
is assigned a unique 128-bit Host
number known as the host/router
217.1.1.34
interface's IP address.
217.1.1.25
217.1.1.1
Host 217.1.2.1 Router 217.1.3.1
Host
217.1.2.26 217.1.3.24
217.1.2.130 217.1.3.131
Questions
1 Convert the following IPv4 32-bit addresses from binary to their equivalent dotted decimal form
(a) 11001100 00111111 01010101 00111111 (b) 11011001 10000111 00011001 00010011
Subnet
In Figure 9.4.3.2 one router with three interfaces is used to interconnect twelve
hosts. Key concept
The four hosts in local area network A and the router interface to which Subnet:
they are connected all have an IP address of the form 217.1.1.x with x A network of directly connected
interfaces.
chosen from the range 1 to 254. This means IP addresses in this range are
guaranteed to have the same most significant 24 bits in their IP address, i.e.
Subnet address:
110110010000000100000001 (see last row of Table 9.4.3.1). IP addressing assigns an address
In IP terms, local area network A connecting four host interfaces and one router to a subnet in a format a.b.c.d/x
which enables the network to be
interface forms a subnet.
identified, e.g. 217.1.1.0/24
IP addressing assigns an address to this subnet in the form a.b.c.d/x which where the value 24 indicates
is expressed as 217.1.1.0/24, where the /24 notation indicates that the most that the most significant 24 bits
identify the network.
significant 24 bits of the IPv4 32-bit IP address define the subnet address.
Additional hosts added to local area network A will be required to have
interfaces with an address of the form 217.1.1.x. This means that network A can
use 256 different IP addresses, i.e. 217.1.1.0 to 217.1.1.255.
However, 217.1.1.0 and 217.1.1.255 are both reserved IP addresses not to be used
for host/router interface identification, this leaves 254 IP address for identifying
host/router interfaces.
Figure 9.4.3.2 contains two other subnets:
• 217.1.2.0/24
• 217.1.3.0/24
Questions
2 How many bits identify the network in each of the following subnet addresses:
(a) 129.12.0.0/16 (b) 192.173.2.0/23?
The suffix identifies a particular host interface connected to the subnet. The
Key concept
suffix is more commonly known as the Host ID. In Figure 9.4.3.2, the Host
Division of an IP address:
Conceptually, each 32-bit IP ID is given by the least significant 8 bits of the IP address, e.g. 3 in 217.1.2.3.
address (IPv4) (and each 128-bit This division is used as the basis of traffic routing between the physical
IP address (IPv6)) is divided
networks making up the Internet.
into two parts:
1. a prefix or network identifier For example, suppose a host with host interface IP address 217.1.1.25 in local
part (Net ID) area network A wishes to send an IP datagram to a host with host interface IP
2. a suffix or host identifier address 217.1.3.64 in local area network C.
part (Host ID) The LAN A host's IP software is able to determine by examining the
The prefix (most significant bits)
destination's IP address that the destination host is on a different subnet
identifies the physical network
to which the host computer (217.1.3.0/24) with network ID 217.1.3 and therefore cannot be reached
is connected. This physical directly.
network is called a subnet. The LAN A host therefore sends the IP datagram to the router interface
The prefix is more commonly
217.1.1.1.
known as the network ID or
Net ID.
This router examines the network ID of the destination's IP address contained
The suffix identifies a particular in the IP datagram and forwards this datagram to its interface with IP address
host interface connected to the 217.1.3.1 since this interface is connected to subnet 213.1.3.0/24. This
subnet. The suffix is more com- interface places the IP datagram onto the link it is connected to and which host
monly known as the Host ID.
interface 217.1.3.64 is also connected.
The IP datagram is is then read by host interface 217.1.3.64.
The Global Internet
Every interface on every host and router in the global Internet must have an IP address that is globally unique
(except for interfaces behind NATs - see Chapter 9.4.8). IP addresses are assigned in a coordinated manner so that
routing is facilitated. The strategy is called Classless InterDomain Routing (CIDR). It generalises subnet addressing
to the global Internet. As with subnet addressing, the 32-bit IP address in IPv4 is divided into two parts using the
same dotted decimal form a.b.c.d/x, where x indicates the number of bits for the prefix. The prefix constitutes the
network part of the IP address.
Every organisation that wishes to send and receive e-mail, or gain access to the Internet, needs at least one globally
unique IP address.
An organisation is typically assigned more than one unique IP address as a block of contiguous addresses with a
common prefix. The IP addresses of all devices within the organisation will share this common prefix.
For example, the organisation Jisc Services Limited, more commonly known as Janet, is a private, UK government-
funded organisation, which provides computer network and related collaborative services to UK research and
education (Further and Higher). To see how blocks of contiguous IP addresses are assigned for the Janet network
visit https://ipinfo.io/AS786. Table 9.4.3.2 shows a sample of these blocks.
Table 9.4.3.2 Sample of IP address contiguous block allocation within the Janet organisation.
The University of Kent has subdivided its block of IP addresses into 128 subnets. Table 9.4.3.3 shows a sample of
these addresses of these subnets.
Table 9.4.3.4 shows binary equivalent of the subnet addresses shown in Table 9.4.3.3 and the corresponding
Network ID expressed in binary. Note that the most significant 16 bits of each subnet Network ID is the same. This
corresponds to the prefix for the University of Kent given by 129.12.0.0/16.
Table 9.4.3.4 Network IDs for the subnets for University of Kent. The university's Network ID is shown in red
The gateway router's routing table entries indicate that the most significant 23 bits of the IP datagram should be
examined to determine which subnet to forward it to. The 23-bit prefix of 129.12.3.237 in binary is
10000001 00001100 0000001
Network ID Link no
10000001 00001100 00000000 00000000 1
10000001 00001100 00000010 00000000 2
10000001 00001100 00000100 00000000 3
10000001 00001100 00000110 00000000 4
● ●
● ●
10000001 00001100 11111000 00000000 125
10000001 00001100 11111010 00000000 126
10000001 00001100 11111100 00000000 127
10000001 00001100 11111110 00000000 128
Table 9.4.3.5 Network IDs for the subnets for University of Kent expressed
as 32-bit values and the corresponding Link no expressed in decimal.
Questions
3 The following 32-bit value is an entry in the University of Kent's gateway router routing table (see Table
9.4.3.5) 10000001 00001100 00000110 00000000
Which of the following destination IP addresses expressed in dotted decimal notation match this entry
(a) 129.12.7.38 (b) 129.12.6.123 (c) 129.12.5.223 (d) 129.13.6.45?
Questions
4 An IP datagram is addressed to a remote destination whose IP address is 129.12.33.114. Internet routers
forward this datagram to the gateway router at the University of Kent. Explain how this datagram is
forwarded to the host interface with IP address 129.12.33.114 within the University of Kent's network.
You may assume that Figure 9.4.3.3 describes this university's network setup.
The suffix is the Host ID. In this example nine bits are allocated to the suffix.
The prefix is the Network ID or subnet address. In IPv4 it is expressed as a
32-bit value by replacing the suffix or Host ID part with zeroes. For example,
Key concept
the Network ID of the host interface with IP address 129.12.7.38 is in binary
Using a subnet mask:
The subnet mask is used by a 10000001 00001100 00000110 00000000
computer/router to obtain the which in dotted decimal form is 129.12.6.0/23.
Network ID or subnet address
of the subnet to which its Therefore to obtain the Network ID for a given IP address we need to know
network interface is connected. how many bits are assigned to the prefix part. This is known as the subnet
This may be done with an mask. The Network ID is also known as the subnet address.
AND operation applied to the
computer interface's IP address Given a 32-bit IP address such as 129.12.7.38 it is possible to obtain its
and a mask of the same number Network ID expressed as a 32-bit value by a bitwise AND operation applied to
of bits constructed from the the 32-bit mask derived from the subnet mask and the IP address. If the subnet
subnet mask.
mask is 23 then the 32-bit mask for the bitwise AND operation will consist of
twenty three ones for the most significant bits followed by nine zeroes.
This shows that the Windows 7 computer and the default gateway are on the same subnet (192.168.1.0) and
therefore directly connected.
Questions
1 Table 9.4.4.1 shows subnet masks and IP addresses expressed using IPv4 dotted decimal notation
Subnet mask IP address Network ID or subnet address
255.255.255.0 192.168.2.15
255.255.0.0 192.168.253.234
255.255.252.0 192.168.253.234
Table 9.4.4.1 Subnet masks and IP address expressed using IPv4 dotted decimal notation
Complete the Network ID column for the given subnet masks and IP addresses.
2 A computer with a host interface IP address 192.168.1.5 sends an IP datagram to a computer with
IP address 192.168.2.3. In each case the subnet mask expressed in IPv4 dotted decimal notation is
255.255.255.0. The default gateway for the computer 192.168.1.5 is 192.168.1.1. Explain why the IP
datagram is sent to the default gateway for forwarding to 192.168.2.3.
internet, as a shorthand for internetworking. TCP/IP followed later in the decade and the Internet was born along
with IP addressing.
IPv6
In response to the projected problem of running out of IPv4 addresses, the technical community came up with a
specification for IP version 6 (IPv6) in the mid-1990s.
IPv6 allocates 128 bits for IP addresses. This gives a theoretical total number of addresses of 2128 or roughly 3.4 x
1038 addresses. This is more than enough for the foreseeable future. By way of comparison, the number of addresses
required to uniquely label all the grains of sand on planet earth would be 7.5 x 1018. However, adoption of IPv6
is not a straightforward matter because the public Internet is an IPv4 router network designed to work with
32-bit addresses not 128-bit addresses. Whilst new IPv6-capable systems can be made backwards compatible, i.e.
send route and receive IPv4 datagrams, already deployed IPv4-capable systems are not capable of handling IPv6
datagrams.
Questions
1 What is the theoretical total number of IP addresses for
(a) IPv4? (b) IPv6?
Table 9.4.6.1 The three private IP address spaces, their range and total number
Key concept Non-routable IP addresses
Non-routable IP address: Hosts within a given private TCP/IP network can send IP datagrams to each
Private IP addresses are non-
other using addresses assigned to each which are chosen from a private address
routable.
Routers in the public, global
space, e.g. 192.168.0.0/24. However, IP datagrams forwarded from the private
Internet will reject IP datagrams network into the larger public, global Internet cannot use these addresses as
with source and/or destination source or destination address because there will be many other connected
IP addresses which fall within networks which use addresses from the same private address space. Routers
a private address range and will
in the public, global Internet will reject such IP datagrams and will not route
therefore not route them.
them. Private IP addresses are therefore said to be non-routable.
Routable IP addresses:
Routable IP addresses
Public IP addresses are routable
IP addresses because their
The assignment to hosts of IP addresses is coordinated by IANA and the
assignment is coordinated by Regional Internet Registries to ensure that each host's IP address is globally
IANA and the Regional Internet unique. This enables routers of the global, public Internet to use the source
registries to ensure that hosts/ and destination public IP addresses of two communicating hosts to route IP
routers are uniquely identified
datagrams through the Internet from one host to the other. Public IP addresses
globally. Routing tables in
routers contain globally unique are routable IP addresses because the coordination provided by IANA and
address information to enable the Regional Internet registries ensures that routing tables in routers contain
successful and unambiguous globally unique address information to enable successful and unambiguous
routing of IP datagrams. routing of IP datagrams.
Questions
1 Read sections 1 to 4 of RFC 1918 ( https://tools.ietf.org/html/rfc1918) then answer the following
questions
(a) Give three examples where external connectivity of hosts and routers in an organisation's TCP/IP
network might be unnecessary.
(b) Give one reason why it is good practice to use private IP addresses where possible in IPv4 TCP/IP local
area networks that are connected to the Internet.
2 State the three regions of the IPv4 address space that are reserved for private TCP/IP networks.
• In "dynamic allocation", DHCP assigns an IP address to a client for a limited period of time (or until the
client explicitly relinquishes the address)
• In "manual allocation", a client's IP address is assigned by the network administrator, and DHCP is used
simply to convey the assigned address to the client.
A particular network will use one or more of these mechanisms, depending on the policies of the network
administrator.
In addition to host IP address assignment, DCHP also
allows a host to learn additional information, such as
the subnet mask for the subnet it is connected to, the
address of its first hop-router (often called the default
gateway) to which it sends IP datagrams for hosts on
different subnets, and the address of a DNS server.
Figure 9.4.7.1 shows the TCP/IPv4 Properties window
for a Microsoft Windows 7 host. It shows the manually
configured properties for the host interface's IP address,
subnet mask and default gateway. The DNS servers
have not been configured. If the radio buttons "Obtain
an IP address automatically" and "Obtain DNS
server address automatically" were selected then these
property fields would be completed automatically by
preconfigured information obtained from a DHCP
server in a client-server operation.
The four-step DCHP process Figure 9.4.7.1 TCP/IPv4 properties
Step 1 window for a Microsoft Windows 7 host
The first task of a new host (client) that wishes to join
an existing TCP/IP network is to find a DHCP server. It does this by broadcasting a DHCP discover message
over the network. The host at this point does not know the subnet address of the network or the IP address of any
DHCP server. So it uses a special broadcast destination address of 255.255.255.255 and a "this host" source IP
address of 0.0.0.0. This broadcast will reach all nodes attached to the subnet.
Step 2
A DHCP server receiving a DHCP discover message responds to the client with a DHCP offer message that is
broadcast to all nodes on the subnet, again using the IP broadcast address of 255.255.255.255. The DHCP offer
message contains the proposed IP address for the client, the network mask, an IP address lease time (the amount
of time for which the IP address will be valid), and a transaction ID extracted from the DHCP discover message
which links this message with the new client.
Step 3
The new client responds to the offer with a DHCP request message which echoes back the configuration
parameters.
Step 4
The server responds to the DHCP request message with a DHCP ACK message, confirming the requested
parameters.
Questions
1 What is the primary purpose of the DHCP system?
2 Give two examples where using DHCP to configure new client hosts is preferable to manual configuration
by other means.
192.168.1.2
NAT-enabled
① router ②
④
192.168.1.3 source =92.23.29.162 : 5431
192.168.1.1 destin. =129.11.26.33 : 80
The NAT-enabled router deals with this issue by presenting itself to the outside
world not as a router but as a single device with a single IP address. In Figure
9.4.8.1 all traffic leaving this NAT-enabled router for the global Internet has a
source IP address of 92.23.29.162. All traffic entering this router must have a destination address of 92.23.29.162.
The role performed by this NAT-enabled router is to hide the details of the home LAN from the outside world.
Figure 9.4.8.2 NAT-enabled router showing four LAN sockets labelled Ethernet 1, 2, 3 and 4, a WAN
socket, and a broadband socket (an alternative way of connecting to the Internet)
The use of port numbers
To understand how port numbers are used to enable LAN hosts to use the WAN, consider the example shown in
Figure 9.4.8.1. In this figure, LAN host with IP address 192.168.1.2 requests a Web page from a Web server with
IP address 129.11.26.33 listening on port 80. The LAN host assigns an arbitrarily chosen source port number 4236
to the request and sends the IP datagram into the LAN as a link-layer packet addressed to the NAT-enabled router -
see Figure 9.4.8.1 1. Note that the destination IP address is 129.11.26.33 and the destination port number is 80.
The NAT-enabled router receives the IP datagram and makes two changes to this datagram:
• It generates a new source port number 5431 for the datagram and replaces the original source port
number 4236 with this new source port number
• It replaces the source IP address 192.168.1.2 with its WAN-side IP address 92.23.29.162.
The NAT-enabled router adds the entry 92.23.29.162 : 5431 192.168.1.2 : 4236 to its NAT translation table as
shown in Figure 9.4.8.1. and sends the IP datagram 2 into the public Internet.
When generating a replacement source port number, the NAT router chooses a source port number which is not
currently present in the NAT translation table. Port numbers are 16-bit numbers so the NAT router with a single
WAN-side IP address is able to support 216 simultaneous connections.
The Web server responds with an IP datagram whose destination address is the WAN-side IP address of the router,
and whose destination port number is 5431 - see Figure 9.4.8.1 3. Note that the Web server is not aware that the
Web page request has come from the LAN host with interface IP address 192.168.1.2.
On arrival at the NAT-enabled router, the received datagram's destination IP address 92.23.29.162 together with
the destination port number 5431 is matched to the corresponding entry in the NAT translation table to obtain
the LAN host IP address and port number. The router then constructs an IP datagram using these as destination IP
address and destination port number, respectively, before dispatching this datagram into the home LAN where it
is read by host 192.168.1.2 and passed to the application (identified by port number 4236) that initiated the Web
page request - see Figure 9.4.8.1 4.
Questions
1 Why does an IPv4 TCP/IP LAN need a NAT-enabled router if hosts with private IP addresses are to
connect to the Internet?
2 Explain how port numbers are used to enable LAN hosts using private IPv4 addresses to exchange IP
datagrams with the global Internet.
3 Port numbers are associated in the TCP/IP specifications with layer 4, the Application layer, of the TCP/IP
protocol stack (Chapter 9.4.1). How does the use of port numbers by the NAT protocol differ from their
use by the Application layer protocol?
4 The end-to-end principle is one of the underlying system principles of the Internet (Chapter 9.4.1).
(a) Why is the NAT protocol a violation of this principle?
(b) Why might using IPv6 addressing obviate the need for the NAT protocol?
192.168.1.2 192.168.1.3
Symbol for a router
Home LAN 2
192.168.1.0/24
192.168.1.2
192.168.1.2 NAT-enabled
router
129.11.26.33
192.168.1.1
192.168.1.3 192.168.1.3
192.168.1.1
146.23.18.5
Web server
192.168.1.4 92.23.29.162
192.168.1.4
Port mapping table
Internet
Minecraft WAN side LAN side Another LAN
server
192.168.1.5 92.23.29.162 : 80 192.168.1.4 : 80
Another LAN Another LAN
92.23.29.162 : 25565 192.168.1.5 : 25565
Home LAN 3
Home LAN 1
Figure 9.4.9.1 Port forwarding to enable access from the Internet to a Web server and a Minecraft server
assigned unregistered private addresses in home LAN 1
In a similar manner, clients in other LANs may connect to the Minecraft server
in Home LAN 1 via port forwarding set up in the NAT-enabled router with
public IP address 92.23.29.162.
Figure 9.4.9.2 shows the setup window for port forwarding for a NAT-enabled
router.
Single licence - Abingdon School 457
9 Fundamentals of communication and networking
Questions
1 In what circumstance would port forwarding be used?
2 A Web server connected to a LAN uses the private IP address 172.31.78.4. The Web server listens for
HTTP requests on port 80. A NAT-enabled router with two interfaces, 172.31.78.1 and 146.31.18.97 is
also connected to this LAN. Its 146.31.18.97 interface is connected to the Internet. Explain how the port
mapping table in this router would be set up to allow port forwarding of HTTP requests arriving from the
Internet.
3 The Web server in Question 2 is switched off for maintenance for two days. When it is brought back
online, port forwarding no longer works. State one likely reason why port forwarding no longer works and
suggest a solution.
• R – Replace
REQUEST
• U – Update
24 hour
• D – Delete Customer 3 Customer 2 Customer 1
• GET → SELECT
• DELETE → DELETE Figure 9.4.10.1 The client server model for Pizza House service
• PUT → UPDATE
There is a protocol that client and server use in this scenario that is common to
■■Compare JSON (Javascript many similar scenarios in which a customer (client) orders something over the
Object Notation) with XML
telephone or online from a supplier (server):
1. there is a menu to choose from that defines the identifier for the
resource, e.g. Margherita;
2. a request which begins “I would like to order…” and ends with “for
delivery to 42 Acacia Avenue, Dingley Dell, NeverNeverLand”;
3. the server issues a response to the request addressing it to the client.
The protocol is the “glue” or connector that enables the client-server interaction
to perform as expected.
The client-server model is the most commonly employed of the architectural styles for when one application
interacts with another. A server is one application, a client is another. A server application listens for requests, from
client applications, for services which it offers and the client applications consume these services.
A client application, needing a service to be performed, sends a request to the server application via a standard
interface protocol e.g. HTTP.
IP address: 192.168.2.25
USER INTERFACE SERVER
application
100
s a n d p
Celsiu
50
50 in Fahrenheit is 122
Port
s
e r a t u re in degree
mp
Type a te Enter
nd press
53152
C e ls iu s a
the client, e.g. 53152, that is connected to the requesting client application. The response of the server application
is to return the temperature in Fahrenheit to the requesting client application, using as return address the client
application’s IP address and port number. The client application connects to the server application for this service.
In this example, the connection uses the Telnet (a less secure alternative to SSH) protocol.
Figure 9.10.4.2 illustrates a very important principle: Separation of Concerns.
Background
The client application is developed quite separately from the server application.
The client application API:
An API is an application
focuses on the user Questions
programming interface. An
interface design and the application programming interface
1 Explain the client-server model.
server application focuses separates the data from the
on the design for the operations that may be performed
processing and formatting of data. Both designs can be changed without regard to on the data. Applications that use
a particular API do not know how
each other whilst preserving a uniform interface between the two, e.g. the Telnet
the data is stored only how to
protocol. access it via API calls.
REST
Web server
The idea of REST is that HTTP
Browser Server application
application data can be queried HTTP Server
GET method
and changed using verbs and POST Server Facebook’s
nouns, represented by HTTP DELETE issues request to
uest graph API
Req
PUT Facebook’s graph
methods and URLs, respectively. A REST API issues
User makes Response
REST request will typically return request to website
data in a machine-readable form, Server sends response
User sees
such as JSON or XML. in format
rendered response
Response returned by API call
Suppose that you wanted to
HTTP Graph
order a book from Amazon, the Client application method Dataset
online bookseller. Do you have
to download a special app to Figure 9.4.10.3 Interaction via an HTTP server between a browser and a
do this or can you use software server application accessing Facebook’s social network graph
that you have on your computer
already? Of course, the answer is that you can use browser software you have on your computer already. Typing the
following URL (uniform resource locator) into the address bar of your browser returns a web page that contains
information about a resource that Amazon uniquely identifies with the ID 0241003008, i.e. the book Very Hungry
Caterpillar Board Book: http://www.amazon.co.uk/Very-Hungry-Caterpillar-Board-Book/dp/0241003008/
Amazon’s web site is a web service based on a design pattern called REST which stands for REpresentational State
Transfer. Central to the concept of REST is the notion of resources. Resources are represented by URIs or Uniform
Resource Identifiers, the Amazon example is one such URI. There are two types of URI: URLs and URNs. We
will focus on URLs (Uniform Resource Locators). Another key aspect of the design of REST web services is that
resources should be linked together and representations of these resources should enable a user to move from one
resource to another by following these links.
Figure 9.4.10.3 show an interaction via an HTTP server between a client application, a browser, and a server
application, that provides a Facebook RESTful web service that accesses Facebook’s social network graph dataset
of Facebook users. Facebook’s graph API is an application programming interface (API) that is run on Facebook
servers. If the following URL is typed into the browser’s address bar for the scenario in Figure 9.4.10.3 then what is
returned is a representation that the browser renders as shown in Figure 9.4.10.4:
https://graph.facebook.com/4?oauth_token= CAACE...... See Task 1 below for how to get an access token
(CAACE....) to use with oauth_token.
In the Facebook social network graph, Mark Zuckerberg’s
user node has the identifier “4”, not surprisingly his
identifier was one of the very first to be allocated. The
representation returned is expressed in JSON (JavaScript
Object Notation)
By following the link in the representation shown in
Figure 9.4.10.4, a client application, i.e. a browser,
Figure 9.4.10.4 Mark Zuckerberg’s User node can obtain the next resource representation, Mark
in Facebook’s social network graph, node id is 4. Zuckerberg’s Facebook home page, from the server
Mark’s home page is www.facebook.com/Zuck application. Facebook’s social network graph is, in 2015,
the largest social network dataset in the world.
State 3
State 1 url: https://www.face- Figure 9.4.10.5 shows how the client application’s state
url: empty book.com/Zuck
changes as it interacts with the server application which
in turn accesses Facebook’s social network graph dataset.
Thus, the client application changes (Transfers) to a new
State 2 State when it obtains a new resource REpresentation
url: https://graph.facebook.com/4
from the server application, this is why REST stands for
REpresentational State Transfer!
Figure 9.4.10.5 The client application’s changes of
Questions
state, each transition to a new state causes the server
application to transfer a representation to the client 2 Explain with an example what is meant by REST.
application
The REST architecture is characterised by
• Client-server: separation between server application that offers a service, and the client application that
consumes it
• Stateless: each request from a client application must contain all the information required by the server
application to carry out the request. Session state is kept entirely on the client not at the server
• Uniform interface: the method of communication between a client application and a server application
must be uniform, e.g. via HTTP requests and responses
• Code-on-demand: server applications can provide executable code or scripts for client applications to
execute in their context
• Cacheable: the server application must indicate to the client application requests that can be cached.
• Every resource has a unique ID, e.g. https://graph.facebook.com/4
• Multiple representations of resources provided for different needs, e.g. html, JSON, XML, jpeg, png, gif,
csv
• Resources are linked together, e.g. Facebook’s social network graph.
Uniform interface
Communication between the client application and the server application typically uses HTTP as the uniform
interface. HTTP provides four principle methods for Creating, Retrieving, Updating and Deleting a resource. The
methods are sometimes referred to as CRUD operations:
• C – Create HTTP
Action Examples
• R – Retrieve method
• U – Update GET Retrieve a Retrieve Mark Zuckerberg’s social network graph entry –
• D - Delete resource or GET /4 HTTP/1.1
Host: graph.facebook.com
The HTTP methods that a collection
Retrieve AQA CS Unit 2 main sections
map onto these operations of resources
GET /v1/csunit2s/ HTTP/1.1
are shown in Table Host: cs.apispark.net
9.4.10.1. POST Create Creates a new resource –
The REST API created and a new POST /v1/csunit2s/ HTTP/1.1
Host: cs.apispark.net
run at the server may be resource
{ “id”: “7”, “title”: “Fundamentals of
connected to a relational compter organisation and architecture”,
“link”:[“a test”] }
database such as MySQL.
PUT Update or Updates a resource, “computer” misspelt–
In which case the HTTP
replace an PUT /v1/csunit2s/7 HTTP/1.1
verbs will be converted into Host: cs.apispark.net
existing
SQL equivalent database { “id”: “7”, “title”: “Fundamentals of
resource computer organisation and architecture”,
commands as follows : “link”:[“a test”] }
• GET → SELECT DELETE Delete a Deletes a resource, 8 –
• POST → INSERT resource DELETE /v1/csunit2s/8 HTTP/1.1
Host: cs.apispark.net
• DELETE → DELETE
Table 9.4.10.1 The four HTTP methods for CRUD
• PUT → UPDATE
in order to carry out operations on the database (see Chapter
10.4 for SQL commands). var http = require('http');
var work = require('./mydatabase');
var mysql = require('mysql');
The REST API may also be connected to a NOSQL database var db = mysql.createConnection({
such as MongoDB. host: '127.0.0.1',
user: 'myuser',
Multiple representations of resources password: 'mypassword',
database: 'mydatabase'
What is returned to the client application is not the resource });
but a representation of the resource. The REST service provided
by the server application is usually designed to provide at least Table 9.4.10.2 Server-side Javascript to set up
JSON and XML representations. Figure 9.4.10.6 shows the a MySQL database connection to database,
JSON response and Figure 9.4.10.7 shows the equivalent XML mydatabase. This is part of a larger script that
response. The URL in each case is runs under node.js.
https://cs.apispark.net/v1/csunit2/6
The client application has no knowledge of how the resource is actually stored on the server. The resource sits
behind an interface which hides this information. This means that the resource can be restructured without
affecting any client application which simply continues to use the uniform interface of HTTP CRUD operations.
<Csunit2>
<id>6</id>
<link>
<link>https://cs.apispark.net/v1/csunit2s/subsections/6.1</link>
<link>https://cs.apispark.net/v1/csunit2s/subsections/6.2</link>
<link>https://cs.apispark.net/v1/csunit2s/subsections/6.3</link>
</link>
<title>Fundamentals of computer systems</title>
</Csunit2>
So it should be possible to navigate from node to node by following edges, e.g. “likes”. If we start with a user with
Userid 784889091561000 (this is a fictional example), we need our client application, e.g. POSTMAN, to send the
following request to Facebook’s server application to access the open graph(og) part of the social network graph:
https://graph.facebook.com/784889091561000/og.likes?oauth_token= CAACE......
Figure 9.4.10.8 shows a section of the response from the application server. See Task 1 below for how
to get an access token
Questions (CAACE....) to use with
{ oauth_token.
"data": [
3 In the context of REST explain, {
with examples, what is meant by "id": "670741596309143",
"from": {
(a) uniform interface "id": "784889091561000",
(b) multiple representations "name": "Fred Bloggs"
},
(c) every resource has an id
"start_time": "2014-06-17T15:48:54+0000",
(d) resources are linked. "publish_time": "2014-06-17T15:48:54+0000",
"application": {
4 What is meant by CRUD? "name": "Og_likes",
"namespace": "likes",
"id": "193042140809145"
5 A REST API is connected to a },
server-side relational database. "data": {
"object": {
What SQL commands will the "id": "222386191275871",
HTTP verbs, GET, POST, "url": "http://www.needateacher.co.uk/",
DELETE and UPDATE map "type": "website",
"title": "http://www.needateacher.co.uk/"
to? }
},
3 Continuing on from Task 1. Select version v2.2 (Facebook reduced information returned in later
versions). Enter /4 into the address bar next to GET. Click on submit. Copy the link to Mark
Zuckerberg’s home page and visit it in a browser.
4 Google supports several REST web services, one is finding directions. Try the following URL:
https://maps.googleapis.com/maps/api/directions/json?origin=Aylesbury Grammar
School&destination=Tesco Tring Road
5 Experiment with changing the origin and destination parameter values for the URL in Task 4.
Tasks
6 Another Google REST service is staticmap. Try the following URL in a browser:
https://maps.googleapis.com/maps/api/staticmap?center=Aylesbury Bucks&zoom=13&size=600x300&
maptype=roadmap&format=png Information
Google Maps Javascript
7 Experiment with the center parameter value for the URL in Task 6 and the
reference:
maptype (roadmap, satellite, terrain, hybrid). https://developers.google.
com/maps/documentation/
8 Install POSTMAN REST client from Google Chrome store. Once installed javascript/reference#Map
run it from outside Chrome web browser. Select the normal page and the
GET method.
Enter http://www.youtypeitwepostit.com/api/ onto the GET line. Click Send. You should get a response
which is a collection of posts to a message board.
9 Use url http://www.youtypeitwepostit.com/api/. Select POST and raw then enter the following below
the raw bar:
{
"template": {
"data": [
{
"prompt":
"Text of message",
"name":
"text",
"value":
"Arfur"
}
]
}
}
Click Send.
10 Using POSTMAN, check that the new message has been posted to the message board using GET.
Click on the hyperlink URL, e.g. http://www.youtypeitwepostit.com/api/25134141254238784, so
that it appears in the GET line. (Id 25134141254238784 must exist for this to work). Change GET
to DELETE and Send. Check that this message has been deleted by doing a GET on http://www.
youtypeitwepostit.com/api/.
12 Obtain an API key from flickr (Yahoo). Enter the following URL into a browser URL address bar to use
the REST web service of Flickr inserting your API key where indicated:
https://api.flickr.com/services/rest/?method=flickr.photos.search&api_key=this is where you put your
key&format=json&nojsoncallback=1&text=cats&extras=url_o
Find a jpg image in the JSON response and copy its URL into a browser address bar.
Table 9.4.10.3 shows Javascript code embedded in an HTML web page, GoogleMap.html. This HTML file uses a
Google map api Javascript library at http://maps.googleapis.com/maps/api/js. The script calls a REST API provided
by Google to pull down a road map. After loading this web page the browser executes the Javascript function
initialize(). This function loads the map data into the web page using a call to an API constructor
google.maps.Map. This creates a new map inside the given HTML container, the Div element, identified by
the identifier “googleMap”. The script section is placed at the end of the web page to ensure that it is executed
after the web page is loaded. The function initialize also places a marker on the map, using the latitude and
longitude coordinates.
Tasks
14 Experiment with the values assigned in mapOptions:
(a) Change zoom from 1 to 20. Note that the map object can be dragged to move location.
(b) Change ROADMAP to SATELLITE.
(c) Change the latitude and longitude to match your location.
(d) Change the marker text that appears when the mouse is over the marker on the map.
<!DOCTYPE html>
<html>
<head>
<script
src="http://maps.googleapis.com/maps/api/js">
</script>
</head>
<body>
<div id="googleMap" style="width:600px;height:400px;"></div>
<script>
function initialize()
{
var LatitudeLongitude = new google.maps.LatLng(51.8168, -0.8124);
var mapOptions = {
zoom: 15,
center: LatitudeLongitude,
mapTypeId:google.maps.MapTypeId.ROADMAP
}
var map = new google.maps.Map(document.getElementById('googleMap'),
mapOptions);
var marker = new google.maps.Marker({
position: LatitudeLongitude,
map: map,
title: 'Marker test!'
});
}
initialize();
</script>
</body>
</html>
Table 9.4.10.3 Web page GoogleMap.html that contains a Javascript script to load a Google roadmap centred on
latitude 51.8168 and longitude -0.8124 at zoom level 15.
Table 9.4.10.4 shows a Javascript client side script that accesses a REST web service at To access this url login
https://cs.apispark.net/v1/csunit2s/subsections/subsubsections/5.1.1 credentials are required.
The response is then rendered by adding it to the paragraphs according to their class ids, greeting-id, greeting-title,
greeting-link. The script uses jQuery, $ is short for jQuery, e.g. $.ajax is equivalent to jQuery.ajax.
Figure 9.4.10.9 shows the response rendered in a browser window.
<!DOCTYPE html>
<html>
<head>
<title>Hello jQuery</title>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js">
</script>
</head> Save this script with extension .html. Launch
<body>
<div> in Google Chrome. If an access token is not
<p class="greeting-id">The ID is </p> supplied then the Javascript console window
<p class="greeting-title">The title is </p>
<p class="greeting-link">The link is </p> will show the error message:
</div> Failed to load resource: the server responded
</body>
with a status of 401 (Unauthorized)
<script>
$.ajax({
headers: {"Authorization": "Basic replace with access token here"},
dataType: "json",
url: "https://cs.apispark.net/v1/csunit2s/subsections/subsubsections/5.1.1"
}).then(function(data) {
$('.greeting-id').append(data.id);
You will need to register with
$('.greeting-title').append(data.title);
$('.greeting-link').append(data.link); cs.apispark.net and set up your
}); own REST service. The urls used
</script>
</html> here are the author’s own setup.
Table 9.4.10.4 HTML web page containing a jQuery Javascript script to access a REST web service
Questions
6 Give three advantages that JSON has over XML. 7 Give two advantages that XML has over JSON.
Websocket protocol
The use of HTTP is well suited to the applications described so far. However, in applications which require the
server to send data to the client the very moment when it knows that new data is available, HTTP is inadequate
because it is client-driven.
The HTML5 WebSocket specification defines an API that establishes a full-duplex communication channel
operating through a single socket over the Web between a web browser and a server. HTML5 Websocket has been
designed for real-time, event-driven web applications. The Websocket protocol creates a persistent connection
between the client and the server in which both parties can start sending data at any time.
As soon as a connection to the server is established, the client application can start sending data to the server using
the send('a message') method on the connection object.
Equally the server application can send messages to the client application at any time.
The http:// is replaced by ws:// in a url, e.g. ws://echo.websocket.org,
https:// is replaced by wss://, e.g. wss://echo.websocket.org.
The Websocket protocol is used where multiple users need to interact in real time such as in multiplayer games.
Tasks
15 Visit http://demo.kaazing.com/livefeed/ to see a live demo that uses the WebSocket protocol.
Questions
8 Explain why the Websocket protocol has replaced the HTTP protocol for a certain type of client-server
interaction.
Thin-client
Key Concept In a thin-client network, the client is a diskless workstation with a little
Thin-client: RAM, a low specification CPU, a network interface adapter with a boot
A thin-client is a diskless ROM, interfaces for keyboard, mouse, VDU, and possibly interfaces for other
workstation with a little RAM,
peripherals. The boot ROM on the network card is used to load a stripped-
a low specification CPU, a
network interface adapter with down operating system from a central server as well as to obtain an IP address
a boot ROM, interfaces for and other networking configuration information, e.g. a subnet mask and a
keyboard, mouse, VDU, and gateway IP address.
possibly interfaces for other
peripherals.
In a thin-client network, patches/updates are applied to the software stored on CPU, less RAM, no magnetic
hard drive/SSD for thin-client
the terminal server. This is much less time consuming because the maintenance
compared with thick-client
occurs on just one machine. means lower cost to produce
Hardware maintenance Power consumption:
• Lower specifications and less
Fanless thin-clients have no mechanical moving parts which is the principle
hardware means thin-client
cause of failure in thick-clients with magnetic hard disk drives and fans to cool workstations can consume
the CPU. The life expectancy of thin-clients is about two years longer than about one-seventh the power
thick clients with magnetic storage. that PC thick-client does
Reduced licensing costs:
Security • Only as many licences as will be
One of the primary benefits of using thin-client hardware is security. The used, need to be purchased in a
absence of local storage means that users are unable to install unauthorised thin-client system but as many
software onto the system. There is also no local storage to infect with viruses or licences as thick-clients are
needed in a thick-client system
spyware. Virus protection is applied at the terminal server where it is centrally
Apps best run in a thick-client:
managed. • Applications that result in
Cost considerable latency such as
graphic intensive and video
Thin client hardware has a very low price tag compared with thick-client
editing ones. These require a
workstations. The cost of a thin-client station in a Raspberry Pi LTSP thin- lot of processing power, large
client network is £20 to £30. The Jammin series are about £200 but have 2-4 amount of RAM, large file sizes
GiB memory and a fast Gigabit network adapter. and higher network bandwidth
Power consumption
Thin client devices tend to consume much less power than thick-client workstations. Power consumption varies
among makes and models, but some estimates indicate that thin-client devices only consume about one-seventh the
power of PC thick-clients.
Reduced licensing costs
In a thick-client environment application software is installed on each client. If the software requires a licence
then the total number of licences that need to be purchased equals the total number of thick-clients, assuming
that users can work at any thick-client. In a thin-client environment only as many licences as will be used, need
to be purchased as the licensed application software runs in the terminal server whilst being accessible from any
thin-client.
Applications that are best run in a thick-client workstation
Thin-client computing works well for typical applications such as email, web browsing, office applications such
as word processing but it doesn't work quite so well for high-level graphics processing and video editing because
there will be considerable latency because of the higher processing power, larger RAM, larger file sizes and network
bandwidth required for these applications. These applications are best suited to running in a thick-client.
Questions
2 Facing tighter budgets and the need to be more responsive to public requirements, local and city councils
are finding it increasingly difficult to meet targets with their existing PC-centric computer infrastructures of
4500+ machines.
(a) Give four reasons why a solution to this problem could be to replace the thick-client systems (PC-
centric) with a thin-client system.
(b) Suggest one reason why a thin-client system might not be the best option for the following
(i) the architects department (ii) computer science students in local council schools
3 In Business Studies lessons at one school, students need to use stand-alone machines. Before each lesson the
teacher has found it is necessary to reset all twenty machines to their base configuration, spending 15 or 20
minutes each time on this task. Explain why replacing the stand-alone machines by a thin-client network
could improve this situation.
4 Explain why clients in a thin-client network have been called dumb terminals.
Data Constraints
Data constraints are also usually expressed in English statements, e.g.
‘Students may not study more than three modules per term.’
Formally,
A conceptual model is a representation of the data requirements of an
organisation constructed in a way that is independent of any software that is
used to construct a database.
Entity
An example of an entity is a Student. Students are of interest to an
organisation such as a school or college. The college will need to record the
name of each student currently enrolled, his/her date of birth, home address,
Key concept
and other data about the student.
Entity:
An entity is an object, person,
Formally,
place, concept, activity, event
An entity is an object, person, place, concept, activity, event or thing of interest
or thing of interest to an
to an organisation and about which data are recorded.
organisation and about which
data are recorded.
Questions
1 List three entities for each of the following organisations:
(a) Hospital
(b) Lending library
(c) Athletics club
Attribute
Key concept
The particular items of data such as name, date of birth, home address, etcetera
belonging to an entity such as Student are called attributes. Attribute:
An attribute is a property or
characteristic of an entity.
Formally,
An attribute is a property or characteristic of an entity.
Questions
2 List two attributes of each of the following entities that belong to a
hospital in-patient system:
(a) Patient
(b) Ward
(c) Nurse
Student
Date of
Enrolment Surname Forename Address Tel. No
Entity birth
Number
Occurrence 1 19/1/80 433991
Briggs Sarah 42 Benn Av.
or
2 Carter David 22/2/80 10 Acacia Av. 484132
Instance 3 Teng Lee 13/4/80 23 Queens Road 472611
4 Khan Imran 29/5/80 3 Stannier St 447334
Entity identifier
Key concept An organisation will need to select or identify a particular occurrence of
an entity from among others. It does this with the entity identifier. This is
Entity identifier: sometimes referred to loosely but incorrectly as the primary key.
The entity identifier is an
attribute or combination of
attributes which uniquely Formally,
identifies an instance or The entity identifier is an attribute or combination of attributes which
occurrence of the entity. uniquely identifies an instance or occurrence of the entity.
Sometimes referred to loosely
but incorrectly as the primary
key. Suppose that each student is assigned a number called the student enrolment
number, such that no two students have the same number. This number is
then unique to each student. Therefore, by making this number an attribute of
the entity Student it can be used as this entity’s identifier. An entity identifier
must have a value, it can never be null, i.e. without a value.
If no single attribute possesses the property of uniqueness, then two or
more attributes must be selected to achieve this goal. Such a combination of
attributes is known as a composite entity identifier.
For example, the entity ClassRoomTimeTable has the following attributes:
Class Room Number, Period Of Day, Day Of Week, Class Code,
Subject Code
Some example occurrences are shown in Figure 10.1.3.
Entity ClassRoomTimeTable
Questions
3 How many occurrences of an entity must a value of an entity
identifier identify?
A. None B. One or more
C. Zero or more D. Exactly one
Questions
6 Which attributes of the following entities would make suitable entity
identifiers?
a) Patient
b) Ward
c) Nurse
Formally,
An entity description is a formal description of an entity consisting of its name
and attributes with the entity identifier indicated by being underlined, e.g.
GP( GPId, GPName)
Questions
The attributes of the entities Aeroplane and FlightStaff are listed below
Key concept
Relationship:
Relationships
A relationship is a two-way
Within an organisation or within a system in an organisation, entities do not association or link between two
exist in isolation but have links with other entities. entities.
Formally,
A relationship is a two-way association or link between two entities.
For example,
• A patient is nursed by nurses the nursed by relationship associates
a patient with nurses who nurse that
patient
• A nurse nurses patients the nurses relationship associates a
nurse with the patients who the nurse
nurses
In the school example, a link exists between the entities Student and Staff.
Students are taught by staff whose role is teaching, and staff whose role is
teaching teach students. In the direction from student to staff, the relationship
has the name Is Taught By. In the direction from staff to students the
relationship has the name Teaches.
Degree of a relationship
Key concept When the data requirements are described more precisely as follows:
Degree of a relationship: A member of staff may teach zero or more students (zero because
The degree of a relationship
not all staff teach, e.g. caretaker) and a student is taught by one
between two entities refers
or more members of staff
to the number of entity
occurrences of one entity which It becomes clear that a member of staff who teaches may teach many students
are associated with just one and a student may be taught by many members of staff.
entity occurrence of the other
and vice versa.
Formally,
The degree of a relationship between two entities refers to the number of entity
occurrences of one entity which are associated with just one entity occurrence of
the other and vice versa.
• one-to-one
• one-to-many
• many-to-one
• many-to-many
Teaches
Student Staff
Is Taught By
Teaches
Student Staff
Is Taught By
Figure 10.1.7 Many to many E-R diagram
However, it is not always necessary to label both ends of a relationship. Often,
common sense dictates that a single label describes a relationship adequately.
This label is placed in the middle of the relationship line connecting the two
entities.
Questions
9 For each of the following relationship definitions choose the
correct degree classification:
Ward Patient
(i) m : n
(ii) 1 : n
(iii) n : 1
(iv) 1 : 1
Driver Lorry
(i) m : n
(ii) 1 : n
(iii) n : 1
(iv) 1 : 1
Computer Program
(i) m : n
(ii) 1 : n
(iii) n : 1
(iv) 1 : 1
Order Part
Questions
10 The data requirements for a hospital in-patient system are
defined as follows:
(i) an attribute
(ii) a relationship?
(b) State four entities for the hospital in-patient system and
suggest an identifier for each of these entities.
• primary key Modelling data logically emphasises that we are still engaged in a stage of
modelling which is independent of a particular database system, e.g. Microsoft
• composite primary key
Access or MySQL. No details of how the data is to be physically stored and
• foreign key accessed will be considered.
The focus of interest in this chapter is the relational model and relational
database. In this model:
Key concept
A relational database is a set of relations or a collection of tables.
Relational database:
A relational database is a set of
relations or collection of tables.
A later chapter covers an alternative model, the fact-based model.
The conceptual model in the previous chapter concentrated on the structure
and meaning of the data for a specific organisation, without answering the
question: “how should the data be structured and interpreted in a software
system?”
A logical database model concentrates on the structure and meaning of the data
in a particular database approach or system, e.g. the relational database model
approach in which relationships between entities are modelled by shared or
common attributes alone.
The goal of the logical model is to create schemas from which a database can
be physically built. A schema is another name for a plan describing what is to
be built.
Relational Data Model
Mathematical relation
Information We focus first on the approach to logical data modelling known as relational
modelling.
The inclusion of the
reference to the concept This type of modelling is based upon a mathematical concept called a relation.
of a mathematical relation For example, if we have two sets named SetOfStudents and SetOfSubjects,
and its link to relational respectively, and populated as follows
database relations is purely
for background purposes. SetOfStudents = {Sarah, Jim, Kevin}
SetOfSubjects = {CS, Physics, Maths}
and a relationship between these two sets, called Studies, then we might model
Background this relationship as a set of ordered pairs as follows
Mathematical relation: Studies = {(Sarah, Physics), (Jim, Physics),( Jim, Maths) }
Relation Studies is a subset of
the Cartesian product The ordered pair, (Sarah, Physics), records that Sarah studies Physics.
The set Studies is a subset drawn from the set of all possible ordered pairs
SetOfStudents x SetOfSubjects
formed from the two sets SetOfStudents and SetOfSubjects.
or an element of the power set
Questions
℘ (SetOfStudents x
1 Write down the set of all possible ordered pairs formed by
SetOfSubjects).
the Cartesian product of SetOfStudents and SetOfSubjects of
See Unit 1 4.2.2. which (Sarah, Physics) is one example.
This way of organising these facts and their relationship uses the concept of a
relation – a relation corresponds to a relationship such as Studies used in the
above example.
Key concept We can use the relation to obtain information. For example, we can conclude
Relation: that Kevin does not study any of the subjects in SetOfSubjects, Sarah and Jim
A relation in the relational study Physics and Jim also studies Maths.
data model can be regarded
loosely as a form of table, with Relational data model relation
attributes being named columns A relation in the relational data model can be regarded loosely as a form of
of the table.
table, with attributes being named columns of the table. More precisely, the
table is a depiction of the relation presenting actual values for the relation in
tabular form.
Key concept
Rows of values of the table are called tuples. They correspond to instances of
Attribute:
records in a programming language.
An attribute is a named column
of a table. Relation ClassRoomTimeTable
Attributes
Relation name
Class
relation Period Of Day Of Class Subject
Room
attributes Day Week Code Code
Number
1 1 Monday 12C1 CS
tuples 2 2 Monday 12C1 CS
1 1 Tuesday 13D2 Phy
1 2 Tuesday 12C1 CS
Figure 10.2.1 Table depiction of relation ClassRoomTimeTable
If a combination of attributes is required to ensure uniqueness then all the Key concept
attributes in the combination are underlined. In this instance, the primary key Composite primary key:
is called a composite primary key. Minimal combination of
attributes that uniquely
For example, the relation ClassRoomTimeTable is written as follows:
identifies a single occurrence or
ClassRoomTimeTable(ClassRoomNumber, PeriodOfDay, DayOfWeek, tuple of the relation.
ClassCode, SubjectCode)
With the attribute combination ClassRoomNumber, PeriodOfDay and
DayOfWeek chosen as the primary key.
A value of this primary key, such as “2, 2, Monday” selects just one row or
tuple of the table because the combination is unique in the table. It is also
minimal, i.e. has no more attributes than necessary. If we removed one of
ClassRoomNumber, PeriodOfDay, DayOfWeek we forfeit uniqueness.
Questions
2 A SwimmingGalaRaceResult relation has the following attributes:
Questions
3 A CDTrack relation in a CD music collection system has the following
attributes:
(b) What is the name given to a primary key that consists of more
than one attribute?
Foreign Key
In a relational database, relationships are modelled by the foreign key
Key concept mechanism.
Foreign key: For example, in a hospital system the relationship between the two entities
A foreign key is an attribute
Ward and Patient has degree one-to-many as shown in Figure 10.2.2. Each
in one relation/table which is
also the primary key of another ward is occupied by zero or more patients.
relation/table. It forms a link
between two relations/tables via
this attribute.
Ward OccupiedBy Patient
Occupies
Figure 10.2.2 E-R diagram for the entities Ward and Patient
The entity definitions from the conceptual modelling stage are
Ward (WardName, NoOfBeds)
Patient (PatientNo, Name, HomeAddress, DateOfBirth, Gender)
The equivalent relations are defined as follows:
Ward (WardName, NoOfBeds)
Patient (PatientNo, Name, HomeAddress, DateOfBirth,
Gender, WardName)
The relationship between the two entities is represented in the relation Patient
by the additional attribute WardName. Foreign keys are usually indicated by
italicising them or placing a line over them.
WardName is the primary key of the relation Ward as well as the entity
identifier of the entity Ward and is known as a foreign key in the relation
Patient.
WardName is therefore common to both Patient and Ward.
A foreign key is an attribute in one relation/table that is also the primary key
of another relation/table. It forms a link between two relations/tables via this
attribute.
StudentId Forename
1 Jim
2 Kevin
3 Sarah
SubjectId SubjectName
1 CS
2 Physics
3 Maths
Table 10.2.2 Relation Subject as a table of values
Figure 10.2.3 shows the entity-relationship (E-R) diagram from the conceptual
modelling stage, the stage before the relational database modelling stage. The
diagram conveys that a student studies zero or more subjects and a subject is
studied by zero or more students (actually the E-R diagram needs an additional
symbol on each crow foot to indicate “zero or more”).
Studies
Student Subject
StudiedBy
Figure 10.2.4 shows the E-R diagram after the relational modelling stage. The
entity Studies has now been included (on an E-R diagram we use the term
entity).
A student is associated with zero of more subjects and a subject is associated
with zero or more students. The relation Studies provides the link that we need
between the relations Student and Subject. That is why we call the entity on
the E-R diagram corresponding to this relation, a link entity.
Questions
4 The entities Ward and Nurse are defined below
Ward(WardName, NoOfBeds)
Nurse(StaffNo, Name)
Questions
5 Each patient is registered with one GP whereas a GP has
many patients. The entity definitions for these data
requirements are
GP(GPId, GPName)
Patient(PatientNo, Name, HomeAddress, DateOfBirth,
Gender, NHSNo)
Treats
GP Patient
RegisteredWith
ParticipatesIn
Event Team
Involves
(b) Using the new E-R diagram from (a) write down the
relations.
Questions
7 The entity-relationship diagram for a hospital system is shown below.
OccupiedBy
Ward Patient
StaffedBy Nurses
Nurse Treats
Consultant
The entities and their attributes for this E-R diagram are:
Ward(WardName, NoOfBeds)
(b) Using the new E-R diagram from (a) write down the
relations.
time. It might make sense to model with separate entities if drivers are assigned
and re-assigned to different cars, frequently.
ManagedBy
Table Nurse
Manager
StaffNo Name Rank
StaffNo
1 Sarah Briggs Staff Nurse 3
2 John Doe Staff Nurse 3
3 Sue Cripps Ward Sister 4
4 Mary Downs Senior Nurse Null
Table 10.2.4 Example occurrences of the relation Nurse
Note that it is possible to have no value at all in a foreign key column, i.e. a
null value, in any relationship, recursive or not. If a value is present then there
must be a matching value present in the primary key column, e.g. 3.
Questions
Manages
Teacher
9 Some teachers manage other teachers.
The entity description for the ManagedBy
Teacher entity is as follows
Teacher (TeacherId, Name, SubjectId)
(i) BloodDonor
(ii) BloodGiven
718 Bloggs
Key concept 345 Khan
1 Smith
234 Teng
Repeating group: 456 Nunn
Domain for attribute(s) is a
structured type so an instance
1118 Archer
is a group of values, known as
1305 Ali
a repeating group because its 2 Smith
2214 Singh
group-like structure repeats
6541 Nunn
from row to row.
8600 Bloggs
4341 Sorensen
3011 Minns
1678 Ng
2999 Zog
Repeating group
Domain for attribute(s) is a structured type so an instance is a group of values,
known as a repeating group because its group-like structure repeats from row to
row.
To indicate that a relation contains a repeating group a line is placed above the
attribute(s) that contain groups of values
GP (GPId, GPName, Patient)
The solution to the repeating group problem in this example is to remove
the Patient values to their own relation called Patient as shown below, and to
add the primary key of the GP relation to this new relation to model the link
between GP and Patient relations as follows
GP (GPId, GPName)
Patient (PatientId, PatientName, GPId)
There is now no repeating group in either relation – see Table 10.3.2 – and
what’s more
Every non-key attribute is a fact about the key,
the whole key and nothing but the key.
where key means primary key.
Because this is the case we then can say that relations GP and Patient are fully
normalised.
Key concept
Single-Valued Fact (SVF):
When a relation is fully normalised, instances of a single-valued fact (SVF) will be recorded just once, in just one place.
Single-valued facts assert that one thing is associated with just one other thing.
The instances of single-valued facts for the fully normalised GP and Patient
relations are recorded just once, in just one place:
• GPId determines GPName because, given a specific value of GPId, say 2,
there is just one and only one value of GPName, Smith, associated with
this value of GPId.
• PatientId determines PatientName because, given a specific value of
PatientId, say 1305, there is just one and only one value of PatientName,
Ali, associated with it.
Information
Determinant
Another way of expressing a single-valued fact is to draw a determinancy diagram.
PatientId → PatientName
The diagram asserts that attribute PatientId determines attribute PatientName and therefore PatientId is a
determinant. There is a single value of PatientName for each value of PatientId, i.e. given a value of PatientId
such as 345 the name Khan is returned from a search of the Patient table – Table 10.3.2. However, it is not
true that there is only one value of PatientId associated with each value of PatientName. Two patients can have
the same name but must have different values of PatientId, e.g. given the name Bloggs a search of the Patient
table returns the values of PatientId 718 and 8600.
We say that PatientId determines PatientName or PatientName depends upon or is functionally dependent upon
PatientId or is a fact about PatientId.
Determinants are useful because from a list of determinants drawn up from the data requirements, it is
possible to immediately write down a set of fully normalised relations.
Questions
1 Write down the determinants from the data in Table 10.3.1.
Using these determinants can you see how to write down the
normalised relations?
2 The relation Ward is unnormalised
Teacher Student
Table 10.3.4 shows the TeacherTeaches table for a sample of teachers and the
students that they teach
The table contains a repeating group and so its relation is unnormalised.
We indicate the repeating group with an overline as follows
TeacherTeaches (TeacherId, TeacherName, Student)
where the Student repeating group is an array of StudentId, StudentName,
SubjectName.
A first step to resolving the issue is to eliminate the repeating group, Student,
in the TeacherTeaches relation. The simplest way of doing this is to make a
new row in TeacherTeaches for each row of the repeating group and to replace
the Student column by three columns StudentId, StudentName, SubjectName,
respectively. This step is called putting the relation in First Normal Form or
1NF.
Task
1 Draw up the table for this new form of the relation using the data in
Table 10.3.4.
Clearly it is not true for this new form of the relation TeacherTeaches that
Every non-key attribute is a fact about the key, the whole key and
nothing but the key.
For example, TeacherName is a fact about TeacherId which is only part of the
key.
Questions
3 What is
(a) StudentName a fact about?
(b) SubjectName a fact about?
Teacher Student
Teaches
Figure 10.3.2 E-R diagram for Teacher, Student and Teachers entities
Questions
4
SalesmanId CustomerId Salesman
1
1 Archer
2
3
2 Dent
4
5
3 Rogers
6
Questions
6 A student is taught by one or more tutors and a tutor teaches one or
more students as shown in the table below.
7
PatientNo PatientName WardName WardType
1456 Smith Nightingale Orthopaedic
1461 Berry Barnard Cardiac
1468 Thomas Barnard Cardiac
1472 Harley Guttman Orthopaedic
1478 Smith Barnard Cardiac
1483 Noggs Spens Geriatric
1497 Smith Nightingale Orthopaedic
Information
A fully normalised set of relations contains no redundant data.
Redundant data:
Data is redundant when it
can be deleted without loss of
There may be duplication of data in a fully normalised set of relations but it is
information.
non-redundant and necessary duplication.
Normalising relations to third normal form
Why don’t we just use E-R modelling to arrive at a set of fully normalised
relations?
Key principle
If we have an alternative then we have a means of checking the completeness,
A fully normalised set of
accuracy and consistency of the E-R model.
relations contains no redundant
In this section, we describe a formal technique for arriving at a set of fully data.
normalised relations which is an alternative to E-R modelling.
This formal normalisation technique consists of stages known as Normal Form
(NF).
Starting with the unnormalised relations
Key principle
• Remove repeating groups to place relation in First Normal Form (1NF) Normalising relations to Third
Normal Form:
• Remove partial dependencies to place relation in Second Normal Form
(2NF) 1. Remove
repeating groups
• Remove transitive dependencies to place relation in Third Normal Form
→ First Normal Form
(3NF) (1NF)
A relation which is in 3NF is automatically in 2NF. A relation which is in 2NF 2. Remove
partial dependencies
is automatically in 1NF.
→ Second Normal Form
(2NF)
Information 3. Remove
There are several higher forms of normalisation than 3NF. These forms are transitive dependencies
used in highly specialised cases. In normal cases, a relation that is in 3NF is → Third Normal Form
also in these higher forms. (3NF)
We have already used the informal test for BCNF in earlier sections
Every non-key attribute is a fact about the key, the whole
key and nothing but the key.
It contains within it the tests for 2NF and 3NF
Information
Functional dependency:
Given a relation R, attribute R.B, e.g. PatientName, is functionally
dependent on attribute R.A, e.g. PatientId, if and only if each value of R.A
is associated with precisely one value of R.B, at any one time.
Questions
8 The relation GP is unnormalised GPId GPName Patient
718 Bloggs
GP (GPId, GPName, Patient) 345 Khan
1 Smith
234 Teng
456 Nunn
The table opposite shows some data for this relation.
1118 Archer
the patient data is a combination of PatientId and 1305 Ali
2 Brown
PatientName. 2214 Singh
6541 Nunn
Table StudentCourse
StudentId StudentName Gender CourseCode CourseTitle TeacherId TeacherName
AQA0643 A Level CS 1234 Mead
15898 Bond M UCL0675 A Level Maths 5678 Davies
EDE0187 A Level Art 9123 Milsom
UCL0675 A Level Maths 5678 Davies
24298 Smith F AQA0643 A Level CS 1234 Mead
AQA0432 A Level ICT 1234 Mead
EDE0187 A Level Art 9123 Milsom
10598 Robert M UOC0987 A Level French 4567 Crapper
AQA0432 A Level ICT 1234 Mead
13497 Nixon F UOC0987 A Level French 4567 Crapper
Table 10.3.7 Table StudentCourse unnormalised
Repeating group shown in red.
Stages of normalisation
1NF: Remove repeating groups to place relation in First Normal Form.
The First Normal Form of the table, Table 10.3.8 is shown below.
StudentId StudentName Gender CourseCode CourseTitle TeacherId TeacherName
15898 Bond M AQA0643 A Level CS 1234 Mead
15898 Bond M UCL0675 A Level Maths 5678 Davies
15898 Bond M EDE0187 A Level Art 9123 Milsom
24298 Smith F UCL0675 A Level Maths 5678 Davies
24298 Smith F AQA0643 A Level CS 1234 Mead
24298 Smith F AQA0432 A Level ICT 1234 Mead
10598 Robert M EDE0187 A Level Art 9123 Milsom
10598 Robert M UOC0987 A Level French 4567 Crapper
10598 Robert M AQA0432 A Level ICT 1234 Mead
13497 Nixon F UOC0987 A Level French 4567 Crapper
Table 10.3.8 Table StudentCourse in First Normal Form
StudentId on its own no longer satisfies the criterion of uniqueness and so a
new primary key must be found. StudentId together with CourseCode becomes
the new primary key.
The relation in first normal form is
StudentCourse(StudentId, CourseCode, StudentName,
Gender, CourseTitle,TeacherId, TeacherName)
2NF:
Second Normal Form - Every non-primary key attribute is functionally
dependent on the whole of the primary key.
The next stage is to remove any non-primary key attributes that depend only
upon part of the primary key to separate relations to achieve Second Normal
Form.
Task
2 Discuss why TeacherName is dependent on the primary key
CourseCode of relation Course. Look at the data requirements.
Questions
9 Table 10.3.9 shows some data for a TeacherTeaches relation
Questions
10 Swimming galas take place at different venues during the course of
the swimming season. Each gala consists of several races. Each gala
race consists of several swimmers and is for one particular swimming
stroke, e.g. breast stroke and one particular distance, e.g. 50m.
Races are numbered starting from 1 for each gala. All galas code
the strokes in the same way: strokes are numbered from 1 and each
has a stroke name, e.g. 1 is always breast stroke. Each venue has a
unique name, e.g. Leeds. Each swimmer is assigned a country-wide
unique swimmer number so that no two swimmers will use the
same swimmer number at any gala. The swimmer’s time for a race is
recorded.
Questions
A band will never have more than one booking on any particular date.
Extension Material
Boyce-Codd Normal Form
We have already stated that Boyce-Codd Normal Form is the next stage after 3NF.
Informally, when a relation is in BCNF, every attribute which is not part of the primary key, is a fact
about the key, the whole key and nothing but the key (“so help me Codd”).
Formally,
BCNF a relation is in Boyce-Codd Normal Form (BCNF) if and only if every determinant is a
candidate key.
We have already seen that determinants are useful. From a list of determinants drawn up from the data
requirements, it is possible to immediately write down a set of fully normalised relations in BCNF.
The BCNF test will immediately reveal if there are any redundant data in a relation or table.
BCNF &
3NF and BCNF are equivalent for a relation with only one candidate key.
3NF
Minimality The minimum number of attributes which guarantee uniqueness are used.
A relation may have more than one candidate key. One is chosen as the primary key and the others are
alternate keys.
For example, if each student is allocated a locker to store their things and pays a refundable deposit, the
Student relation becomes
Student(StudentId, StudentName, LockerNo, Deposit)
Both StudentId and LockerNo each determine StudentName
and both StudentId and LockerNo are unique and candidate keys for the Student relation but only one can be
chosen as primary key.
Extension Questions
12 The table below shows some sample data for relation Patient.
Patient is not fully normalised
PatientId GPId GPName PatientName
1 1 Biggs Singh
2 1 Biggs Brown
3 2 Smith Mian
4 2 Smith Fadhil
5 3 Biggs Fadhil
6 3 Biggs Brown
(a) State which of the following are candidate keys for relation Patient.
(i) PatientId
(ii) GPId
(iii) GPName
(iv) PatientName
(c) Normalise the relation Patient for the data in the table into
a set of relations in BCNF.
(d) Why is the set of BCNF relations identical to the 3NF set?
Student
StudentId Gender
Name
1 Ames M
2 Baloch F
3 Cheng F
4 Dodds M
5 Groos M
6 Smith F
Table 10.4.1 Relation Student in table form
The ANSI/ISO SQL standard requires that a semicolon is used at the end of the SQL statement but some systems
relax this requirement. When writing SQL the convention is to use upper case for the SQL commands.
If we wanted just the data for StudentName we would refine the query as follows
SELECT StudentName
FROM Student;
We could refine the search even further by adding a WHERE clause that applies a search condition as follows
SELECT StudentName
FROM Student
WHERE Gender = 'F';
The result set that would be returned when this query is applied to relation Student would be as follows
Baloch
Cheng
Smith
because only these rows of the table/tuples of the relation match the search condition Gender = 'F'.
Gender = 'F' is actually called a predicate because it evaluates to either TRUE or FALSE.
If we also wanted the values of StudentId returned then the query would be
SELECT StudentId, StudentName Questions
FROM Student 1 Write an SQL query that returns the names of all students in
WHERE Gender = 'F'; Table 10.4.1 who are male.
Table 10.4.3 shows data in table form for the Patient relation Table 10.4.2 Relation Ward in table form
Patient (PatientId, Surname, WardName) PatientId Surname WardName
The two relations are linked via a shared or common attribute 1 Bond Aylesbury
WardName. The existence of an attribute common to both relations 2 Smith Victoria
is not enough to join data from the corresponding tables correctly, as 3 Jones Aylesbury
the following SQL query demonstrates 4 Biggs Victoria
SELECT Ward.WardName, Ward.NurseInCharge, Table 10.4.3 Relation Patient in table form
Patient.PatientId 1
Victoria Sister Bunn
FROM Ward, Patient; Victoria Sister Bunn 2
The part of the query Ward.WardName references the WardName Victoria Sister Bunn 3
attribute in relation Ward and the part Patient.PatientId Victoria Sister Bunn 4
references PatientId attribute in relation Patient. Aylesbury Sister Moon 1
Aylesbury Sister Moon 2
The FROM Ward,Patient part joins both relations without regard
Aylesbury Sister Moon 3
for the way that the data is actually linked via matching values of the
Aylesbury Sister Moon 4
shared attribute, WardName. The result set returned by the query is
shown in Table 10.4.4. Table 10.4.4 Result set ignoring
relationship between Ward and Patient
520 Single licence - Abingdon School
Using SQL to retrieve, update, insert and delete data
The problem is caused by the fact that our query does not make use of the common attribute that models the real
world relationship between ward and patient, WardName.
When the search condition
WHERE Ward.WardName = Patient.WardName
is added to the SQL query we are able to exclude values that are not linked by the attribute WardName and to
include only those that are. This SQL query will return the result set that corresponds to the real world situation
shown in Table 10.4.5.
Aylesbury Sister Moon 1
SELECT Ward.WardName, Ward.NurseInCharge, Patient.PatientId
Victoria Sister Bunn 2
FROM Ward, Patient Aylesbury Sister Moon 3
WHERE Ward.WardName = Patient.WardName; Victoria Sister Bunn 4
The two relations have been joined on their common attribute, WardName, i.e. Table 10.4.5 Result set taking
where the value of WardName is the same in both relations. account of relationship between
Ward and Patient
Writing the query as follows would return the same result set because dropping
the relation name prefix before NurseInCharge and PatientId in the SELECT part of the SQL query is allowed
where there is no ambiguity as to what is intended.
SELECT Ward.WardName , NurseInCharge, PatientId
FROM Ward, Patient Background
WHERE Ward.WardName = Patient.WardName;
Integrity:
Means maintaining the accuracy
Questions and consistency of the data and
relationships between data.
2 Write the SQL query that returns from Tables 10.4.2 and 10.4.3 the
name of the nurse in charge of the ward, surnames of all patients in
Referential integrity:
their ward and the ward name. Each foreign key value must
have a matching primary key
Role of a foreign key value in a related table, or be
Null.
What purpose does a foreign key serve if it doesn’t automatically join the
corresponding two relations? Null:
The answer is so that the relational database system can perform a referential Special value that is used to
indicate the absence of any data
integrity check, i.e. check that the referenced primary key value exists before it
value.
is used as a foreign key in another relation. For example, if an attempt is made
to add a new patient to the Patient table, either 'Victoria' or 'Aylesbury' must
be chosen for WardName because these are the only two values present in the linked-to table Ward. The database
system would use a referential integrity check to trap any other entered value for WardName and report an error.
Similarly, if an attempt was made to delete the row in the Ward table for ‘Victoria’ ward, an error would occur
because this value is used in the Patient table. In both of the cases described, preventing these changes occurring
maintains the integrity of the data in the database of those attributes which make references to other attributes.
Questions
5 Write the SQL query that returns the ward names and patient surnames , from Tables 10.4.2 and 10.4.3,
for which the patient identifier is less than or equal to 3. Order the result set in descending order of patient
identifier.
Customer
Ordering on more than one attribute CustNo Location
Name
Table 10.4.8 shows two database tables, Customer and Order,
Kauai Dive Kapaa Kauai
respectively. 1221
Shoppe
The effect of applying the following SQL query to this database 1351 Sight Diver Kato Paphos
SELECT Customer.CustomerName , Order.OrderNo
Tom Sawyer Christiansted
FROM Customer, Order 1356
Diving Centre
WHERE Customer.CustNo = Order.CustNo
Blue Jack Waipahu
ORDER BY Customer.CustomerName, Order.OrderNo; 1380
Aqua Center
is to return the following result Blue Jack Aqua Center 1001 Davy Jones’ Vancouver
2156
set with columns CustomerName, Blue Jack Aqua Center 1009 Locker
OrderNo. Sight Diver 1005
OrderNo CustNo SaleDate
Note that, the result set is Sight Diver 1007
1007 1351 20140412
ordered on the first column, Tom Sawyer Diving Centre 1003
1009 1380 20140417
CustomerName and then the Tom Sawyer Diving Centre 1004
1004 1356 20140420
second OrderNo. 1005 1351 20141106
In the Order table, the order of the values of OrderNo for the 1003 1356 20141113
1007 1001 1380 20141202
1009 orders placed by these customers is shown on the left. The data
1004 is extracted in this order. It is next sorted on the first column, Table 10.4.8 Tables Customer and
1005 CustomerName to produce the order of values shown on the Order
1003 right abbreviated. The rows with the same CustNo values are
Blue… 1009
1001 then sorted on OrderNo to produce the result set that is returned by the SQL
Blue… 1001
query.
Sight… 1007
Logical operators for search condition Sight… 1005
Table 10.4.9 shows some of the logical operators that can be used in SQL queries. Tom… 1004
Tom… 1003
Logical Operator Description
The AND operator connects multiple search conditions in an SQL statement’s WHERE
AND
clause that must all be true for a successful match.
The OR operator allows multiple conditions to be combined in an SQL statement’s
OR
WHERE clause any of which can be true for a successful match.
The NOT operator reverses the meaning of the logical operation with which it is used e.g.
NOT
NOT (age > 14 AND age < 19).
Table 10.4.9 Logical operators for SQL queries
When more than one logical operator is used in a statement, NOT is evaluated first, then AND, and finally OR.
To impose a different order of evaluation parentheses can be added to the query. The logical conditions inside
parentheses are evaluated independently before logical operators outside of the parentheses are applied.
Table 10.4.10 and Table 10.4.11 show data for relations Borrower and BooksOnLoan.
Table 10.4.10 Table showing some Table 10.4.11 Table showing some values for the relation
values for the relation Borrower BooksOnLoan
The result set returned when the following SQL query SELECT Borrower.Surname, Borrower.Initial
The search condition "WHERE BorrowerId NOT IN BooksOnLoan" matches BorrowerId 3 because this
BorrowerId is not present in the BooksOnLoan table.
The result set returned when the following SQL SELECT Borrower.Surname, Borrower.Initial
query FROM Borrower
WHERE BorrowerId IN (SELECT BorrowerId
is applied to these tables is shown below
FROM BooksOnLoan
Smith, K
WHERE BooksOnLoan.DateDueBack < '5/9/2014');
The search condition
WHERE BorrowerId IN (SELECT BorrowerId
FROM BooksOnLoan
WHERE BooksOnLoan.DateDueBack < '5/9/2014')
matches BorrowerId 1 because this BorrowerId has a DateDueBack value of ‘4/9/2014’ in the BooksOnLoan table
and this value is less than ‘5/9/2014’.
The result set returned when the following SQL SELECT Borrower.Surname, Borrower.Initials
query is applied to these tables is shown below FROM Borrower
WHERE BorrowerId =
Barnes W (SELECT BorrowerId
FROM BooksOnLoan
WHERE BooksOnLoan.ISBN = 9781907982514
AND BooksOnLoan.CopyNo = 1);
7 SELECT Capital, Population What result set is returned when this SQL query is
FROM Country applied to the data in Table 10.4.12?
WHERE Name IN ('Chile',
'Cuba', 'Guyana');
Questions
What result set is returned when this
SQL query is applied to the data in Table 8 Rewrite the query in Q7 so that it uses
10.4.12? OR instead of IN.
The second form specifies both the column names and the values to be inserted:
INSERT INTO table_name (column1, column2, column3, ...)
VALUES (value1, value2, value3, ...);
In the first form, a value of the correct data type must be supplied for every attribute of the relation and the order of
the supplied values must be the same as the corresponding columns.
In the second form, a value for every specified column must be supplied and each value must match in data type the
corresponding specified column, i.e. value1 corresponds to column1, value2 to column2, etc. The value Null will be
inserted for any columns not referenced.
WardName NurseInCharge NoOfBeds
For example, for relation Ward, Table 10.4.2, reproduced here Victoria Sister Bunn 30
Second form:
INSERT INTO Ward (WardName, NurseInCharge) VALUES ('Savernake', 'Sister Teng');
This second form creates a new row in Table 10.4.2 with values 'Savernake', 'Sister Teng', Null
Questions
11 Write the SQL statement to add a new row to the Ward table (Table 10.4.2) for ward 'Amersham',
containing 25 beds. The nurse in charge is 'Sister Brody'.
12 Write the SQL statement to add a new row to the Country table (Table 10.4.12) for 'UK', 'London'.
The empty table can be filled with data using the INSERT INTO statement.
SQL Constraints
SQL constraints are used to specify rules for the data in a table and are applied either at the level of column or at
the level of the table.
Column level:
CREATE TABLE table_name
(
column_name1 data_type(size) constraint_name,
column_name2 data_type(size) constraint_name,
column_name3 data_type(size) constraint_name,
....
);
There is a variation on syntax for specifying primary key. It can also be defined at table level as shown below. You
will need to check which syntax is accepted by the database system that you use. A foreign key is expressed at table
level as shown below.
Table level:
CREATE TABLE Patient
(
PatientId SMALLINT,
Surname VARCHAR(25) NOT NULL,
FirstName VARCHAR(25),
Gender CHAR(1) CHECK (Gender = ‘M’ OR Gender=’F’),
WardName VARCHAR(25),
PRIMARY KEY(PatientId),
FOREIGN KEY(WardName)
REFERENCES Ward(WardName)
);
Data types
Table 10.4.13 shows some general data types used in SQL (SQL:1999 standard) when defining tables using
CREATE TABLE.
CHARACTER(n) or CHAR(n) Character string. Fixed-length n. Default length is 1
when n is omitted. Use CHAR when the sizes of the
column data entries don’t vary much.
VARCHAR(n) or CHARACTER Character string. Variable length. Maximum length n.
VARYING(n) or CHAR Default length is 1 when n is omitted. Use VARCHAR
VARYING(n) when the sizes of the column data entries vary
considerably.
NCHAR(n) National character string type that uses fixed width
Unicode string. Width specified by n.
NCHAR VARYING(n) National character string type that uses variable width
Unicode string. Maximum width specified by n.
BOOLEAN Stores TRUE or FALSE values.
SMALLINT Integer numerical (no decimal). Signed unless keyword
UNSIGNED present. Precision 5. Exact numeric type.
INTEGER Integer numerical (no decimal). Signed unless keyword
UNSIGNED present. Precision 10. Exact numeric
type.
DECIMAL(p, s) or DECIMAL Exact numerical, precision p, scale s. Example: decimal
(5, 2) is a number that has 3 digits before the decimal
point and 2 digits after the decimal point. Exact
numeric type so can be used to represent money. Can
omit (p, s) to obtain default precision and scale.
FLOAT(p) or FLOAT Approximate numerical, mantissa precision p. A
floating number in base 10 exponential notation. The
size argument for this type consists of a single number
specifying the minimum precision. Approximate
numeric type. Can omit (p) to obtain default precision.
REAL Approximate numerical, mantissa precision 7.
DATE Stores year, month, and day values.
TIME Stores hour, minute, and second values.
Table 10.4.13 Table for some general data types used in SQL:1999 (ISO/IEC 9075-2:1999 (E))
If it does exist then it will be opened by the same command and command line argument.
Next, the database’s first table is created with the Data Definition Language (DDL) command
The data types applied to the attributes are VARCHAR(20) and SMALLINT.
The command .schema when executed at the command prompt returns the DDL script that was used to create
tables and any other structures such as indices, triggers, etc.
The DML command to add to table Ward the tuple ('Victoria', 'Sister Bunn', 30)
is as follows INSERT INTO Ward VALUES ('Victoria', 'Sister Bunn', 30);
Confirmation of the success of this insert is shown in Figure 10.4.1 using the Data Manipulation Language
(DML) query SELECT * FROM Ward;
Next, the database’s second table, Patient, is created with the Data Definition Language (DDL) command below,
also shown in Figure 10.4.2.
CREATE TABLE Patient
(PatientId SMALLINT PRIMARY KEY, Surname VARCHAR(20), WardName VARCHAR(20),
FOREIGN KEY(WardName) REFERENCES Ward(WardName));
This table is linked to the Ward table by a foreign key which is created in table Patient by
The DML command to add to table Patient the tuple (2, 'Smith', 'Victoria')
is as follows INSERT INTO Patient VALUES (2, 'Smith', 'Victoria');
The value 'Victoria' used for the attribute WardName of Patient satisfies the referential integrity condition of
existence of this value in the Ward table. If it didn’t the INSERT action would be aborted and the reason for the
failure to carry out the command reported in an explanatory error message.
Confirmation of the success of this insert is shown in Figure 10.4.2 using the Data Manipulation Language
(DML) query SELECT * FROM Patient;
Questions
14 Write the DDL statements to create tables in database Library for the relations
Borrower (BorrowerId, Surname, Initial)
BooksOnLoan (ISBN, CopyNo, BorrowerId, DateDueBack)
as shown in Tables 10.4.10 and 10.4.11.
Concurrent access
Key term Transactions submitted by users at client workstations can sometimes target
Client server database: the same data items in the database. Access to the same data items by different
In a multi-user client server users can occur over the same time interval, i.e. concurrently. This can result in
database system, stored data
an inconsistent database if not carefully controlled.
items in a database on a server
An inconsistent database is one in which the state of the database does not
may be accessed simultaneously
by programs running on client truly reflect reality, e.g. the number of airline seats sold should be 120 but
workstations. the database has recorded only 118 seats sold. The integrity of the data in the
database will be affected. Data integrity refers to the accuracy and consistency
of the data in the database.
Key term Lost update
To illustrate this potential problem consider an airline flight reservation
Concurrent access:
Means access occurs at the same
database system which stores information about each flight such as:
time or over the same time • flight number
interval.
• date of flight
• number of seats sold
• the number of seats left to sell
Suppose that a computer terminal located in a travel agent’s office in Birmingham attempts to book three seats
about the same time as a terminal located in a travel agent’s in Swindon attempts to book five seats on the same
flight. They each request copies of data for this flight from the server located in London. Figure 10.5.2 illustrates
what could happen. The problem that ensues is known as the lost update problem. This occurs when two
transactions that access the same database items have their operations interleaved in such a way that makes the value
of some database item incorrect.
Key term
Flight Code AY67
Lost update problem:
An update is lost when two Flight Date 21/11/2016
transactions that access the
same database item have their No of seats sold 120
operations interleaved in such
No of seats unsold 140
a way that makes the value of
the database item incorrect or
inconsistent.
Each office gets a copy
of the same information
If we consider the detail of how individual seats are booked then we see that it is a two-stage process as shown in
Figure 10.5.3:
1. The database is first queried to obtain the seating plan for a specific flight, e.g. flight AY67 on 21/11/2016,
using an SQL Select statement sent from the client to the database server
2. The SQL query results returned are used to select a seat which is not already reserved (SeatBooked =
FALSE) and then an SQL update operation is sent to the database server to set the chosen seat to reserved
(SeatBooked = TRUE) for the given flight and date.
Update Flights
Set SeatBooked = TRUE, CustID = 2003
Where FlightNo = 'AY67'
And Date = '21/11/2016'
User Client And SeatID = '5B',
1
Database
Server
User
Client A B C D E F
2 Update Flights
Set SeatBooked = TRUE, CustID = 2165 1
Where FlightNo = 'AY67'
2
And Date = '21/11/2016'
And SeatID = '5B', 3
4 AY67
5 21/11/2016
Which customer is allocated seat 5B? 6
7
8
Figure 10.5.3 Two users, User 1 and User 2, attempting to reserve the same seat at the same
time and in the process allowing one user to overwrite the other’s reservation
Single licence - Abingdon School 535
10 Fundamentals of databases
A problem arises if the SQL Update operations proceed as shown in Figure 10.5.3. Both User 1 and User 2 believe
that seat 5B is available at the time when they each send an SQL Update operation to the database server to reserve
seat 5B. The database can only record this seat as being booked by one customer, either the customer with CustID
2003 or the customer with CustID 2165. If a database server allows uncontrolled access to a specific record in
the manner described then it will become inconsistent, i.e. its record of the bookings will not agree with its users’
record. To avoid this problem the database server must control concurrent access to its database.
Record locking
Key concept
One approach relies upon a locking mechanism. Consider a simplified version
Record lock: of the scenario shown in Figure 10.5.3 in which relation Flight consists of
A record lock is a concurrency attributes, FlightNo and NoOfSeatsSold. Its table depiction consisting
control method which can
of two columns is shown in Table 10.5.1.
prevent a lost update. The lock
ensures exclusive-access to a FlightNo NoOfSeatsSold
record when it is being updated. 1 50
Other transactions are blocked 2 120
from accessing the record until 3 87
after it has been updated and the 4 156
change permanently recorded.
The blocked transactions then Table 10.5.1 Relation Flight consisting of two columns in table form
see the updated record. Now suppose that two clients, User 1 and User 2, attempt to access the same
row of the Flight table, e.g. row 1.
Let User 1 retrieve row 1 first. After that, assume that User 2 retrieves the same
row.
Time User 1 User 2
postgres=# BEGIN; USER 1 starts transaction postgres=# BEGIN; USER 2 starts transaction
BEGIN BEGIN
postgres=# SELECT * FROM Flight WHERE
FlightNo = 1 FOR UPDATE; FlightNo | NoOfSeatsSold
-------------------------
1 | 50
postgres=# SELECT * FROM Flight WHERE
FlightNo = 1;
postgres=# UPDATE Flight SET NoOfSeatsSold BLOCKED
= 52 WHERE FlightNo = 1;USER 1 sells 2 seats
UPDATE 1
Table 10.5.2 shows User 1 using the BEGIN command to start a transaction on the PostgreSQL database containing
the table Flight. User 1 wishes to update the NoOfSeatsSold field of the record in the first row.
The "FOR UPDATE" part locks the record to prevent other users from accessing this record until the lock is
released. This lock is known as an exclusive-lock for this reason.
User 2 sends the following SELECT query to the database server just after User 1 gets the query result shown
above
SELECT * FROM Flight WHERE FlightNo = 1
User 2’s query is blocked because the record with FlightNo = 1 is locked.
User 1 then sends the following UPDATE command to the database server
UPDATE Flight SET NoOfSeatsSold = 52 WHERE FlightNo = 1;
To commit this update to the database and end the transaction, User 1 sends the COMMIT command.
This update is applied and the lock is removed. Whereupon, the pending SELECT query from User 2 is executed
by the database server and the following query results are returned to User 2
FlightNo | NoOfSeatsSold
--------------------------------
1 | 52
If the isolation mode is set to SERIALIZABLE then the database server can detect and prevent the LOST
UPDATE problem occurring. A characteristic of the LOST UPDATE problem is that the transactions that cause
this conflict are not serialisable, i.e. do not behave as if they were executed one right after another in a serial fashion.
The isolation level can be set using the SET TRANSACTION CHARACTERISTICS command or the SET
SESSION CHARACTERISTICS. These set the default transaction characteristics for a transaction and for
subsequent transactions of a session, respectively.
Table 10.5.3 User 1 and User 2 interacting concurrently with the Flight table
when isolation level is READ COMMITTED
Table 10.5.3 shows User 1 and User 2 interacting concurrently with the Flight table when the isolation level
is READ COMMITTED. This setting fails to prevent the Lost Update problem. What is needed is a way of
detecting and preventing concurrent transactions that would not execute serially, if attempted. Figure 10.5.4 shows
User 2’s attempt to update the same record as User 1 but later in time than User 1. The database server forces User
2 to abort its update because the server has concluded that it would not be possible to serialise these updates. When
538 Single licence - Abingdon School
10.5 Client server databases
User 2 restarts the transaction it finds that the NoOfSeatsSold field has been updated. User 2 uses the new value
52 recorded in the database, adding one to calculate the value 53 which it uses to update the database.
NoOfSeatsSold = 50 Make change permanent NoOfSeatsSold = 52
Figure 10.5.4 The effect of serialisation on
BEGIN SELECT UPDATE COMMIT USER 1 two transactions interleaved in time when
both attempt to update the same data item
NoOfSeatsSold ← 52 Update record
Could not serialize access NoOfSeatsSold = 53
NoOfSeatsSold = 50 due to concurrent update NoOfSeatsSold = 52
NoOfSeatsSold ← 51 NoOfSeatsSold ← 53
ROLLBACK
cancel change
Table 10.5.4 shows the sequence of events for two users interacting concurrently with a postgreSQL database when
the transaction isolation is set to SERIALIZABLE.
Table 10.5.4 Effect of serialisation on two transactions interleaved in time when both update the same data item
and the isolation level is SERIALIZABLE
Single licence - Abingdon School 539
10 Fundamentals of databases
Key concept Typically, timestamp values are assigned in the order in which transactions are
submitted to the system, so a transaction timestamp can be thought of as the
Timestamp ordering:
transaction start time. We refer to the timestamp of transaction T as TS(T). A
A timestamp is a unique
identifier created by a database simple counter which starts at zero can be used as the source of transaction
server that indicates the relative number. Each new transaction causes the counter to be incremented and the
starting time of a transaction. resulting new counter value is assigned to the transaction.
The database records the
transaction timestamp of the last The database server records the largest timestamp for a data item, X, when
transaction to read data item X a read transaction is performed on the data item, X - ReadTS(X). It records
and the transaction timestamp separately, the largest timestamp for a data item, X, when a write transaction
of the last transaction to write is performed on the data item, X - WriteTS(X). The counter starts initially at
data item X.
zero.
The database server applies rules
using these to determine if a Let’s suppose User 1 begins a transaction, T1, that contains a Read request
transaction’s actions will result followed by a Write request. The timestamp counter is incremented and its new
in the integrity of the database
value 1 (it started at 0) assigned to transaction T1.
being compromised. If it will
the server aborts the transaction. User 2 begins a transaction T2 and is assigned transaction time stamp value
2. T2 also contains a Read request and a Write request. Now suppose the time
ordering of these Read and Write requests is as shown in Table 10.5.5. User
2 completes before User 1. Therefore, User 1’s Read data is potentially out of
date. Therefore, User 1’s transaction, T1 must be aborted.
Table 10.5.5 Timestamp ordering protocol ensures serializability among transactions in their conflicting
read and write operations
The following rules are used with timestamps to determine whether a transaction operation is allowed or not.
Rules: Another transaction has read
1. Transaction T1 issues a WriteItem(X) operation: X since T1 started or another
transaction has modified X
If ReadTS(X) > TS(T1) Or WriteTS(X) > TS(T1)
since T1 started
Then abort and rollback T1 and reject the operation.
Else
Another transaction modified
If WriteTS(X) <= TS(T1)
X but before T1 started
Then execute the WriteItem(X) operation of T1
and set WriteTS(X) to TS(T1)
Else
Another transaction modified
If WriteTS(x) <= TS(T1)
X but before T1 started
Then execute the ReadItem(x) operation of T1
If we revisit the User 1 and User 2 lost update problem, then an attempt by User 1 to update the Flight record
for which FlightNo = 1 will result in User 1’s transaction being rescheduled because User 2’s transaction
timestamp has a larger counter value than User 1’s.
Commitment ordering
Commitment ordering is used in client server database systems such as mobile banking, and e-commerce.
A transaction is an “all or nothing” unit of work. A transaction is either committed, i.e. its effects on all the
resources (data items) involved become permanent, or it is aborted (rolled back), i.e. its effects on all the resources
are undone. The term that summarises this is atomicity:
In database systems, an atomic transaction is an indivisible and irreducible series of database operations such
that either all occur, or nothing occurs.
When a transaction is in conflict with another transaction, i.e. committing Key concept
both would lead to database inconsistency such as the lost update problem, one
Commitment ordering:
of the transactions is aborted and started again. This is a form of concurrency
Used in a distributed database
control. system such as mobile device
Concurrency control applied to a client server database system where clients are client server systems where
mobile devices presents its own special problems. Mobile devices move from transactions are created locally
on the mobile device before
base station to base station and the connection with the database server can be
being sent to the server for
intermittent. What happens if a transaction is started on a mobile device which execution. The transactions are
fails to complete in a reasonable time because the connection is lost? There scheduled at the server so that
is a danger, if locking is used, of a data item being locked in exclusive-access they are committed in an order
that avoids concurrency conflict.
mode for a lengthy period of time thus preventing other transactions from
completing.
One solution is for transactions to be done locally on the mobile device before they are sent to the server for the
final stage. In a non-distributed system, the transactions are created centrally at the server. However, mobile device
database systems uses a distributed model in which the transactions are generated locally. Relative timestamps
are assigned locally to operations (Read, Update, Insert, Delete). The list of operations with timestamps is then
transferred to the server. The absolute time of the execution of the operations is calculated in the server using
the relative timestamps. The server uses a scheduling algorithm to produce a commit order which avoids conflict
between transactions. This is what is meant by commitment ordering.
Questions
1 What is a client server database system?
4 Describe three methods that are used in non-distributed client server database systems to prevent the Lost
Update problem occurring.
5 Describe a method used in distributed client server database systems such as mobile banking to prevent the
Lost Update problem occurring.
Information
Google recognised in 2000 that Figure 11.1.1 Schematic of the three V’s, Volume, Velocity and Variety
the custom-built infrastructure
it had been relying on to provide Questions
its search engine service was
no longer fit for purpose. 1 What is meant by Big Data?
This infrastructure could not
be scaled to cope with the
2 Name the three defining characteristics of Big Data.
increasing volume of data being
collected nor the demands its
users were placing upon it.
Volume
Google were unable to buy
Volume in Big Data refers to the size of the data to be processed. Data-analysis
a solution off the shelf, at the
time, because no commercially
scenarios that process hundreds of terabytes or petabytes fall into the Big Data
available software existed that category if that data must be analysed as a single dataset.
could handle this volume and
A physical data centre that hosts an exabyte of data is not necessarily dealing
level of demand.
with Big Data. This data might be nightly backups stretching back over ten
Google’s solution was to design
and build their own new data years. However, if, say, a petabyte of data needs to be analysed to answer a given
processing infrastructure to solve question, then this is definitely a Big Data problem. Thus the “Big” in Big Data
this problem. is more than an assessment of the size of the data to be stored, it really relates to
The two key services in this the processing of this size of data.
system were
• a distributed file system, When the data volumes that needed to be processed outgrew the storage and
the Google File System, or
processing capabilities of a single host an alternative to the traditional data
GFS, which provided fault-
tolerant, reliable, and
processing approach was required.
scalable storage In a traditional data processing system,
• MapReduce, a data Information
processing system based ■■ file blocks belonging to a file of
on a functional data would be stored on a single A file is divided into blocks of a
programming paradigm server fixed, pre-determined size. The
that allowed work to be split chosen block size corresponds
among large numbers of ■■ The program written to execute to the block size written to a
servers and carried out in this traditional data processing physical disk.
parallel. activity would also execute in a
Both GFS and MapReduce single server.
were designed to run on the
commodity server hardware In the Big Data era, the blocks of a file have to be distributed across more
that Google used throughout its
than one server simply because there are too many of them to fit one server. In
data centres.
traditional data processing systems the block size is typically 512 bytes whereas
Information
in a Big Data processing system, the block size is typically 64 or 128 megabytes.
Growth in data:
Secondly, the program that does the processing must now be written so that it
The seventh annual Digital Universe
can execute on more than one machine at the same time.
study (2014)
Summarising, the Big Data approach to processing a dataset that has already (http://www.emc.com/about/news/
hundred thousand servers in a data centre this means a probability that a disk
drive will fail in the next hour of about 33%.
Information
Figure 11.1.4 shows Google’s first
Block 1 Block 3’
server rack circa 1999. Their first
search engine ran on this. It was
retired from service after five years Block 3 Block 1’
to the Computer History museum,
Mountain View, California, USA.
Block 2’ Block 2’’
Block 2
Information
HDFS stores files across a cluster.
A Hadoop cluster is a special type Block 1’’
of computational unit designed
specifically for storing and analysing Server +
Disk Drive Block 3’’
large datasets across a distributed
computing environment. One
Rack 1 Rack 2
machine in the cluster is designated
as the NameNode and another
machine as JobTracker - these are Figure 11.1.5 In HDFS each block is written three times and at least one
the masters. The rest of the machines of the three is written to a different server rack
in the cluster act as both DataNode
and TaskTracker; - these are the Both GFS and HDFS are scalable. More servers (known as nodes) can be
slaves. Hadoop clusters are often added if the file grows in size because all the racks of servers are interconnected
referred to as “shared nothing”
and the file system is able to keep track of which blocks belong to which file
systems because the only thing
that is shared between nodes is the
wherever the blocks are stored whether Rack 1 or Rack 10,000.
network that connects them.
Questions Tasks
5 What is meant by a distributed file system? 1 Watch a YouTube video
about a Facebook data
6 What is meant by fault-tolerant? centre
https://www.youtube.com/
7 Explain how fault-tolerance is achieved in HDFS. watch?v=0pB9falsA9k
8 What is meant by the term scalable in the context of HDFS and GFS?
Information
Function-to-data model Vertical scaling:
The second aspect of the Big Data approach is parallelising the execution of Takes place in a single server
programs. This is done with the function-to-data model. In the function-to- when more CPUs, memory, hard
drives are added.
data model because there is so much data, the analysis program is sent to the
data, i.e. a copy is sent to each server. This model is used in Hadoop which is Horizontal scaling:
an open source highly scalable, distributed batch processing system for large Takes place when more servers
datasets. At the heart of of the function-to-data model is a technique borrowed are added. It is like adding another
Information
Information Traditional database systems
use special database servers
MapReduce
of high reliability. Such systems
MapReduce is a programming model which uses a parallel, distributed
attempt to solve the scaling
algorithm to process large datasets “at rest” in a cluster. MapReduce takes
problem by vertical scaling. This
an input, splits it into smaller parts, executes the code of the mapper on
is an expensive option because
every part, then gives all the results to one or more reducers that merge
highly-reliable components are
all the results into one.
expensive.
For example, suppose that we wanted to demonstrate determining the
number of times each word appears throughout a collection of texts, e.g.
Google’s solution was to
the novels War and Peace, Les Miserables, Three Men in a Boat and Fireside
scale horizontally using much
Stories.
cheaper commodity hardware.
First, the files containing the text of these novels would be copied from
Commodity hardware is cheaper
where they are stored to Hadoop’s distributed file system HDFS as shown
but less reliable. Google’s solution
in Figure 11.1.6.
was to build in fault-tolerance so
that the failure of a component
Next a copy of a Map function would be distributed to each HDFS server.
would not bring down the system.
In this example, the Map function would be something along the lines
The collection of key-value pairs emitted by each copy of map would be stored temporarily before being
grouped and sorted by shuffle and sort operations, e.g. all <A, 1> key-value pairs would be grouped together
and placed before all <About, 1> key-value pairs. Finally, copies of a Reduce function count all <A, 1>s, all
<About, 1>s and so on to produce a total count for each word.
function wordcountReduce (word, value)
{sum = 0
for (nextvalue in value) {sum += nextvalue}
emit(word, sum)
} Map code sent to After shuffling key-value
servers hosting pairs sorted alphabetically
the input files’
blocks so net-
work traffic Sort
HDFS across the cluster Shuffle
Blocks minimised Phase
Finally, Reduce
<A 1>
<A 1> <A 1> produces a total
Map <cat 1> Shuffle
Linux File A cat sat on a mat....
<sat 1> <About 1> Reduce for each word
System <About 1>
<THREE 1>
THREE MEN IN A BOAT (TO
Map Sort
SAY NOTHING OF THE DOG)....
<MEN 1>
Output file
<IN 1> Shuffle <THREE 1>
<THREE 1> spread across
several
<Well 1> Reduce
<Well 1> HDFS
Linux File System Blocks
block size Map Shuffle
4kB
Map emits key-value pairs, Sort
HDFS e.g. <cat 1>
block size for each word it encounters
64MB Running MapReduce on a
hduser@ubunutu: hadoop fs -put /etc/gutenberg/ /user/hduser/gutenberg Ubuntu
hduser@ubuntu: hadoop jar hadoop-mapreduce-examples-2.7.0.jar wordcount /user/hduser/gutenberg /user/hduser/gutenberg-output
Figure 11.1.6 MapReduce applied to text of several novels. The text is copied from the host operating system to
Hadoop’s HDFS file block system which sits on top of the host file system
Extension Question
9 Describe MapReduce in the context of counting word occurrences in a large dataset consisting of text.
Variety Information
Data can appear in many forms from structured through semi-structured to
unstructured. The simple, structured data
managed by legacy Relational
Structured data database management systems
Structured data is data that can be represented in tabular form. Such data is (RDBMS) invented in the 70s
modelled in the relational model as described in Chapter 10.2. Structured data continue to be used for simple,
lends itself to formal modelling of the data before building a database because structured data. They are not
suitable for complex data
the data has a clear and identifiable structure. In the relational model, the data
because such data does not
is structured into tuples that record values for a fixed number of predefined easily fit the row and column
attributes, format required by relational
<memo>
e.g. attributes databases. Data that fits the
<to>Sarah</to> latter is highly structured and
StudentId,
<from>Fred</from> can be pre-modelled prior to
StudentSurname,
<about>Milk for tea and coffee </about> collection and storing.
etc. However, a different kind
<status>Urgent</status>
Semi-structured of data classified as semi-
<message>
data structured or unstructured
Don’t forget to buy a pint of milk! now predominates.
This is data such
</message> Unstructured data can be
as XML- or
</memo> difficult to pre-model because
JSON-formatted of its lack of structure. It is
files that do not also characterised by sheer
<memo> volume and frequency of
have a formal
<to>Fred</to> generation. Examples are Web
structure but
<from>Sarah</from> logs of events such as mouse
nevertheless have
<about>Milk for tea and coffee </about> clicks on Web pages; text,
some structure, images, video, audio, sensor
<moan>I am always buying the milk</moan>
albeit of a variable readings, output from scientific
<reply>Bought one pint!</reply> modeling, and other complex
kind. Figure
</memo> data types streaming into data
11.1.7 shows
centres today.
Figure 11.1.7 Two related examples of XML two related New data processing
data showing the semi-structured nature of examples that are techniques must be applied to
this type of data XML-formatted. extract the value contained
The structure within this data.
Algorithms tailored to both
of each memo is clearly identified by tags that could be predefined but the
the data and the questions that
actual structure of each memo cannot be formally modelled because it is not can be answered are applied to
consistent. No schema is enforced to ensure consistency. each of the data types such as
images, video, or audio.
Unstructured data The Big Data approach is then
This is data such as text whose content is so variable that able, by linking the results, to
deliver insights that would not
■■ it cannot be modelled in advance or
be possible otherwise.
■■ it cannot be fitted to a column-row structure required by relational
database modelling or
■■ its (key) elements are not identified with tags.
Information Examples include, the body of e-mail messages, web pages, text files, video,
Spring XD can work with
images, data from sensors. Even though each of these examples does have
both the Twitter Search API structure it is not of the semantic kind, e.g. a Web page contains HTML
(twittersearch) and data from mark-up such as paragraph tags, but this is solely for the purpose of
Twitter’s Streaming API. rendering the Web page and not for specifying its meaning (semantics) and
therefore what information it conveys.
Information
Before you can start consuming Questions
data from Twitter into Spring XD,
10 Explain the meaning of the Big Data characteristic known as variety.
you need to set up a development
account and install the Twitter
development environment. If
Velocity
you don’t have a Twitter account,
you can sign up for one at http:// Velocity refers to whether the data is in motion and with what frequency
www.twitter.com. The developer or at rest.
website enables users to create
Data at rest is data that has been stored on some permanent data storage
keys for their applications to
access Twitter’s APIs. These device. The data may be processed at any time because it is permanently
keys need to be placed in a stored and speed of processing is not critical as the rate of arrival and
configuration file - xd/config/ processing of this data can be controlled. Big Data at rest is usually batch
modules/modules.yml:
processed. In batch processing, processing, once started, is carried out to
twitter:
completion without user interaction.
consumerKey:
consumerSecret: Data in motion is data that is streamed at some frequency
accessToken: continuously, e.g. 1000 events per second. There is an expectation that it
accessTokenSecret:
will be collected at the rate it arrives. In stream processing, data is processed
as it arrives and before it is stored permanently.
Information In the Big Data scenario, this data is likely to arrive at a high rate, and
Spring XD can be downloaded
from multiple sources simultaneously - e.g. log data, Twitter streams,
as a zip file from https://spring.io/
and RSS feeds. The site http://developer.usa.gov/1usagov sends event
blog/2015/04/30/spring-xd-1-2-m1-
and-1-1-2-released for the version notifications at the rate of less than 10 per second. The stream is composed
available at time of writing this of JSON events, each of which is generated every time someone clicks
book. on any US .gov or .mil URL that has been shortened. When this site is
Unzip and install. To start the
launched in a browser a simple long-lived HTTP connection is established
server on a Windows machine
click on
and data is subsequently streamed back to the browser until the HTTP
\spring-xd-1.2.0.RELEASE\xd\bin\ connection is ended. 10 events or less per second is manageable by one
xd-singlenode.bat for single node machine but what if the velocity was increased to about 46,000 events per
operation. second - see www.internetlivestats.com. This equates to the rate at which
Launch the command shell by
Google receives search hits.
clicking on
\spring-xd-1.2.0.RELEASE\shell\ Imagine that Google streamed these search request events to the world.
bin\xd-shell.bat. Any system receiving these would have to cope with collecting the events
See www.educational-computing.
at a velocity of 46,000 per second. This would require scaling the collection
co.uk/CS/Book/SpringXD/
SpringXDExamples.txt for
part of the system so that it would cope, i.e. the equivalent of adding some
examples to try. more lanes to a highway.
Information
Figure 11.1.8 shows a basic stream of event-driven data passing from source to sink in Spring XD, a stream processing system applied to
Big Data. The source converts data arriving at unpredictable times into an internal message format. The messages travel along channels
between any number of processors. One kind, a “wire tap”, is shown in the figure. The data travelling in the stream is tapped into and
some processing carried out. In this example it is a simple count of tweets arriving from Twitter. The main stream is set up and deployed
with the shell command
stream create tweetcount --definition “tap:stream:tweets>aggregate-counter” --deploy true
The wire tap is set up with
stream create tweets --definition “twitterstream | file” --deploy true
Using Spring XD a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export
sensors xd>stream create tweets --definition "twitterstream | file " --deploy true
(e.g. temperature)
data feed
(e.g. stock change, message Wire tap message
credit card transaction)
Source Sink
mobile
tweets
tweets tweets
social ts
ee message
(tweets, tw HDFS
channel
count of tweets
blogs)
message
tweets.out
storage
sink for
Spring XD count
xd>stream create tweetcount --definition "tap:stream:tweets> aggregate-counter" --deploy true
Figure 11.1.8 Streaming using Spring XD tool to pull tweets from Twitter, perform some processing on the
stream as it is in motion with atap into the stream and storing all tweets received in tweets.out for later batch
processing
The role of the sink is to store the stream data permanently. This could be as a file using the file system of the operating system or it
could be HDFS, for example, a distributed file system built on top of the native file system of the operating system. In addition to the
stream data being “tapped into”. It could be processed in transit before being stored, e.g. case converted from lower case to upper case.
Figure 11.1.9 shows the visualisation of tweets filtered by the “lang” field in tweets at a wire tap. The visualisation updates in real time.
To see this in action download the video from www.educational-computing.co.uk/CS/Book/Videos/TweetLang.avi.
A stream can be thought of as a series of connected operators. The initial set of operators (or single operator)
are typically referred to as source operators. These operators read the input stream and in turn send the data
downstream. The intermediate steps comprise various operators that perform specific actions. Finally, for every
way into the in-motion analytics platform, there are multiple ways out, and in streams these are outputs called sink
operators - see Figure 11.1.8. In a more technical sense, a stream is a graph of nodes connected by edges. Each
node is an operator.
Questions
11 Explain the meaning of the Big Data characteristic known as velocity.
Machine learning
Machine learning techniques are needed to discern patterns in data and
to extract useful information. Big Data datasets are analysed with machine
learning techniques which enable the value in the datasets to be extracted. This
can take the form of a predictive model that can then be used in the algorithm
that processes streaming data to extract the value from the data in the stream.
The value of data in large datasets was revealed in 2009 when a new flu virus,
H1N1 hit the streets. Public Health authorities feared a pandemic on the scale
of the 1918 Spanish flu that killed millions.
Doctors were requested to inform health authorities of new flu cases. However,
people might feel ill for days but wait before seeing a doctor. With other delays
in the reporting system and a rapidly spreading disease the authorities were
unable to get a clear picture.
About the same time, Google engineers had reported in the science publication
Nature how Google could “predict” the spread of the winter flu in the United
States, not just nationally, but down to specific areas of the States by what
552 Single licence - Abingdon School
11.1 Big Data
people searched for on the Internet using the three billion search queries a day that they were receiving and saving.
This led to Google identifying the areas infected by the flu virus.
Google processed some 450 million different mathematical models in order to test search terms, comparing their
predictions against actual flu cases from 2007 and 2008. The result was their software found a combination of
45 search terms that, when used together in a mathematical model, strongly correlated their predictions with the
official historical figures. The result: Google could tell where the flu had spread in near real time, not a week or two
after the fact. Google’s result was built on Big Data. It is Big Data and machine learning applied to Big Data that
enables the value in data to be extracted and used to provide useful insights of significant value.
To understand how machine learning applied to a dataset or datasets can produce a predictive model, consider a
much simpler case of trying to find the relationship between growth rate of something of interest and temperature.
Figure 11.1.10 shows some data plotted on an x-y graph together with a best-fit straight line generated using linear
regression, a common machine learning technique. The programming language used was R, a strongly functional
programming language for statistical computing. The code to produce this graph and to generate the model and
plot the best-fit straight line is shown below.
R programming language
Information
R programming language:
> temperature <- c(10,20,30,40,50,60,70,80,90,100,110,120,130,140
R is a free software environment
,150)
for statistical computing and
> growthrate <- c(20,26,38,49,45,55,55,55,67,74,74,77,83,95,108) graphics.
R can be downloaded from
> plot(temperature, growthrate)
https://cran.rstudio.com/.
> model <- lm(growthrate~temperature) It can also be run online at
www.getdatajoy.com
> abline(model)
Figure 11.1.10
Plot of growth rate
against temperature
generated in the
programming
language R
Information
Alarm HDFS:
system Hadoop Distributed Filing System.
Data stream
Figure 11.1.11 Predictive model algorithm applied to stream data from a temperature sensor
Generating a predictive model is done with the data at rest. The model is then
applied to streaming data. A classic example is fraud detection on credit card
transactions - see http://www.fico.com/en/wp-content/secure_upload/FICO_
Real_Time_Fraud_3095IN.pdf.
Recommendation systems also fall into this category - see http://techblog.
netflix.com/2012/04/netflix-recommendations-beyond-5-stars.html.
Questions
12 Explain how machine learning is used in Big Data systems to leverage
value in stored datasets. Give real examples of two different types of
machine learning Big Data systems.
Programming tasks
1 Gain access to the R programming system and launch the GUI.
Create your own example dataset of temperature and growth rate.
2 Plot these with growth rate on the y-axis and temperature on the x-axis.
> plot(temperature, growthrate)
Section 3
Fact-based model
Learning objectives:
The fact-based model is a conceptual model for modelling data. Physical
■■Be familiar with the: models based on the fact-based model are Bigtable used by Google, Cassandra
• fact-based model for used by Apple, Facebook, EBay, Twitter, Instagram. It models data in a
representing data completely different way from the relational model and it fits the Big Data
approach better than the relational model does. Each fact within a fact-based
• graph schema for capturing
model captures a single piece of information.
the structure of the dataset
Data is immutable
• nodes, edges and properties In the fact-based model data is immutable, i.e. it cannot be altered except to
in graph schema delete data which has been entered erroneously as a result of human error. This
is completely different from the relational model in which data is mutable, i.e.
Principle
Fact-based model:
can be overwritten with new values in an update operation. In the relational
• Raw data stored as atomic model, an erroneous entry caused by human error cannot be undone once
facts committed, i.e. it is not possible to go back to a previous value. Table 11.1.1
• Each fact captures a single shows a section of student data from a fact-based model system. As students
piece of information
move from year to year they move classes. This information is recorded as a
(i.e. atomic)
• Facts are kept immutable historical record, e.g. student 1 was assigned to class Year 11 D on 03/06/2014
and eternally true by using and then roughly a year later to class Year 12 B on 27/08/2015. The historical
timestamps record for 03/06/2104 is not removed because in the fact-based model, data is
• Each fact is made immutable.
identifiable so that query
processing can identify StudentId YearClass TimeStamp
duplicates (facts with same 1 Year 11 D 03/06/2014 10:35:16
identity) 2 Year 11 A 03/06/2014 10:38:05
• A nonce is used to make 3 03/06/2014 10:44:45
Year 11 D
identical facts identifiable (a
1 Year 12 B 27/08/2015 12:45:51
nonce is a randomly
generated 64-bit number) 2 Year 12 E 26/08/2015 11:15:31
Table 11.1.1 Student data for a fact-based model
If data had been modelled using the relational model then student 1’s data
Information would have been updated, and in the process the value Year 11 D would have
Immutability in the fact- been overwritten with the value Year 12 B.
based model:
The ability of a Big Data system to store vast quantities of data in a single
The immutability concept applied
to datasets in the fact-based dataset lends itself well to the fact-based model which requires a change to
model was borrowed from be recorded as a new fact and not applied as an update - the master dataset
functional programming. continually grows with the addition of immutable, timestamped data.
Human fault tolerance
People will make mistakes. The impact of such mistakes is minimised in the
fact-based model because it is an immutable data model, no data can be lost. If
bad data is written, earlier (good) data units still exist.
In comparison, in the relational model, a mistake can cause data to be lost
because values are overwritten in the database.
Data is true forever
The key consequence from immutability is that each piece of data is true in
perpetuity, i.e. forever: a piece of data, once true, must always be true. Key fact
Simplicity
Advantages over relational
Databases based on the relational model rely on indexes to retrieve and update
model:
data. These must be built and subsequently modified as data is updated/altered/ • Simplicity - no indexing
deleted. • Append - new data units just
added to end of dataset
In the fact-based model, data is immutable so all that is required is the ability
• Perpetuity - data is true for
to append new data units to the master dataset. This does not require an index ever
for the data. This is a significant simplification. • Human fault-tolerant -
Atomic facts immutable facts means
human errors can be
In the fact-based model, data is broken down into fundamental units that are
corrected by returning to
called facts. A fact, for example, is student 1 is in class Year 10 D. Another fact
earlier “good” facts
is, student 1 is in class Year 12 B. What differentiates these two facts in time are • Historical queries - queries
their timestamps, 03/06/2014 10:35:16 and 27/08/2015 12:45:51. can be historical because
facts immutable
A fact possesses two core properties:
• Partial information - null
■■ it is atomic value not required, simply
omit missing information
■■ it is timestamped.
Facts are atomic because they can’t be subdivided further into anything
Key fact
meaningful.
Fact-based model:
Collective data such as the classes for a student and their subjects studied are
With a fact-based model, the
represented as multiple, independent facts as shown in Figure 11.1.12. master dataset will be an ever-
growing list of immutable,
1 in Year 10 D atomic facts.
(02/06/2013 09:05:26)
Big data techniques are required
1 in Year 11 D 1 in Year 9 A in order to accommodate the
(03/06/2014 10:35:16) (04/06/2013 13:43:05) growing list of immutable atomic
facts, i.e. more than one server
will typically be required for the
master dataset.
1 in Year 12 B Raw data about 1 studies maths
(27/08/2015 12:45:51) student 1 (02/06/2013 09:33:03)
Concept
1 studies Physics Atomic fact:
(02/06/2013 09:38:29) 1 studies English Facts are atomic because they
(02/06/2013 09:35:13)
can’t be subdivided further into
anything meaningful.
1 studies CS
(02/06/2013 09:36:18)
Firstname: Surname:
Student (StudentId): Kevin Bond
2
PageView Class:
Nodes are the entities Student (StudentId) Year 11 D
in the system. 1
PageView
Figure 11.1.13 The graph
Page(url): PageView Class:
schema for students and http://192.168.0.32/physics Year 12 B
intranet page access.
There are two node types: Timestamps are not
Page(url):
students and pages. http://192.168.0.32
shown to avoid cluttering
Student nodes have Total views: the diagram.
150
properties Firstname,
Properties are information about
Surname and Class. Page Edges are the
entities, e.g. student or page.
nodes have one property, relationships between
Total views:
Totalviews nodes. 50
Concept The graph in Figure 11.1.13 represents the facts for the student and intranet
Graph schema: page access dataset. The graph highlights the three core components of a graph
A graph schema captures the schema—nodes, edges, and properties:
structure of a dataset stored
■■ Nodes are the entities in the system. In this example, the nodes are
using the fact-based model.
Structure means a description students and pages.
of the types of facts contained ■■ Edges are relationships between nodes. In this example an edge
in the dataset, e.g. student
between a student and a page represents the relationship between a
has surname of data type
string, and a description of the page and a user who has visited this page.
relationships between entities, ■■ Properties are information about entities. In this example, surname,
e.g. student visits intranet page,
firstname, class, total views.
student and intranet page are
entities Edges are strictly between nodes and are shown visually connected
by a solid line. Dashed lines are used for connecting properties to the
corresponding node.
In the example an intranet page is able to identify a student by StudentId, e.g. Information
student 1. A relationship labelled “PageView” exists between a student and a page. Graph schema:
Another way that an intranet page could identify students is by using a cookie. The graph schema provides a
complete description of all the
The intranet site places a cookie on the student’s computer and when a student
data contained within a dataset.
returns to this intranet site, the student is identified by the stored cookie, e.g.
cookie abs123. At some point, the intranet site might be able to associate
student 1 with this cookie if it is stored on student 1’s computer. A node labelled Principle
Student(CookieId) abs123 could then be connected to node labelled Student Nodes, edges, properties:
Tasks
2 Research Google’s data storage system, Bigtable (http://static.googleusercontent.com/media/research.google.
com/en//archive/bigtable-osdi06.pdf ) and Google’s Cloud Bigtable (https://cloud.google.com/bigtable/).
Questions
15 Describe the fact-based model.
19 In the fact-based model, what is (a) a node (b) an edge (c) a property?
20 Complete the graph schema in Figure 11.1.14 to represent Location : Name : Alex
the following facts: Reading
(a) Alex, Kevin, and Sue are friends Age : 23
(b) Alex’s age is 23, Kevin’s is 24 and Sue’s is 22
(c) Alex lives in Reading, Kevin lives in Aylesbury, Sue lives in Bristol
Person ID:1
21 Alex and Sue each post a message on an electronic message board.
Extend your graph schema to include these facts.
What use would be made of timestamps in this graph schema? Figure 11.1.14
22 Why does the fact-based model require Big Data techniques?
23 Give three advantages of the fact-based model over the relational model.
■■The set A is called the domain maps 0 to 0, 1 to 1, 2 to 4 and 3 to 9 when the rule is: output the square of the
and the set B is called the input.
co-domain.
Function as process
■■The domain and co-domain In function as process, a function is a rule that tells us how to transform some
are always subsets of objects in information into some other information, e.g. the integer 2 into its square 4.
some data type.
Function as object
Key principle In function as object, the function is a thing in its own right.
Function as process: For example, a pencil sharpener is an object. If the focus of attention is a pencil
A function is a rule that tells then the pencil sharpener just represents a process - sharpening pencils, input:
us how to transform some
unsharpened pencil; output: sharpened pencil.
information into some other
information.
In the function as process view, we are applying the function sharpen to
Function as object: pencils; it’s the pencil that counts. But we can also think about the pencil
The function is a thing in its sharpener as a thing in its own right, when we empty it of pencil shavings, or
own right. worry about whether its blade is sharp enough. This is the function as object
view.
Questions
A function f
f : {0,1,2,3} → {0,1,2,3, ..., 25, 26, 27}
maps 0 to 0, 1 to 1, 2 to 8, 3 to 27.
1 What is the rule?
A function f
f : {0,1,2,3} → {0,1,2,3, 4, 5, 6}
maps 0 to 0, 1 to 2, 2 to 4, 3 to 6.
2 What is the rule?
Questions
3 For each of the following what is the function as process and
what is the function as object?
(a) A single sheet of A4 paper containing text is placed in
the machine whose action is to produce a
printed copy of the sheet.
(b) A kitchen tool is used to remove skin from
potatoes.
Key principle
What is a function type?
Function type:
Just as data values (e.g. 6, 9.1, True) have types (integer, real, Boolean
A function f which takes an
argument of type A and returns respectively) so do functions. Function types are important because they state
a result of type B has a function what type of argument a function requires and what type of result it will return.
type which is written
A→B
A function f which takes an argument of type A and returns a result of type B
has a function type which is written
A →B
To state that f has this type, we write
f:A→B
For example,
1) squareroot : real → real
2) square : integer → integer
square (2) → 4
Practical Activity
Use a text editor such as NotePad++ to write Haskell programs. Save these
Haskell programs using extension .hs.
Figure 12.1.1.1 shows NotePad++ being used to create a function named
square with one parameter x of data type Integer and a body x∗x. This file has
been saved with filename square.hs in folder c:\book\haskell.
The :: operator (read as has type) is used in Haskell to express what type an
expression has.
Integer is the type of mathematical integers (int could have been used and is
the type of integers that fit into a word on the computer - this will vary from
computer to computer).
Launch WinGHci if you are using a machine running the Windows operating
system (ghci on Linux-based machines). The WinGHci window is shown in
Figure 12.1.1.2.
Information For example, in Python a function may be created and assigned to a variable v
Alternatively, using the keyword lambda as follows x**2 means x2
v = lambda x: x**2
def square(x):
return x*x It may be called in a print procedure to square the value 5 as follows
v = square print (v(5))
print(v(5))
Figure 12.1.2.1 shows the above coded and executed in IPython using Python
3.4.1.
The value 25 is returned by the call v(5) and the procedure print then
outputs this value to the screen.
The function is defined with
lambda x: x**2
This is an anonymous function with one argument x and a body x**2. It
is a first-class object because it can be assigned to a variable and passed as an
argument to another function/procedure.
The following Python code defines a function exp with one argument n that
returns a function also with one argument x and a body x**n.
def exp (n): return lambda x: x**n
print (f(4))
For example, if f is called with argument 4 then the function with body x**3
is returned with x replaced by 4. The function call f(4) will therefore return
43 which is 64. We can then display this value on a screen with the print
procedure.
Questions
1 What requirements must be satisfied for an object or value to be
classified as first class?
Functions as arguments
Functions as arguments is particularly useful when a common underlying
pattern can be identified. For example, suppose we need to double the result of
squaring, cubing, etc. Although trivial, it does illustrate how this generalisation
could be used for more complicated cases.
We first define a function double which takes a single argument
afunction. The argument afunction is a function type. The function
double returns 2*afunction.
Key principle
In Python 3.4, double would be defined as follows
def double (afunction) Functions as arguments:
return 2*afunction Functions as arguments is
particularly useful when a
We can now define maths functions square, cube and so on in Python 3 as common underlying pattern
follows can be identified. The common
def square(x): programming patterns that
recur in code, but which are
return x*x
used with a number of different
def cube(x) functions can be abstracted and
return x*x*x then given a general name, e.g.
sum where summing might be
Now we can use double, square and cube as follows a sequence of natural numbers,
double(square(4)) numbers squared, numbers
cubed, terms in an infinite
The function call double(square(4))returns 32. series, etc.
double(cube(4))
The function call double(cube(4))returns 128.
Programming task
1 Using a text editor such as NotePad++ enter the following Haskell
program and save as summy.hs.
summy afunction 0 = 0
square n = n*n
cube n = n*n*n
identity n = n
Formula
A formula is an expression containing variables. It represents a whole class of
computations, since the variable(s) can assume different values.
For example, the formula corresponding to the computation above
a x b + c
uses the variables a, b, c.
Substituting integers for the variables, a, b, c, produces an expression
whose value can be computed. For example, a = 3, b = 4, c = 2
3 × 4 + 2 → 12 + 2 → 14
For example, suppose we wished to calculate the sum of the first n natural
numbers. We have two choices of method or formula for calculating this sum:
n × (n + 1)
2
and
1 + 2 + 3 + 4 + ….+ (n – 1) + n
All that a user needs to know are the number and order of the inputs and
the name of the function in order to be able to apply the function to these
inputs.
Parameters
The formula used to define a function is sometimes called the body of the
function. The name used for the quantity that can vary is called the parameter
of the function. Therefore in the example above n is the parameter.
n × (n + 1)
The function associated with the formula is given a name, e.g.
2
sum.
To use the sum function, we apply it to an argument.
For instance, to find the sum of the first 6 natural numbers, the function sum
is applied to the argument 6.
Thus the parameter, n, is the name used in the function body to refer to the
argument, 6.
To compute the value of the function for some argument, replace the parameter
in the body of the function by the argument and compute the expression.
For example,
n × (n + 1)
n=6
2
6 × (6 + 1)
=
2
= 21
-> Integer
See Unit 1 Chapter 4.2.2 for
an explanation of the Cartesian
product integer x integer,
Questions
1 In Haskell a function square is defined as follows
square :: Integer -> Integer
square n = n*n
What is the result of the function application
(a) square 3 (b) square 5?
In this chapter you have covered: 3 What are the values of the arguments used in question 1?
■ What is meant by
• a computation 4 What is meant by function application?
• a formula
• procedural and functional abstraction
• parameter of a function
• argument of a function
■ Function application is when a function is applied to a particular
argument, e.g. 6 in Sum(6) or Sum 6.
Station frequency
Electromagnetic waves FM Radio Sound waves
Input Output
If we apply the function to two arguments, the result is a number; so that, for
instance, add 2 3 equals 5.
2 add 5
3
What happens if add is applied to one argument 2? Pictorially we have
2
y add
Input Output
573 Single licence - Abingdon School
12.1.4 Partial function application
The function add2 is a function. It is function add partially applied to one Key principle
argument, the argument that substitutes for input x, i.e. in Haskell
Partial function application:
Integer -> (Integer -> Integer) Any function taking two
Input Output or more arguments can be
partially applied to one or more
We feed in the particular value, 2, for input x and get as output a function arguments.
add2 which takes an integer and returns an integer.
The function add2 is denoted by output (Integer -> Integer)
y output
as shown in Figure 12.1.4.2.
This is an example of a general principle: any function taking two or more Information
arguments can be partially applied to one or more arguments. Partial function application makes
It also means that sense for two or more argument
Integer -> Integer -> Integer functions but not for one
argument functions.
is equivalent to
Integer -> (Integer -> Integer)
Questions
1 A computer can be considered a “partially applicable” device.
A computer is used to calculate some output from a given input, but before performing
the calculation, it is provided with a program – its first argument.
program output
computer
input
Redraw the figure to show the computer as a “partially applicable” device executing a
program with the name prog1.
Redraw the figure to show this function partially applied to the argument 10.
3 In Haskell the partially applied multiply function in question 2 has the function type
Integer -> (Integer -> Integer)
Input Output
If we apply the function to three arguments, the result is a number; so that, for
instance, addxyz 2 3 4 equals 9.
2
3 addxyz 9
4
What happens if addxyz is partially applied to one argument 2? Pictorially we
2 have
2
Integer -> (Integer -> Integer -> Integer) y addxyz
Input Output z
Input Output
y add2yz
z
Function add2yz is thus a two argument function.
What happens if addxyz is partially applied to two arguments 2 and 3?
2 3
Integer -> Integer -> (Integer -> Integer) 2
3 addxyz
z
Input Output
This function application computes 2 + 3 + z. If we treat 2 + 3 + z
as a new function it represents a function add23z, with one input z. The
function addxyz applied to arguments x = 2, y = 3 returns a function
add23z of function type Integer -> Integer
Figure 12.1.4.3 Shows partial function application for three-argument function addxyz
Programming task
1 Using a text editor such as NotePad++ write a Haskell program for functions multiplyxy which
takes two integer arguments and returns x times y. Function multiplyxy has function type
Integer -> Integer -> Integer
Using the existing definition of multiplyxy, add the function multiply2y which takes one integer
argument y and returns 2 times y.
Function multiply2y has function type
Integer -> Integer
Save as multiplyxy.hs.
Launch WinGHCi and at the Prelude prompt type
:load multiplyxy.hs (your working directory must contain multiplyxy.hs).
Test function multiplyxy with x = 2 and y = 3
Test function multiply2y with y = 3
Questions
6 A function multiplyxyz takes three integer arguments x, y, z and
returns x ∗ y ∗ z. The function is applied to arguments 2 and 3 as
shown below.
2
3 multiplyxyz
z
Figure 12.1.5.3 Application of Figure 12.1.5.1 Function composition for functions f and g, g.f
the function composition Figure 12.1.5.3 shows (g.f) applied to argument 4 two ways. Figure
(g.f) to argument 4 12.1.5.2 shows g and f being defined.
Programming tasks
1 Using a text editor such as Notepad++, enter the following Haskell
code and save as doublesquare.hs
square :: Integer -> Integer
square x = x∗x
double :: Integer -> Integer
double w = 2∗w
Launch WinGHCi then at the Prelude prompt type
:load doublesquare.hs
At the *Main prompt write function compositions using the function
composition operator '.' to
(a) double the square of 6
(b) square the result of doubling 6
(c) repeat (a) and (b) without using the function composition operator.
2 Define a function add which takes two integer arguments and returns
their sum. Incorporate this definition into doublesquare.hs and save as
addsquare.hs.
Launch WinGHCi then at the Prelude prompt type
:load addsquare.hs
At the *Main prompt write a function composition which adds the
squares of 2 and 3. Note you will not be able to use the function
composition operator for this.
3 Change the argument type from Integer to Float in both functions add
and square. Save as addsquarefloat.hs.
Launch WinGHCi then at the Prelude prompt type
:load addsquarefloat.hs
Haskell has a built-in function sqrt which takes one argument of type
Float and returns a Float.
At the *Main prompt write a function composition which calculates the
square root of the sum of the squares of 3 and 4. Note you will not be
able to use the function composition operator for this.
Questions
1 What is function composition?
If you then type q and press Tab again, only one possibility is left, sqrt, which
is automatically displayed ready to be used. To find the square root of 2, just
write at the Prelude prompt:
GHCi Prelude> sqrt 2
Prelude> sqrt 2
1.4142135623730951
Programming tasks
1 Try (a) sqrt 3 (b) div 5 2 (c) mod 5 2 (d) cos 0
(e) (^) 2 4 (f ) (^) 2 8 (g) gcd 128 32 (h) lcm 4 5
(i) take 5 "Hello World!" (j) drop 6 "Hello World!"
(k) splitAt 6 "Hello World!" (l) words "The black cat sat on the mat"
For functions with more than one argument, each argument type is separated
from the next with
WinGHCi Prelude Data.Char> max 5 3 a single arrow. For
5 example, the max
Information
Prelude Data.Char> :t max function takes two
=>:
max :: Ord a => a -> a -> a ordinal arguments, e.g. Everything before => is a class
of type Integer, and constraint. Ord a => means that
Figure 12.2.1.6 Function type of a function with
returns the smallest the type of the arguments and
two arguments return value must be members of
one.
the Ord class.
Its function type is expressed as a -> a -> a because this function is Ord contains all data types that
polymorphic, i.e. it will work with any ordinal type - Figure 12.2.1.6. allow data values to be put into
an order e.g. numerical order or
Ord is the language-defined name for an ordinal type.
alphabetical order.
Programming tasks
2 At the Prelude prompt in WinGHCi type import Data.Char.
Try (a) chr 48 (b) chr 49 (c) chr 50 (d) chr 57 (e) chr 65
(f ) chr 127 (g) ord '0' (h) ord '1' (i) ord '2' (j) ord '9'
(k) ord 'A' (l) min 45 32
Numbers
Haskell supports a great variety of number types.
• Int is a fixed-precision integer type with at least range [-229 .. 229-1].
These are bounded, machine integers, represented by 29-bit signed binary
at least. The exact range for a given implementation can be determined by
using minBound and maxBound.
• Integer is an unbounded integer type (mathematical integers, also
known as “bignums”): it can represent any value without a fractional part
without underflow or overflow. This is very useful for writing code without
caring about the range.
• Exact rational numbers using the Ratio type. Rational values are
created using n % m.
• Float and Double are floating-point Function Module
truncate
types of single and double precision, round
respectively. In module Prelude
floor
Haskell provides functions to convert between ceiling
toRational
types. Table 12.2.1.1 shows a sample of these. In module Data.Ratio
fromRational
%
Infix operators
Key point
The infix operators +, −, ∗, /, ^ are used as follows
Infix operators as functions:
Prelude> 4 + 5
Functions whose name is
9
built entirely by symbols, like
+, must be called using infix They may also be used in a similar way that a function is applied by placing
syntax: writing them the operator in brackets as follows
between the arguments, instead
of in front of them. So you Prelude> (+) 4 5
write x + y, not + x y. To use 9
a symbol function in the Care must be taken with negative numbers, e.g. −4 in the vicinity of an
normal fashion, you must use infix operator or any of the functions (+), (−), (∗), (/), (^). We must
parenthesis as follows wrap in parentheses the expression it applies to, e.g. 4 ∗ (−3) = −12 where
(+) x y.
the parentheses have been used to surround (−3).
Programming tasks
Key point
5 Try (a) (+) 9 7 (b) (∗) 3 4 (c) (/) 4 3 (d) (−) 4 3
^:
(e) (−) 4 (−3)
^ is the exponentiation
operator, e.g. 4^2 = 16 Relational operators
because it is 4 raised to the
power of 2. Op Description Op Description Op Description
We can also write this in == Equal to > Greater than < Less than
Haskell as (^) 4 2.
Greater than or Less than or
>= <= /= Not equal to
equal to equal to
Strings
If characters are grouped together then we have what is called a string. In
Haskell a string is enclosed in double quotes, e.g. "Hello World!".
If the WinHGCi interpreter is asked what type is a string, it responds WinGHCi Prelude> :t "Hello World!"
with [Char]. The square brackets, [ ]surrounding Char indicate
"Hello World!" :: [Char]
that "Hello World!" is not Char but a list of characters, each of
type Char. Lists are the most commonly used data structure in functional
programming. We study lists in greater detail in the next chapter. Information
WinGHCi Joining two strings together is ++ operator:
++ can also be used to
Prelude> "Hello " ++ "World!" called concatenation.
concatenate two lists, e.g.
"Hello World!" The ++ function is used for this, [1,2,3,4] ++ [5,6,7,8]
(\x -> x + 2)
False
Prelude> not False
This means given a value for x add 2 to it. The argument to this anonymous
True
function is x and the result returned is the value of the expression x + 2. To
apply this anonymous function we supply an actual argument e.g. 4 as follows
(\x -> x + 2) 4 WinGHCi Prelude> (\x-> x + 2) 4
A shorthand form of this is possible if the function is an operator: 6
(+2) 4
Information
Programming tasks Anonymous functions:
Anonymous functions are useful
6 Write anonymous functions and apply these to the integer 5
where there is no requirement
(a) double a given integer (b) double a given integer and add 3
to make the function visible
to other parts of the program
Conditional expressions code or to reuse the function.
We can write general conditional expressions by means of the They are often used for callback
functions in event-driven
if ... then ... else construct, e.g. the value of the following expression
programming.
if condition then x else y
16
Programming tasks
Pattern matching
8 Write a named function Pattern matching is an important aspect of Haskell and functional programming
using let to languages in general.
(a) double a given
The left-hand side of equations such as fac 0 = 1 and
integer
(b) cube a given integer fac n = n ∗ fac (n - 1) contain the patterns 0 and n, respectively.
When a function is applied these patterns are matched against argument values.
If the match succeeds, the right-hand side of the equation is evaluated and
returned as the result of the application. If it fails, the next equation is tried.
If all equations fail, an error results. All the equations that define a particular
function must appear together, one after another. The definition of the fac
function is a recursive one because fac refers to itself on the right-hand side of
the second equation.
The Haskell program, e.g. factorial.hs may then be loaded into WinGHCi
at the Prelude prompt as follows
:load factorial.hs followed by the Enter key
Programming tasks
Filter
Key principle
The filter function is a higher-order function that processes a data structure,
Filter:
The filter function is a higher- e.g. a list, in some order to produce a new data structure containing exactly
order function that processes those elements of the original data structure that match a given condition.
a data structure, e.g. a list, For example, filter can apply the even function to every element of the
in some order to produce a integer list [1,2,3,4,5,6,7,8] and return a list containing integers that
new data structure containing
possess the property of evenness WinGHCi
exactly those elements of the
original data structure that filter even [1,2,3,4,5,6,7,8] Prelude> filter even [1,2,3,4,5,6,7,8]
match a given condition. Similarly, filter can also apply the odd [2,4,6,8]
function to every element of the integer
list [1,2,3,4,5,6,7,8] and return a list containing integers that possess the
property of oddness WinGHCi
filter odd [1,2,3,4,5,6,7,8] Prelude> filter odd [1,2,3,4,5,6,7,8]
[1,3,5,7]
Programming tasks
Information
11 Try the following in WinGHCi or GHCi
Function application operator $:
(a) filter (\x -> x > 3) [0..10] When a $ is encountered, the expression on its
(b) filter (\x -> (mod x 3)== 0) [1..100] right is applied as the argument to the function
(c) :m Data.Char on its left, e.g. in 11(d) filter odd [1..100]
filter isUpper ['!'..'z'] is evaluated first, the result then becomes the
argument to map (⋆2).
(d) map (⋆2) $ filter odd [1..100]
Reduce or fold
Reduce or fold is the name of a higher-order function which reduces a list of values to a single value by repeatedly
applying a combining function to the list of
values.
r o
u lat H
In the folding or reduction process, a m e
cu
A c a
function, e.g. sum, is applied to the list 0 d
1
element by element, returning something such
2 3
as the total sum of all elements. A fold takes a
binary function, a starting value (often called
4
an accumulator), and a list to fold up. The
12.2.1.7 Folding from the left
fold reduces the entire list down to a single
accumulator value. In Haskell folding from
the left is done with foldl. For example to sum a list of integers [1,2,3,4] using foldl do the following
foldl (+) 0 [1,2,3,4]
WinGHCi Prelude> foldl (+) 0 [1,2,3,4]
foldl takes three arguments: the binary function (+), the
10
accumulator which has value 0, and the list [1,2,3,4].
Binary means the function (+) takes two
operands. The first of these is the accumulator.
The second operand starts with the first element
of the list (the head). (+) returns their sum and T
1 2 a
this becomes the new accumulator. foldl or
il u lat
3 4 c um
then applies (+) to the new accumulator Ac
value and the second element in the list to 0
produce a new value of the accumulator
and so on until all the integers in the list are 12.2.1.7 Folding from the right
summed.
To start the traversal from the opposite end of the list, the tail, we use foldr as follows
3^2 = 9
2^9 = 512
1^512 = 1
WinGHCi With foldl (^) 2 [1,2,3] in WinGHCi, the accumulator is 2. The list is
folded from the left as follows
Prelude> foldl (^) 2 [1,2,3]
accumulator ^ list value
64
2^3 = 8
8^2 = 64
64^1 = 64
Programming tasks
Questions
1 In a functional programming language, a higher-order function map takes two arguments, a function f,
and a list and applies the function f to each element of the list returning the results in a new list.
(a) A function named square is defined below
square x = x ∗ x
The result of making the function call square 2 is 4.
Calculate the result of making the function call
Function call Result
map square [1,2,3,4]
(b) Another function defined as (++ "!") takes a single string argument and returns a string which is the
concatenation of the string argument with !
++ is the concatenation operator. For example,
(++ "!") "Wow" returns "Wow!"
Calculate the result of making the function call
Function call Result
map (++ "!") ["SLAP", "BAM",
"WALLOP"]
Questions
2 In a functional programming language, a higher-order function filter takes two arguments, a function f
that returns a Boolean value, and a list and applies the function f to each element of the list returning the
results in a new list.
A function defined as (>4) takes a single integer argument and returns true or false depending upon
whether the integer is greater than 4 or not.
Calculate the result of making the function call
Function call Result
filter (>4) [5,3,6,2,-1]
head(list):
head :: [a] -> a
returns a new list containing all but the first element of the original list, list.
Key principle
e.g. for list [Emma, John, Fred, Janet]
tail(list):
tail([Emma, John, Fred, Janet]) returns [John, Fred, Janet]
The operation tail (list)
returns a list formed by In Haskell, a function that returns the tail of a list, h : t is defined as follows
removing the first element of
the original list.
tail :: [a] -> [a] WinGHCi prelude> tail [1,2,3,4]
tail(_: t) = t [2,3,4]
where t is the tail.
Questions Information
1 What result is returned by the following functions applied to the list Convention for representing
lists:
Towns where
In Haskell it is an informal
Towns = [Swindon, Aylesbury, Banbury, Stevenage, Slough] convention to write variables
(a) empty(Towns) (b) head(Towns) (c) tail(Towns) over lists in the form xs, ys
(d) head(tail(Towns)) (pronounced ‘exes’, ‘whyes’) and
(e) tail(tail(tail(tail(Towns)))) so on, with variables x, y, ... for list
elements. For example, x : xs is
a list with head element x and
Return length of list tail xs.
h has a length of 1 to which we add length of the list t. The list t is the original
list minus its head.
Programming tasks
1 The function max is built-in to Haskell. It takes two arguments of an
ordinal type and returns the larger of the two. Use WinGHCi and enter
the following at the Prelude prompt: WinGHCi Prelude> max 9 3
Prelude> max 9 3
9
Now try (a) max 46 57 (b) max 46 43
Ord is Haskell’s ordinal data type, (Ord a) => specifies that the values
of a must belong to an ordinal type.
Load this program into WinGHCi using the command
:load maxy.hs
Try (a) maxy [] (b) maxy [4]
(c) maxy [34,23,7,67,31]
Arithmetic and Logic Unit (ALU) 270, 276 BCNF informally 508
Arithmetic expression 569 B <condition> <label> 282
ARM 283 Beacon frame 371
ARM Cortex-A7 quad-core processor 310 BEQ <label> 297
ARM Cortex processor 289 Bespoke software 217
ARM processor 282 BGT<label> 297
ARM processor architecture 284 Big Data 333, 543
ARM μVision 289 Bigtable 559
ARM μVision V 5.17 Debugger in single-step mode 289 Binary 24
ARPANET 447 Binary addition with Sum and Carry 240
ASC 522 Binary data 325
ASCII 96, 316 Binary digit 36
ASMTutor 276, 288 Binary digits 344
ASN 398 Binary function 589
Assembler 218, 222, 225 Binary, the language of the machine 106
Assembly language 220 Binary to hexadecimal conversion 31
Associative law 249 Binding port and socket 424, 427
Asymmetric encryption 405 BIOS 260
Asymmetry of power 332, 333 Bit 36
Asynchronous data transmission 346, 347 Bitcoin 341
Asynchronous serial data transmission 347 Bit depth of bitmapped image 143
Asynchronous serial data transmission frame 350 Bit depth of sampled sound 166
Atomic facts 556, 557 Bit is the fundamental unit of information 36
Atomicity 500 Bitmap 141, 319
Atomic transaction 541 Bitmapped graphics 139
Attribute 476, 487 Bitmapped graphics uses 163
Attribute domain 498 Bitmapped image 141
Audacity 116 Bitmap resolution in number of pixels per inch (ppi) 145
Authentication 376, 410 Bitmap size in pixels 143
Average no of clock cycles per instruction 308 Bit patterns 106, 221
Bit rate 352, 354
B
Bits 325
Backing up data 329 BitTorrent 366
Bandwidth 168, 353 Bitwise AND 445
Barcode reader 316 B <label> 282
Barcode scanning versus RFID scanning 322 Block-oriented storage device 324
Barcode symbols 316 BLT <label 297
Base 2 24 Blue-ray disc 326
Base 10 24 BNE <label> 297
Base 16 25 BOOLEAN 530
Basic Service Set (BSS) 369 Boolean algebra 247
Baud rate 351, 354 Boolean expression equivalent of a logic gate circuit 238
Bayer filter 318 Boolean function 233
D Decryption 184
Dedicated registers 270, 274
Dark Web 341
Dedicated server 364
Data 123
DEFAULT 528
Data anomaly on deletion 515
Degree of a relationship 481
Data anomaly on insert 515
DELETE 463, 519, 525
Data at rest 543, 550
De Morgan’s laws 252
Database design 498
DESC 522
Database inconsistency 541
Destination hardware address 423
Database normalisation techniques 498
Destination IP address 389
Database transaction 533
Destination register 281
Data bus 258, 259
Determinancy diagram 500
Data compression 177
Determinant 500
Data Constraints 475
DHCP 451, 458
Data Definition Language (DDL) 527
Dictionary-based compression methods 180
Data exhaust 333
Difference between a router and a switch 388
Datagrams 422
Diffie-Hellman key exchange 408, 410
Data inconsistency on update 515
Digit 2
Data in motion 543, 550
Digital camera 317
Data integrity 533
Digital certificate 409, 410, 418
Data link layer 390
Digital certificate authority 410
Data link layer address 390
Digital data 125
Data Manipulation Language (DML) 527
Digital ethics 332
Data model 474
Digital image file 109
Data processing instructions 281
Digital representation of sound 164
Data Protection Act 332
Digital signal 127, 326
Data Requirements 474
Digital Signal Processor 266
Dataset 543
Digital signature 410, 411, 412
Data transfer instructions 281
Digital single-lens reflex cameras 139, 317
Data transfer time 325
Digital Subscriber Line (DSL) 368
Data type 562
Digital to analogue converter (DAC) 134
Data types 530, 531
Digital Versatile Disc (DVD) 323
DATE 530
Direct access to registers 224
Datum 35, 123
Direct addressing 288
DCIM (Digital Camera IMages) 318
Direct Memory Access (DMA) 264
DDL 527, 531, 532
Disadvantages of programming in machine code
Decentralized P2P network 367
and assembly language compared with HLL
Decimal 24
programming 224
DECIMAL 530
Disadvantages of the vector graphic approach 163
Decimal to binary conversion 26, 27
Discrete 126
Decimal to hexadecimal conversion 28
Discrete data 124
Decoder 180
Discrete logarithm problem 407, 408
Decoding 180
Disk block 324
Decrypting with cipher wheel 187
Hardware 216 I
Hardware interface 264
I2C 346
Hardware maintenance 472
IANA 427, 448, 449
Hardware random number generators 202
Identifying a TCP connection 424
Harmonic 167
Idle state 349
Harvard architecture 265, 268
Image quality 154
Hash function 412
Image resolution 144
Haskell 581
Image sensing and acquisition 139
HDFS 546, 554
Immoral action 330
head(list) 593
Immutable data 556
hertz (Hz) 353
Immutable data structures 555
Hexadecimal 25
IMP 447
Hexadecimal as shorthand for binary 31
Imperative high-level language (HLL) 223
Hexadecimal to binary conversion 30
Imperative programming language 593
Higher-order function 555, 587
Improving processor performance 309
High-level language 222
IN 522
High-level programming language 222
Inconsistent database 533
High-level programming language classification 222
Individual (moral) issues and opportunities 330
Horizontal scaling 547
Inferred data 339
Host 384, 437
Infix operators 584
Host aliasing 395
Infix operators as functions 584
Host computer 437
Information 34, 333
Hostname 384
Information carrier 34
Hosts in networks 364
Information = data + meaning 35
How domain names are organised 394
Information-theoretically secure 209
How routing is achieved across the Internet 391
Information types 35
HTML 432
Inkscape 159
HTML5 WebSocket 469
Input and output devices 316
HTTP 432
INSERT 463, 519
HTTP client 426
INSERT INTO 526, 531
HTTP GET 392, 426, 433
Instructional information 35
HTTP GET / 434
Instruction count 309
HTTP GET / request 434
Instruction Register (IR) 275
HTTP methods 461, 463
Instruction set 228, 283
HTTPS 410, 434
Instruction set is processor specific 283
HTTP server 426, 461
Instructions per second 273
Human fault tolerance 556
INTEGER 530
Hyperscale computing 336
Integer numbers 4
Hypertext Markup Language 432
Integer overflow 93
Hypertext Transfer Protocol 432
Integrity 521
Hypertext Transfer Protocol over Secure Sockets 434
Interface 437
Hypertext Transfer Protocol Secure 434
Interface Message Processor (IMP) 385
Hypertransport bus 312
Interference 372
P Photosensor 317
Physical bus network 358
P2P 362
Physical dimensions of printed images 145
Packet 385
Physical phenomenon 35
Packet based serial bus 312
PiNet 471
Packet filtering 400
Single licence - Abingdon School 608
Software and their algorithms embed moral Stop bit 347, 348, 350
and cultural values 338 Storage media and typical applications to which
Software can produce great good but with it comes they can be put 329
Solid-state disk (SSD) 323, 326 Stored program concept 267, 268
Sources of interrupt 305 Structure of a machine code instruction 281, 283, 284