We’ll cover the following:

1. A simple test application
2. Building docker images for each architecture
3. Wrapping images in a multi-architecture manifest
4. A small gotcha
5. Testing our images

The Application

Before we can begin building our containers, we need an application to package. We’ll use this small Go program. When executed, it displays the operating system and CPU architecture it. In this example, we expect to see arm (Raspberry Pi) or amd64 (MacBook). In both cases we’ll be running on Linux.

$ go run main.go
Hello, 世界!
GOOS: darwin
GOARCH amd64

Building Docker Containers

Although we promise “multi-architecture” builds, we start by building two architecture specific images.

Create the following Dockerfile. I’ve named it Dockerfile-amd64 as it builds an amd64 version of our application (note the GOARCH=amd64).

Build and tag the image.

$ docker build -f Dockerfile-amd64 -t billglover/hello-arch:amd64 .

Run the container to test things are working as expected.

$ docker run billglover/hello-arch:amd64
Hello, 世界!
GOOS: linux
GOARCH amd64

Next we want to build the arm version of our application (note the GOARCH=arm). I’ve named this file Dockerfile-arm.

Build and tag the image.

$ docker build -f Dockerfile-arm -t billglover/hello-arch:arm .

Multi-Architecture Manifest

We need to push these images to the repository before creating the combined manifest. It isn’t clear why we need to do this, but I ran into issues if I didn’t.

$ docker push billglover/hello-arch:amd64 $ docker push billglover/hello-arch:arm

Create the multi-architecture manifest.

$ docker manifest create billglover/hello-arch billglover/hello-arch:amd64 billglover/hello-arch:arm Created manifest list docker.io/billglover/hello-arch:latest

The syntax of this command may not be obvious. The first argument, billglover/hello-arch is the name of our multi-architecture manifest. The remaining arguments are the images we want to include.

In theory this should be all we need to do. If we were using architecture specific base images then everything would be fine. We are using the scratch image which is architecture agnostic. Take a look inside the multi-architecture image to see why this is a problem.

$ docker manifest inspect billglover/hello-arch

The architecture of both images inside our manifest is amd64. We need to fix this.

$ docker manifest annotate --arch arm billglover/hello-arch billglover/hello-arch:arm

This corrects the architecture in our manifest list.

Push the multi-architecture manifest.

$ docker manifest push billglover/hello-arch

Testing

Running the multi-architecture image locally (amd64):

$ docker run billglover/hello-arch
Hello, 世界!
GOOS: linux
GOARCH amd64

Running the multi-architecture image on a Raspberry Pi (arm):

pi@node-01:~ $ docker run billglover/hello-arch
Hello, 世界!
GOOS: linux
GOARCH arm

It is possible you will never need to work with more than one CPU architecture. If you do, it is reassuring to know that Docker has you covered. The solution is a manifest list, containing two or more architecture specific images. Docker will select the correct image at runtime.

Originally published at billglover.me on October 30, 2018.

Our Application

The application we’ll be working through is simple. It returns the IP address of the user, nothing more.

Requirement #1: As someone who wants to allow others access to my home network, I need to know my external IP address.

Pre-Requisites

If you want to deploy this application yourself, I’m going to make a couple of assumptions:

1. You have Go installed and know how to build/run a simple application (start here).
2. You have a Google Cloud account and created a project that allows you to deploy to App Engine (start here).
3. You have the latest version of the Google Cloud SDK installed, along with the beta components (start here).

If you don’t have these set-up, don’t worry. You should still be able to follow along. The ease of publishing to Google App Engine will encourage you to give it a try.

Determining the IP address

Before we can write our application, I want to touch on how we can identify the user’s IP address. The Go Standard Library provides the RemoteAddr property on http.Request.

You might think we could use this property to determine the IP address of the user. Often you can, but if your application sits behind a load balancer or reverse proxy this doesn’t work. When we deploy on App Engine, this is exactly the scenario we have. The App Engine Architecture shows that our application will sit behind the Google Load Balancer. As a result the RemoteAddr property is set to the address of the load balancer and not that of the requestor.

This is set-up is so common that load balancers often set a header to get round this. The Mozilla Developer Network documentation describes this header header as follows.

“The X-Forwarded-For (XFF) header is a de-facto standard header for identifying the originating IP address of a client connecting to a web server through an HTTP proxy or a load balancer. When traffic is intercepted between clients and servers, server access logs contain the IP address of the proxy or load balancer only. To see the original IP address of the client, the X-Forwarded-For request header is used.”

X-Forwarded-For: 109.148.211.158

We will be using the XFF header to identify the IP address of requestor and return it in the response.

Note: If the request passes through more than one load balancer, the header may contain several addresses. In this example, I am going to assume that this is not the case and that we only have a single IP address as shown above.

The Code

This is our main.go file.

We can run the application on our local machine. But we’ll need to set the PORT environment variable before execution.

Test the service with cURL in a separate terminal window.

curl -i -H 'X-Forwarded-For: 127.0.0.1' http://localhost:8080/ip

HTTP/1.1 200 OK
Date: Mon, 22 Oct 2018 13:34:15 GMT
Content-Length: 10
Content-Type: text/plain; charset=utf-8

127.0.0.1

So far, this is a standard Go application that compiles to a single binary. We’ve done nothing to support the App Engine deployment target. In the previous edition of App Engine we would have had to do the following.

• Import google.golang.org/appengine
• Register our handlers in the init() method
• Leave out the main() function

Without the main() function, this would not compile to a standalone application. The code would be App Engine specific.

There were further changes required if your application needed to make outbound requests. As we don’t make outbound requests from our application, I’m going to skip over these. An example of a project built for earlier App Engine releases is available here: github.com/billglover/ae_greetings.

Deployment

The latest release of App Engine gets rid of these platform specifics. We are now able to write standard Go applications and deploy them to App Engine.

Create a file called app.yaml in the same folder as main.go.

runtime: go111

You can now deploy your App with the following command.

gcloud app deploy

If you want to avoid having to respond to the prompt you can use the --quiet flag.

gcloud app deploy --quiet

When the deployment is complete, the client will display the URL to your application. You can open it in your default browser with the following command.

gcloud app browse

If everything worked, you should see a browser tab open showing your external IP address.

That is all we need to do to deploy a standard Application to App Engine. There are further configuration options to control the behaviour of your application. None of these are essential, but I chose to limit the number of instances I run. Documentation for the full set of options is available here: app.yaml Configuration File

runtime: go111
automatic_scaling:
min_instances: 0
max_instances: 1

In this post we have covered the following:

• Created a Go application that displays the IP address of the requestor
• Ran the application on our local machine and tested it with cURL
• Deployed the application on App Engine
• Tested the public deployment by making a request in our browser

This example shows how to deploy a standard Go application to Google App Engine. It shows that this is possible without writing specific App Engine code. This is a significant improvement and makes the platform an attractive deployment option for Go services.

Full code is available on GitHub: github.com/billglover/localname. Issues and Pull Requests are welcome.

Originally published at billglover.me on October 22, 2018.

TLDR: The cost of a function call? I still don’t know. The best I can offer is that it depends, well that and an exploration of Go assembler.

In this post, I take you through the analysis I did and highlight what I learned along the way.

A simple program

Rather than use a complex algorithm, I sought to use the simplest possible program that demonstrated the behaviour I wanted to investigate. This Go program takes an int, in this case 1000, and increments it by 1 inside a loop that executes 1000 iterations. The result, 2000, is printed to the screen.

I included the loop to slow down the function and allow me to measure benchmarks. Without the loop, benchmark results are in the low single digit nanoseconds which fall within the margin of error of simple benchmarks.

$ go test -run=X -bench=.
goos: darwin 
oarch: amd64
pkg: github.com/billglover/recursion/inline
BenchmarkInc-8		5000000			331 ns/op
PASS
ok	github.com/billglover/recursion/inline	1.971s

Adding function calls

Now that I had a baseline, I wanted to see how the addition of a single function call would change the benchmark. I modified the inc(n int) int function as show below.

Instead of increment it the value of n directly, we do so via an additional function call to add(). With the additional function call, I expected to see an increase in execution time. The benchmark comparison between the approaches can be seen below.

$ go test -run=X -bench=. ./... | grep -i bench BenchmarkIncFunction-8		5000000			289 ns/op
BenchmarkIncInline-8		5000000			289 ns/op

I ran the benchmark again several more times and results were consistent to within a couple of nanoseconds. Given the differences in recursive function performance I’d been seeing, I was expecting the additional function call to result in a noticeable change in performance. In these examples it appeared to make no difference at all. It was time to dig deeper.

Behind the scenes

I had a suspicion that the compiler was performing some optimisation on my example code and the resulting code was similar if not identical between the two examples. Luckily, the Go toolchain provides an easy way to view the output of the compiler.

$ go tool compile -S inline.go

Note: this results in an intermediary assembler and is not the code that actually ends up running on the CPU. I ignored this technicality as I continued to explore.

Looking at the assembly output for the first example, it was fairly easy to trace through the function execution. Helpfully, the assembler output includes references back to the file and line number for the original Go sourcecode.

The FUNCDATA and PCDATA instructions relate to garbage collection and are inserted by the compiler. I’ve removed these from the following (and future) listings to aid readability. The TEXT directives define the function names and indicate the stack allocation required for the function to execute. This one line took the most time to analyse. I’ve broken out an explanation for each of the components below.

00000	TEXT	"".inc(SB), NOSPLIT, $0-16

• "" is a placeholder that will be filled in later by the linker.
• incInline is a symbol used to reference the function location in memory.
• NOSPLIT I don’t understand this fully, but I think this means that some stack management code can be left out.
• $0 indicates that no additional space on the stack is required for this function.
• $16 indicates that 16-bytes are required for the function parameters and return values (in this case; one 64-bit integer parameter, and one 64-bit integer return value).

A simplified view of the assembly language version of our first example is shown below.

A quick glance at the documentation (X86 Architecture) for Intel CPUs tells us a little more about the registers being used here.

• AX: Accumulator register (AX) – used in arithmetic operations.
• CX: Counter register (CX) – used in shift/rotate instructions and loops.

The Go documentation (A Quick Guide to Go’s Assembler) defines the following psuedo-registers. These aren’t hardware registers but are virtual registers maintained by the Go tool chain. I’ll treat them as if they were hardware registers for this investigation.

• SB: Static base pointer – used to reference global symbols.
• SP: Stack pointer – used to reference the top of stack.

This shows that register CX is being used to hold the value of our loop variable i. And register AX is being used to hold the value of our local variable n.

Now that I understood the inline version of our programme, I took a look at the output of the compiler for our second example. Again, I’ve removed the FUNCDATA and PCDATA instructions and added comments to each line.

In this listing we can clearly see the two function definitions. I’ll leave you to walk through the implementation of the add(n int) int function. Far more interesting is that the listing for the inc(n int) int is identical to the listing in our first example. Although we have defined a new function, this function is never called. The end result is that the code path followed in the two examples is identical. This explains the benchmark comparisons seen earlier.

The compiler has determined that it can move the function add() inline and save on a function call. This is known as “inlining” and is a common compiler trick used to improve performance at the expense of binary size (see Wikipedia). This explains why I was seeing identical benchmarks for both approaches, but not why my recursive functions were seeing slower performance than inline counterparts.

It turns out that the compiler doesn’t move functionality like this inline for all cases. One example where it is unable to do so is recursion as it doesn’t know at compile time how many times the recursive function is going to be called.

I was now convinced that these function calls would demonstrate a performance overhead (otherwise why would they be optimised out), but I was no closer to being able to measure what that overhead was. I needed a way to view the cost of function calls without the compiler optimising them away.

After some digging, I discovered a little known Go pragma directive to disable inlining for particular functions. This should never be used in normal code, but is very useful here for demonstrating the cost of function calls.

With the addition of the noinline pragma directive, I re-ran the benchmark.

$ go test -run=X -bench=. -benchmem ./... | grep -i bench BenchmarkIncFunction-8		1000000			2289 ns/op
BenchmarkIncInline-8		5000000			286 ns/op

This now shows the inline call is ten times faster than the approach using function calls. But, before you panic and start avoiding function calls altogether, remember I’m making 1,000 function calls during each benchmark operation. The overhead of any individual function call is tiny.

Now that I was able to reproduce the overhead associated with function calls, I went back to the assembly to see what was going on behind the scenes.

The good news is that the inc() function now makes a call to add() on line 47. However I was not expecting the additional complexity before and after the main function body. There appears to be a lot more going on than a simple CALL to add(). Much of this additional complexity comes from managing the values on the stack and it was becoming increasingly difficult to trace how values were moving between registers as the functions were called. I turned to pen and paper to work this through.

If you want to follow this through yourself, grab a large mug of your favourite brew and several sheets of paper, it’ll take a while.

I traced through the execution of the programme and learned the following:

• Values are passed to functions via the stack.
• Values are returned from functions via the stack.
• Calling functions are responsible for reserving the required stack space.
• Function calls increase the stack size by one to accommodate the return address.
• Functions make calls to the runtime to grow the stack as necessary.

I now understood a little more about why function calls incur a cost. Allocating stack space and passing parameter and return values around all costs CPU cycles. But I still wasn’t able to articulate how big that cost was.

So how expensive is a function call

Comparing our two approaches through benchmarking we can see that our solution involving a function call took 2289 ns/op whereas our inline solution took only 286 ns/op. You could make the argument that adding the function call takes an additional 2003 ns/op. For 1,000 function calls in each benchmarking operation, this equates to 2 ns/function call.

An alternate way of looking at it would be to look at the increase in the number of instructions executed to make our function call. Our inline method requires 10 lines of assembler. Our method involving a function call requires the execution of an additional 20 instructions. Figuring out how long each instruction takes to execute is non-trivial and so I decided to stop digging further. If you have any thoughts on how I might be able to approximate this I’d love to hear from you.

In conclusion

Whilst I can’t definitively articulate the cost of a function call in Go, I do have a better understanding of why it incurs a cost. To some, this may be obvious, but the journey above thought me a lot more about the Go, the associated toolchain, and general programme execution.

I’m still of the opinion that the best way to determine the difference between two approaches is to benchmark them. If you find yourself questioning the results of your benchmarks, it’s time to dig further. The Go toolchain provides a number of options for further investigation.

Originally published at billglover.me on September 17, 2018.

“Nees was fascinated by the relationship between order and disorder in picture composition. To create this work, he introduced random variables into the computer program, causing orderly squares to descend into a heap.”

I was toying with the idea of recreating the image when I came across the Go Graphics package by Michael Fogleman (GitHub, GoDoc), a beautifully simple way to get started with drawing graphics in Go. The basics involve creating a context, drawing various shapes, stroking (or filling) them and then rendering the result as a PNG image.

package main import "github.com/fogleman/gg" func main() { ctx := gg.NewContext(640, 640) ctx.DrawRectangle(270, 270, 100, 100) ctx.SetRGBA(0, 0, 0, 1) ctx.Stroke() ctx.SavePNG("out.png") }

This code snippet creates a 640 by 640 image and draws a square of 100 by 100px in the center. To generate the image, I needed a few loops, some randomness and the ability to offset each square from it’s regular position. The algorithm seemed fairly straightforward:

• Use a for loop to iterate over the columns
• Use a for loop to iterate over the rows
• Offset each square from its mid-point based on a function of current row
• Rotate each square based on a function of the current row

Tweaking the dimensions and weighting the offset and rotation gives us something remarkably similar to Georg’s original.

With the ability to produce images I quickly found myself spending hours tweaking parameters, padding, colours, etc. I settled on using an exponentially increasing random offset to generate the images below.

I’d love to try plotting some of these on paper (using a real plotter of course) but for now I’ll have to settle for generating wallpapers instead. A zip folder containing a collection of phone wallpapers is available for download below. Full access to the source code is available here: github.com/billglover/aitaoc

Download: iPhone, iPad

Originally published at billglover.me on August 25, 2018.

“What is the best way for an AI to dream?”

– Luba Elliott, #GDGLondon 2017

Ask someone to explain why a reasonably complex neural network arrived at a particular decision and you will likely be met with a shrug of the shoulders. If you are lucky, you will be met with a generic explanation of how neural networks work. Rarely will you hear anything specific to the outcome in question. The same could be said about dreams. There is the theory, but nothing that allows you to determine why an individual dream took the turn it did.

Mathematics can explain the theory behind the artificial intelligence. Engineering advances have given us the compute resources necessary to consume vast troves of data to train these models. As a result, the individual decisions made by machines are now a function of data sets so vast that we struggle to comprehend the information they contain. It is practically impossible to explain why an individual decision was made.

This very challenge has led to some interesting debate around the European Union regulations on algorithmic decision-making and a “right to explanation” where a user can ask for an explanation of an algorithmic decision that was made about them.

What struck me about Luba’s talk was that rather than trying to understand and explain these systems, there was a community of people dedicated to exploring their creativity. But is there was more than creativity to their work. Could art hold the key to explaining these machines? Much of the creativity we see in their work results from peering inside these models, exploring them as they develop, from seeing them dream.

Luba’s question got me thinking: In addition to looking for mathematical explanations for the decisions our models make should we also be looking into the way they dream? Even if a deterministic outcome eludes us, we arrive at some incredibly intriguing art and with it an insight into the models that we have not previously had. I wanted to share a selection of artists mentioned during the talk, not because they are the first to explore the field of Creative AI, but because their work represents an usual look inside the machines that many of us have come to depend on.

Anna Ridler

Fall of the House of Usher from Anna Ridler on Vimeo.

Fall of the House of Usher is a 12-minute animation based on the short story by Edgar Allan Poe, where each still is generated by artificial intelligence. This is done by using a neural net (pix2pix) trained on the artist’s ink drawings made of stills from the 1929 version of the film. Each still shown in the animation is not merely a filter that is applied to an existing image, but an entirely new image by a neural net.

• Portfolio: anaridler.com
• Vimeo: vimeo.com
• Twitter: @annaridler

Gene Kogan

J-train style transfer from Gene Kogan on Vimeo.

Gene Kogan is an artist and a programmer who is interested in generative systems, computer science, and software for creativity and self-expression.

What I particularly like about Gene’s work is how raw it fees. This rawness only serves to highlight the non-deterministic nature of the artificial intelligence algorithms. Gene’s more recent work starts to explore the idea of what it might mean for an AI to dream.

For those interested in exploring this field Gene has been working on a collection of free educational resources, ml4a.

Memo Akten

Learning to see: Hello World! [WIP R&D 1] from Memo Akten on Vimeo.

Artist working with computation as a medium, exploring collisions between nature, science, technology, ethics, ritual, tradition and religion.

I can’t remember what it was like to see for the first time. Sight is something the majority of us take for granted. This video does an incredible job of portraying what it might be like to experience those first few moments of sight. Incredibly, this is not an artists interpretation, this is an algorithm figuring out how to discern objects within its field of vision.

• Portfolio: memo.tv
• Vimeo: memotv
• Twitter: @memotv

Lawrence Lek

Geomancer (2017) — Trailer from Lawrence Lek on Vimeo.

Geomancer is a CGI film by Lawrence Lek about the creative awakening of artificial intelligence. Feauturing a neural network-generated dream sequence, and a synthesised vocal soundtrack, Geomancer explores the aesthetics of post-human consciousness.

• Portfolio: lawrencelek.com
• Vimeo: lek
• Twitter: @prosthetic

Luba Elliot

Luba Elliott is a curator, artist and researcher specialising in artificial intelligence in the creative industries. She is currently working to educate and engage the broader public about the latest developments in creative AI through monthly meetups, talks and tech demonstrations.

I’d like to thank Luba for taking the time to speaking at #GDGLondon and for introducing me to a new dimension to the field of Artificial Intelligence. If you are interested further, take a look at her overview of the Creative AI Landscape.

Originally published at billglover.me on November 13, 2017.

The BBC micro:bit

Whilst many of you will have heard of the Raspberry Pi and the Arduino, there will be few, particularly those outside the UK, who have heard of the BBC micro:bit, a new development board targetted at the next generation of creative thinkers.

In an effort to re-kindle the educational spirit behind the original BBC Micro from the early 1980s, the BBC teamed up with 29 industry partners to launch the BBC micro:bit. The device itself is tiny. It is over 18 times faster and 70 times smaller than it’s predecessor and includes an array of sensors (light, compass, accelerometer, buttons, temperature), a 5x5 array of LEDs, a Bluetooth radio, and a range of general purpose I/O pins.

The target market for this device is clear. Every year 7 (12 years old) student in the UK is being issued with one of these devices for free. Teachers are being provided with training material, tutorials that can be followed in class and a platform on which to share the resulting projects.

During the brief time that I have had to spend with the micro:bit, I have been amazed at some of the other things that have been done to make the micro:bit classroom friendly. I encourage you to explore for yourselves, but here are a few of my favourites;

• 5 programming languages from the Block Editor through to JavaScript, Python and C ensure that those of all levels can be creative from the start.
• Key I/O pins are exposed with pads large enough that crocodile clips (or even fingers) can be used to attach external accessories.
• No software installation required at all. The development environment is browser based (and yes, it works offline). You program the device by copying a file to a USB drive.
• Privacy is considered from the outset with no sign-up at all required for students. To share creations on the gallery, it is the teachers that register on the site.
• The gallery of projects encourages sharing of creative ideas and provides as inspiration for those just getting started.

If you are looking for more advanced micro controllers, there are plenty out there but I have yet to come across anything that comes close to the micro:bit in terms of accessibility at the entry level. And, for those who have long since graduated from Year 7, the devices are available for purchase from a number of approved retailers. I picked up mine from Pimoroni for £13.

The meet-up

With three micro:bits in my possession, I figured I’d ask and see if anyone else was interested in exploring the devices after work one evening. An experienced colleague got the ball rolling by saying he’d support me in hosting an event and pushed me to schedule a date. Our meet-up was arranged.

The level of interest surprised me. Despite it being the height of the summer holiday season, we had 7 people turn up and many more express an interest. What impressed me most though was the level of interest from those with no technical background. All who turned up had little to no coding or technical experience. This was an event driven purely by interest and a desire to learn.

• a desire to learn more about technology and the industry we work in
• a desire to keep ahead of younger relatives who are already involved
• a curiosity that arose after seeing my request for interest

I gave a quick background to the micro:bit, not much more than you have just read but probably more than was needed, and then we let people get stuck in. Some stayed to the end, others didn’t, but all bar one (more on that later) managed to get a little project running on the micro:bit with the device responding in some way to user input.

Ideas can be challenging

With all the sensors available on the micro:bit the creative possibilities are incredible. I hadn’t anticipated that coming up with ideas for starter projects would be challenging, but it was.

• No one wanted to copy someone else's idea or just do something simple
• Some wanted to follow a tutorial, others just wanted to explore
• Balancing the complexity of an idea with something that is accessible as a starter project requires a little experience

I was hesitant to suggest ideas as I really didn’t want to be prescriptive or risk influencing what people would come up with. But in this scenario, starting with the basics seemed to help build confidence. Displaying an image on screen, changing that image in response to user input was a quick way to get people seeing the micro:bit respond to code that they had written.

Note for next time: I noticed that people wanted to work individually rather than in groups. But it was only when people spoke to each other that ideas for projects started to form. Next time I might try spending 10 minutes as a group brainstorming a list of ideas before we start on coding. This would allow groups to form around popular ideas or individuals to pick up ideas and run with them without me worrying that I was being to prescriptive about what people should make.

Problems can always be broken down

Once we had ideas for people to work on, I noticed that a common challenge was breaking those ideas down into small steps. This challenges was universal with all of us struggling with it at some point. But it is this skill that is invaluable not just for developers and engineers, but for project managers and business analysts as well.

Take one of the challenges we faced: How do I make an image flash on screen a number of times?

When you discover there is no block like the one shown on the left (the image is a mock-up), the temptation is to stop and adapt your idea. In some cases the compromise is to go with a static image and abandon the notion of flashing. But challenging people to describe how they would flash an image yields thinking that is closer to the approach on the right which, in this case, is something that you can implement. Re-thinking concepts that we are familiar with, even something as basic as flashing an image, isn’t something that comes naturally to many. It’s like being asked to define a word you use regularly and finding yourself struggling to offer up a defintion. We’ve all been there.

In observing people go through this process of breaking down problems, I found it fascinating that people were starting to make connections between coding for the micro:bit and tasks they considered their day job. A business analyst responsible for mapping out a process is performing very similar tasks to a developer trying to piece together a program. The context may be different, but in both cases the goals are similar; to express a solution to a problem in a concise and logical way.

Note for next time: Challenging people to break down problems is one of the things I found myself doing frequently throughout the evening. In all cases, getting people describing the steps they would go through to solve a problem gave people the insight required to implement it. Given how crucial this skill is, it might be an idea at future events to get people to describe their proposed implementations verbally, either to the group or to each other, before then jumping in to implement them on the device.

Physical devices over simulators

Many of the languages for the micro:bit come with a simulator (Python and C being the exceptions). Whilst the simulator is fun and makes for a quick way to test ideas, the most satisfying moments were when people could see their creations running on the physical devices. There was joy, there were smiles. It was very much a moment of pride. It may seem obvious to anyone who has enjoyed that moment of delight the first time their code runs on a device, but simulators are no substitute for a physical device.

I’m pleased to say that by and large the experience of the simulator was good and that code that ran in the simulator also ran on devices. We did have one case where the compilation of code seemed to result in an empty project file. This was a shame as we were unable to get a fairly simple first project running on the device. What was more frustrating is that I wasn’t able to explain why things weren’t working.

Note for next time: The simplicity of the micro:bit is wonderful, but when things don’t go as expected it can be a little challenging to know where to turn. We had the added complexity of the fact that restrictions on corporate laptops prevent anyone from copying code to the micro:bit devices. This meant we resorted to emailing copies of compiled code for me to load onto the devices. Simplifying this process for next time would reduce the chance of things going wrong. Specifically:

• Encourage people to name their projects (the default names are repetitive).
• Get access to USB ports that are writeable if at all possible. If not, create a shared folder that people can use to simulate loading the code onto their devices.
• Encourage people to remove old files from the downloads folder to avoid confusion between project versions.
• Encourage people to switch between code view and block view to see a different take on the language used to describe their project.

Next-time

Initially I was worried that no one would turn up to the event. When people did turn up, I was worried they’d be bored and feel like they’d rather be elsewhere. I learned that these worries were unfounded.

The micro:bit isn’t just for children — we had great fun. And, whilst I love playing with the Raspberry Pi and the Arduino, I could never have hosted a room of non-techies with those platforms and had as much fun as we did with the micro:bit on Tuesday. I’d recommend this to anyone. If you have an interest, or even just a curiosity, open it up to others.

I’m personally delighted that we were asked to run the event again, both by attendees and those who were unable to make it. This is the best indication that the event was a success. The date is already in the diary and, now that the first one is out of the way, I’m looking forward to it.

Before running any code examples such as this, I attempt to work out what the programme might do. In this case, I expected to see the following

a len=5 cap=5 [0 0 0 0 0]
b len=0 cap=5 []
c len=2 cap=5 [0 0]
d len=3 cap=5 [0 0 0]

If you run this code you’ll see that this isn’t quite what happens and that the capacity of slice ‘d’ is actually 3 and not 5.

a len=5 cap=5 [0 0 0 0 0]
b len=0 cap=5 []
c len=2 cap=5 [0 0]
d len=3 cap=3 [0 0 0]

I tried a couple of similar examples but couldn’t explain the reduced capacity of slice ‘d’. My understanding of slices is that they are a data structure that describes a subset of a fixed capacity array. Based on this understanding, I interpret this code snippet as follows:

• ‘d’ is a slice of existing slice ‘c’
• ‘c’ is a slice of existing slice ‘b’
• ‘b’ is a slice of an underlying array that has a fixed capacity for 5 integers

As ‘b’, ‘c’, and subsequently ‘d’ all represent subsets of the same underlying fixed capacity array I couldn’t understand two things about the result:

1. that the capacity of ‘d’ was 3 and not 5, and
2. if we accept (1) as given, that the capacity of ‘c’ was not 2

At this point, I turned to the this post on the Go Blog, Arrays, slices (and strings): The mechanics of ‘append’. It is extremely well written and well worth a read if you really want to understand the mechanics of slices. But it would have been easy to miss the following statement made mid-way through the post.

“The Capacity field is equal to the length of the underlying array, minus the index in the array of the first element of the slice.”

This sounds like it might explain the behaviour I was seeing, but to fully understand it we need to understand a little bit about the structure of a slice, often referred to as the slice header.

// slice b

slice := sliceHeader{
    Length:        0,
    Capacity:      5,
    ZerothElement: *int,
}

For the initial slice ‘b’ this should be fairly self explanatory, we are describing a subset of an array of integers that has capacity 5 but length 0. Now, bearing in mind the statement on slice capacity, we can attempt to create similar slice headers for our other slices, ‘c’ and ‘d’.

// slice c := b[:2]

slice := sliceHeader{
    Length:        2,
    Capacity:      5,
    ZerothElement: &b[0], // element [0] of b
}

// slice d := c[2:5]

slice := sliceHeader{
    Length:        5,
    Capacity:      3,     // see text for explanation
    ZerothElement: &c[2], // element [2] of b
}

In the case of ‘c’ the capacity of the underlying array is 5, the index of the first element in the slice is 0 and so the capacity is 5. In the case of ‘d’ the capacity of the underlying array is still 5, but the index of the first element in the slice is 2 and so the slice ‘d’ has capacity 3. And this is exactly what we see in the results of the example.

One last point to note is that in the results of the code snippet, the length of ‘b’ is zero. This refers to the length of the slice ‘b’ and not the length of the underlying array which is 5. It is easy to trip up and assume that the length of ‘b’ is refering to the length of the array, it isn’t.

a len=5 cap=5 [0 0 0 0 0]
b len=0 cap=5 []
c len=2 cap=5 [0 0]
d len=3 cap=3 [0 0 0]

Mystery solved.

The thing I don’t yet understand is why this definition of slice capacity was designed this way, any hints would be most welcome.

“Apple Pay launching in China today. Can’t wait for you to try it and see how incredibly easy it is to use! Apple Pay 于今日正式登陆中国。欢迎大家踊跃尝试，体验它卓越的便捷性！” — Tim Cook, Apple CEO, via Sina Weibo

Much has been made of the launch in the technology press with many treating Apple’s chances of success with a healthy dose of scepticism. None of the mainstream technology press brought home the scale of the challenge presented by the incumbents WeChat Pay and Alipay. Not only are they on home turf, but they have also had several years to establish a position of strength in the market. Apple is clearly the outsider. As if this wasn’t enough, Xiaomi (小米科技) and Huawei (华为) are both rumoured to be readying payment platforms of their own.

During my recent trip to China, it became evident just how real the challenge ahead of Apple is. And, whilst many in the press have cast doubt on Apple’s chances of success, I wanted to present my take on what they need to do if Apple Pay is to stand any chance of being successful.

Apple Pay in the UK

I have yet to personally experience Apple Pay in the US and so have based my comparisons with China on my experience of the service here in the UK.

When Apple Pay launched in the UK back in July 2015 Apple were able to take advantage of the proliferation of contactless payment terminals in the majority of retailers. Whilst Apple launched with a number of prominent retail partners, the reality was that Apple Pay was more widely available than people initially realised. Many of the contactless payment terminals supported Apple Pay with no action required on behalf of retailers. In many cases, retailers didn’t even appear to know that they supported Apple Pay. Simply asking to pay by card and holding your phone to the terminal was all that was required to use the service. It wasn’t long before I found myself able to get through a typical week without using cash or even a bank card.

We must remember that prior to the launch, card payments were commonplace even for relatively small transactions. Some smaller retailers insisted on a minimum transaction value to ensure they recovered the card processing costs but you could generally expect to be able to pay for most common transactions by card. I mention this, because this isn’t true of many countries and it is often easy to forget that our norms don’t necessarily apply elsewhere. Apple was able to take advantage of this infrastructure, and both customer and retailer expectation to launch a new service and whilst I don’t wish to trivialise their achievements, it is important to remember their starting position.

The Landscape in China

When I first visited China back in 2006, I went to buy a pair of shoes in a mall in Beijing. With the Visa logo on prominent display next to the cashier’s desk, I handed over my Barclaycard expecting to be asked to type in a pin or to sign a payment authorisation slip. What I did not expect was for the clerk to pick up the phone and make a telephone call to Visa to process the payment. Card payments, at least those not with Union Pay were clearly not common place. Even with Union Pay, it was clear that cash was still the payment method of choice for many retailers.

Now, some ten years later, I’m blown away by the prevalance of the mobile payments platforms, with many retailers actively pushing them over card payments, and in some cases even cash. Following a meal with a friend, we offered to pick up the bill. Our friend had taken a couple of days away from family to show us around the city of Guangzhou (广州) so we thought it only fair to treat her to a meal. But there was a challenge, neither of us had WeChat Pay and the restaurant was offering a significant discount to those who paid using WeChat. Our friend would receive even more of a discount as she had booked the table using Dianping (大众点评), an App in which Tencent, WeChat’s parent company, has an investment. Not only that, the waitress preferred to receive a WeChat payment over cash as it made things easier as she didn’t have to hand over precious small change. I was so surprised by this that before you know it, the waitress had written down the authorisation code and our meal was paid for. We never did get to treat our friend to that meal.

I was curious so probed a little further. Our friend pays for utility bills, travel, meals, groceries, visa renewals and even doctors appointments through WeChat. To her, this was convenient, often cheaper but above all, it was normal.

I asked around to see whether this was the norm and to my surprise I found that not only is it the norm, but it doesn’t seem confined to the younger generations. My in-laws also use WeChat Wallet. And, whilst I don’t want to generalise based on my short time in China and the limited number of people I have spoken to, but the impression I was left with was that if people didn’t use WeChat Pay themselves, they at least knew what it was.

At this point I should highlight that WeChat Pay isn’t the only prominent mobile payments service in China. AliPay has been around longer and is accepted by a wide range of retailers. Whilst in China I didn’t experience of AliPay first hand and they require the use of a Chinese bank card for registration so I was unable to try myself. However, for an indication of how common it is, I refer you to this article by Steven Millward of Tech In Asia when he tried to survive an entire day using WeChat to pay for stuff. There are almost certainly other players in the market but WeChat Pay and AliPay are the two most dominant.

Apple’s Challenge

Whilst the individuals I spoke to cannot possibly be regarded as a representative sample of the Chinese population, I found it interesting that none were familiar with Apple Pay or how it worked. With the launch imminent, the Apple marketing machine would surely address this. But I could see something more than just a lack of knowledge, the people I spoke to just weren’t that interested. They could already pay for things on their phones, why would they want to use a service provided by Apple? Just what was it that Apple was offering them?

Bill / 葛威 on Twitter

If you think Apple Pay breaks new ground, you have to see how WeChat payments dominate China. Household bills, food & transport. #fintech

It would be easy to write Apple’s efforts off. Indeed, many in the industry already have. But here is what I think Apple needs to do in order to be successful with mobile payments in China.

1. Identify a clear value proposition

2. Incentivise retailers

3. Allow peer to peer payments

4. Play nice with the regulatory bodies

Whether or not Apple Pay is successful will depend on how well Apple can address these four essential objectives.

Identify a clear value proposition

When Apple Pay launched in the US, the proposition was clear; convenience and security. By the time they launched in the UK, the value proposition was less clear. We already had the convenience of contactless payments, so perhaps it was just simply security. We don’t yet know how successful these launches have been, but the promise is there. It isn’t yet clear to me that these values alone are enough to entice Chinese consumers away from established offerings. Concerns around data privacy, security, etc. all seem secondary to price and convenience. Apple has to compete on price in order to be successful, even if it means forgoing its usual transaction fees and persuading the banks to do the same. Whilst I don’t see Apple being able to offering the discounts that the other major players are offer, there are signs that Apple and its partners are already starting to offer users some incentives. Bank of China offering everything from loyalty points, discounts, and cash back to customers who use Apple Pay. Being able to compete on price is going to be key to success in China.

But price alone may not be enough. Apple needs to offer the consumer convenience and, in China, that may be at odds with Apple’s desire to control the full end to end user experience. Apple needs to accept that consumers demand services and functionality that exist outside the controlled Apple ecosystem and open up the wallet to its partners.

Concerns around security and privacy may be secondary to price, but this is one area where Apple is able to offer a unique proposition. It may be an American company, and there will undoubtedly be concerns around using an American company to handle payments, but Apple’s brand reputation is strong and their recent decision to fight a court order to assist the FBI in gaining access to an iPhone has gone down well with the Chinese.

Whatever Apple settles on as their value proposition, they need to get the message out to consumers and to do so quickly. With WeChat and AliPay are installed on the vast majority of phones in China (not just those made by Apple), both players have a significant head start.

Incentivise Retailers

Apple’s main competitors offer retailers prominent placement within their Apps, enabling retailers to offer additional services right alongside the interface customers use to make payment. Apple’s desire to control the complete experience around Apple Pay has limited what others can do inside the wallet. Whilst loyalty cards are offered, they are limited in functionality and in reality have retailers have been slow to adopt them even in Apple’s more established geographies. Being able to enhance the wallet experience will be key to encouraging retailers to invest in the platform.

When it comes to infrastructure, Apple needs to incentivise the roll-out of the contactless terminals required to make a payment. Contactless payment terminals are not cheap and unless there is a clear incentive for retailers the Apple Pay platform will be held back by the inability for consumers to use it. If it becomes known as the payment platform that no retailers accept, consumers will quickly lose interest and Apple will struggle to regain the initiative. In some ways, Apple risks falling into the same trap as American Express who missed the boat on the contactless roll-out in the UK and, with their higher transaction fees, provide little incentive for retailers to upgrade their payment terminals to support AMEX contactless payments.

By partnering with Union Pay (Apple didn’t have a choice here), there is hope that Union Pay will support the roll out of the required terminals to retailers. However, both WeChat and Alipay are rumoured to be working on their own point of sale (POS) systems and both have demonstrated a cut-throat ability to muscle their way into a new market.

Allow peer to peer payments

Apple has yet to enable customers in any geography to use Apple Pay to transfer money between consumers. I suspect one of the key limitations behind this is the existing arrangement that Apple has with their partner banks. However the success of competitor peer to peer payment offerings will present a challenge.

During the recent New Year celebrations in China, WeChat users sent 32.1 billion Red Packets (红包) to each other, 8 billion of these on New Year’s Eve alone. These are monetary gifts of anywhere up to RMB200 (GBP20). When sent to groups of friends, the amount received by individuals in the group was randomised to add a sense of luck or chance to the event. These numbers are staggering and Apple Pay offers no alternative. If Apple is going to establish itself as a serious player in the mobile payments market in China, it is going to have to enable the low value, high volume consumer to consumer transactions that drove adoption of both AliPay and WeChat Pay over the last couple of years.

Play nice with the regulator and the party

Apple is a foreign company and it must remember that. The experience of foreign companies in China has shown that Apple is unlikely to be allowed to dominate the industry. If local competitors don’t get there first, you can be sure the relevant organs will help them out. I suspect that along the journey Apple will be required to make difficult decisions that may not endear it to it’s consumers back home. Striking a balance between keeping the Party and the various regulatory factions happy whilst not compromising on its core values will be key to the success of Apple Pay in China.

The Outlook

Foreign technology companies don’t have a great track record when it comes to China. But despite the bearish attitude in the technology press to the recent Apple Pay launch, I suspect Apple is uniquely placed among Western technology companies to give it a go. Brand familiarity is high and that is not something that can be said for many of Apple’s Western competitors. But, investors would do well not to get carried away.

“We think China could be our largest Apple Pay market,” — Jennifer Bailey, vice president of Apple Pay, via Reuters

Apple have been deliberate in their choice of words. And, whilst they are unlikely to be the largest player in China, even small market penetration in China could easily make China the poster child for Apple Pay success. Whatever the outcome, the future of mobile payments in China will be interesting to watch.

Thanks

I’d like to offer my thanks to both @alicialiu, and @the3rocks for help in research and proofreading for this post. I’d also like to extend my thanks to Cassie for the meal in Guangzhou that inspired me to write.

Note: This post has been cross-posted from my blog where it originally appeared on 21st February, 2016.

It doesn’t seem all that long ago when I used to delight in being able to call myself a perfectionist. I was proud of the fact that I could never finish anything because it was never finished until it was perfect. However, 9 years into my career and I am increasingly conscious of the fact that I am still no better at being a finisher. The difference is, I can no longer justify my inability to finish anything on a never ending search for perfection. I am a part of a generation for whom ‘just good enough’ is the end goal.

The Age of Beta

What started as the Gmail Beta has now become standard practice across many industries. In an age where being first matters more than being right, we consider it acceptable to release unfinished applications and software, or to publish unfinished blog posts and articles. We expect 24 hour news networks to deliver credible retrospective analysis on events which are still unfolding as we watch.

It is more important to be first than it is to be right.

Don’t blame the publishers. Collectively we demand to be kept more informed faster than ever before. We demand early access to the next big thing. Who cares, if it is not finished yet? All we demand is that it is ‘just good enough’. We have grown accustomed to watching inane speculation as dramatic world events unfold, reading unfinished articles, and working with incomplete products. Why? We put up with it as a price for early access to tools and information.

For the Twitter generation, even the news networks are not fast enough. By now, we are all too aware that this pace comes at the expense of accuracy. Many a journalist has opined the loss of accuracy in reporting today and so I don’t intend to further that particular debate. But for me we have also lost something else in this rush to be first; we have lost a sense of what it means to finish something. This is true for software development, just as it is true for journalism and publishing.

Release Early and Often

There is a school of thought that says we should release early and release often. Without doubt, this approaches carry a number of benefits.

• Early feedback from your customers/readers
• Validation of a product before significant investment
• First mover advantage

However there is a fine balance between releasing early and releasing prematurely. All too often, the question of when something is actually finished is largely ignored in favour of getting something out there fast. Releasing a product with a carefully selected subset of features that are complete is very different from releasing a product that teases the user with many features, none of which are quite finished.

The distinction may be obvious but it is important. We don’t have to be feature complete to finish something. There is no need to release an article or blog post that is a work in progress. It is possible to complete an article without waiting to see if circumstances change or new events unfold. By all means release early, but release when the article is finished, not before.

Just Good Enough

Releasing early and often should be a concept familiar to those involved in the hip world of start-ups and their associated funding schemes. However it is not just start-ups that are losing the ability to finish anything. We all are, including those of us working in large enterprises. Whether an internal system or a new customer facing product, the goal should always be to finish. At every level from developer, development team, project team or project sponsor it should be clear what it means to be finished. You’d be surprised how often this isn’t clear.

• Developer: Is this code finished when written? When tested? Or when in production? When it performs well?
• Project: Is the release finished when it has features A, B, and C? Or does it need feature D as well?
• Sponsor: Is the project finished when some code makes it through to production? When the first customers use it? When the investment case is realised?

What may also surprise you is that every interested party will have a different definition of what it means to be finished. If no one can agree on what it means to be finished, is it ever possible to finish?

Take any project you are working on now and ask the same question, “When will this be finished?”

Outside of software delivery we can ask similar questions.

• Hacker: Is the hack complete when the you can see how to solve a problem? When you have a proof of concept solution? Or when you have a solution that you are proud to show to others?
• Writer: Is the article finished when drafted? When proof read? When published? When first read by someone else?
• Photographer: Is the picture finished when the shutter closes? When the negative is developed (or edited image saved to disk)? Or when the photograph is framed and hanging on a wall?
• Cook: Is dinner finished when it comes out of the oven? When the last morsel has been consumed? Or when the dishes are washed up and put away?

All to often though, the focus is on “just good enough”. We don’t know what finished means and so we settle for just good enough to move on. As we strive to be first, we have lost the ability to finish anything.

I count myself as one of the biggest offenders. I don’t challenge myself or my team enough with this question, when are we finished? How will we know? Just prior to Christmas I asked a couple of the team at work and was unsurprised to find that they all had different answers.

In Conclusion (Finishing)

The art of ‘just good enough’ is just that, not good enough. As we rush through life, making sacrifices in order to be first (or at least fast), we need to stop and ask ourselves what it means to finish something. When something is finished we have an opportunity to feel pride, an emotion all too often lacking in the workforce of today. But above all, finishing means we can do just that, finish. We can move on to the next thing free from the lingering thoughts and distractions that stem from unfinished work. For me, I’m going to start by finishing this article.