Despite the challenges we faced as a company, I still believe this work represents an inspiring example of early creative AI exploration — back when the focus was genuinely on augmenting human creativity rather than replacing it. These projects emerged from a time before the current generative AI landscape, when we were more concerned with empowering artists and designers than automating them away.

The original text remains unchanged from 2019.

With love, Roelof

creative.ai started out with a big vision and dreams of how artificial intelligence could help making creative work more creative.

The challenge with being early to a market is that there are not as many examples to learn from. We took inspiration from the 60s, 70s and 80s work in creative computer science, cybernetics, and human centric "augmentation", rather than automation, today's computational creativity, as well as the history and practices of the big design behemoths as Adobe, and forward-thinking design firms like IDEO and Frog Design.

While we successfully build some interesting technology, and conducted a lot of user testing and product experimentation, if we're very honest with ourselves we have to admit that we never really managed to turn our efforts into a single engaging and enjoyable product. Instead we found ourselves helping agencies, designers, and brands as a service consultancy, effectively transitioning partly into an agency for creative work "dog-fooding" our internal tools, but without a real urgency about putting these tools out for a wider audience: a clear lack of product focus.

This, combined with disagreements about strategy and business focus with (now former) two other cofounders, made it a difficult journey for everyone involved.

In short: it's time to call it a day, and move on. We — I speak now for me, Sam and Tero — still very strongly belief that the new growing field of Creative AI is an exciting opportunity for developing human-centric technologies, products, services, and tools that aid creatives everywhere. And for myself I can say, perhaps a bit more humility and focus as well :)

This all doesn't mean that it has been a total waste. A lot of people working at creative.ai have told me in person that is has been the most intense, but also the most transforming experience of their career: many have found passions to pursue or career switches to make (designers turning developer, data scientists turning artists, etc), by and large made possible through the supporting culture we build all of us working together.
Next to this we have dreamed up, and developed many interesting "new" (new as in unlike what's around today, but not as much new for those willing to look back a couple of decades, to the early computing pioneers for example) ways of building user interfaces, interfacing with tools, or ui/ux principles and designs.
In what follows a couple of our experiments and projects we've worked on the past two years, many of which we will make available as open source projects for anyone to tinker with. Some of these are rough ideas or prototypes from design sprints or our Friday "hack days", some were possible product ideas, and the last one Bloma which we had in "alpha" for a while and which we're making available as an open source project)

Symbolic Driven Design

Inspired by work from Christopher Alexander idea of a "pattern language" and his series "The Nature of Order", we imagined a design tool that would operate on the level of "patterns", and cognition. The frame of thinking followed here is that a large portion of creative work can be seen as "mechanical execution", where the creative spark comes from ideas, intuition, insights, and experimentation. Could not creative work be greatly accelerated and become more "creative", if the execution was done more automatic or enhanced by AI? In this way, a system could steer in the right direction, putting you in the driver seat of making the choices and deciding on the parameters to explore, a shift towards curating/picking the results one likes, to then move forward on, and the cycle repeats, in a cybernetic fashion between you, the system and its feedback loop.

Next to Alexander's work, the specific design properties, or patterns to play with, were also informed by the Gestalt Theory (a field of design thinking about the organisation of design, with one of the classics Laws of Organization in Perceptual Forms by Max Wertheimer is worth reading).

Generative Design Systems

Similarly as symbolic driven design, but in a more constrained setting (or more constrained design space), is a tool we developed as part of an exhibit for IDEO as part of the future of design.

One area where systemic design takes place is during the development (or "update") of a brand personality. Often such branding exercises revolve around around what archetype you are as a company/brand/person: are you more conservative or extravagant? more emotional or rational? more playful or serious, etc.
Branding exercises usually work with 12 Brand Archetypes of which a primary and secondary type is chosen through a bunch of exercises. There are a lot of methods and exercises to get to these, but the step from choosing which archetype represents "you" and its visual counterpart almost always happens as two very distinct linear steps.

For the above mentioned exhibit we imagined a generative branding system that lets visitors play around with similar binary personality choices. But instead of getting to a single main archetype we used the choices to position a user of the system into a specific spot on this "branding world map" where the outer extreme edges were formed by the more stereotypical archetypes (Hero, Jester, Explorer, Maverick, etc).
The user made its choices in the form of a DJ mixer, where the knobs (like rotary controllers on a midi controller), made you choose between particular attributes (eg Playful or Serious, Friend or Authority, etc) which together decided on you place on this branding landscape (or high dimensional latent space in neural net speak), which were then fed in real-time to a generative design system that generates a poster matching a style and composition. A larger scratch pad let you create variations around this aggregated theme, so a user could get a sense and feel about the particular “brand” and the possible variations contained in that particular place.

Building a generative "system" is a bit like "gardening", as musician Brian Eno likes to say. For a system like the one described, a lot of time is spent on tweaking and fine-tuning the system, it's parameters, and its output, until everything works together well enough that it's cohesive enough to be understandable, and of a quality you want it to be.

Neural multi-modal image search

A quick and simple hack that lets you explore visual themes and the path in between these themes was born out of a Friday hack, we referred to as the "weirdness-engine". The idea is that

3D style transfer

Inspired by the work in style transfer on images, we explored ways of making styler transfer more useful as a tool or brush for people to paint with. Taking this a step further towards product design, we iterated on some work for retailers that would let visitors style clothes, sneakers, or anything of which we could produce a 3d model, keeping in mind both the general visual "content:" to stylize, as well as keeping in mind the structural constraints and folding of a 2d texture in 3d space. Next to this we also made it possible for brands or retailers to select which part of a texture the generative system could have a go with, making sure that branded aesthetics as logo's or visual trademarks, were left untouched.
The technology itself was not much unlike those release by Google under its lucid code base as part of wider research in neural network interpretability.

Generative Visual Design: Bloma

However, we quickly saw that the value of blōma extended beyond this initial step of generating ad variations. Blōma started looking more like a way to accelerate visual design work in general, leaning on a few key ideas:

1. Systemic design. Creating variations based not merely on static templates and placeholder, but systemic rules that give rise to large variation spaces that adapt to different formats and different types of content.
2. Generative design. Instead of starting design work from a blank canvas, starting from large design spaces already generated for you, where your design work then becomes a matter of navigating and filtering the design space, rather than building things from scratch.
3. Embedded brand style guidelines. When you have an existing brand, you usually have visual guidelines for expressing the brand in terms of typography, colours, and image treatments. With blōma, you can embed these guidelines into a living design tool, allowing you to quickly produce new content that is on-brand and high quality.

We believe these ideas to have legs, and we’ve barely scratched the surface with deploying them. For this reason, as creative.ai disbands, we’re releasing the blōma app under an open source license so that anyone can play with it, extend it, and fit it to their needs. You’ll find blōma on Github.

Final words

Finally, I'm very thankful for everyone who was part of our journey, especially Sam and Tero who were part of creative.ai until the bitter (sweet?) end.
And from the three of us: thanks to Petra, Ruby, Nikola, Damir, Denis, Dan, Stéphane, Carmen, Gloria, Ivan, Dina, Sorin, Miguel, Mikkel, Andrei, Imanol and Sofia for being with us for the months or year+.
And big thanks to our investors, Max of Sunstone (now Heartcore), Suzanne of Local Globe, and Jan & Stefan of Discovery Ventures. Thanks for all the support and help these past years!
Last, but not least, I want to thank Alex and Samim for starting this journey together, and for all the time we've shared, and lessons learnt from each other.

Early on at creative.ai, we realized we couldn’t even build the kind of products we wanted to build without changing the way our company operated. Since then, we’ve continued to evolve our internal culture to level-up alongside our own AI systems. Not only is this approach helping us rapidly innovate in technology, but—more importantly—it lead us to develop the humanist principles necessary to take it further!

This article is based on creative.ai’s invited talk at “A.I. & Society” delivered in Tokyo recently, describing the motivation and story behind our company culture.

Watch the video recording of our invited talk here. (17 minutes)

Many of the graphics designers and visual artists that creative.ai works with wonder about the future. They wonder if we’re building a machine with big red button that says “AUTOMATE CREATIVITY” and if someone’s going to press that button what would happen... Are we heading for some kind of cartoon-like moment where we hit the ceiling and slide down the wall? Who knows! That’s the question everyone has and that’s probably why you’re reading this today...

But we’d like to flip the question around; instead of asking what the impact of technology is on society in the future, instead we’d like to ask what society does to technology. What has been the impact of our culture and societal values on technology?

We realize this is coming from the complete opposite perspective as most of the conversation online, and many people at the “A.I. & Society” event too. But we hope it stimulates some neurons and gets you to think about the problem from a different angle.

After a quick survey of our followers on social media, we found that 2 out of 3 people in the AI community and IT industry in general believe that machines will have strictly superior abilities compared to humans at some stage in the future — both intellectually and creatively. That’s two out of three! (67% is also corresponds to the number of Twitter bots that follow us, but it’s likely that’s just a coincidence.)

As a team at creative.ai, 70% of us think that humans will always have superior intellectual and creative abilities to machines. Some things machines may be more efficient or effective at particular tasks, but humans will always be able to do the superset of those abilities. We believe humans will always have the upper hand, so we’re coming at this situation from a different perspective.

At creative.ai we think of Artificial Intelligence as a technology tree, rather than an exponential curve of doom. It’s very much like a game of Civilization to play where you have different branches to pick, many options to choose, mix and match. Some of the techniques in the past we used may not have been funded well enough, we didn’t have the hardware or necessary understanding as a society to pursue this research…

Now we have the choice to be able to go back into the technology tree and decide what we want to achieve. What do we want to accomplish? How do we want society to turn out? We can jump back and make decisions according to these goals, given the extra perspective we have now.

For creative.ai, two of our founding pillars and the inspiration for the company are:

1. Augmentation & Interaction — when you work on creative things, that tactile feel and playful interaction is very important. It all goes back to the work of Douglas Engelbart in the late 1960s. We’re big fans of his work and building on his accomplishments early on in this technology tree.
2. Patterns & Generative Life—when building living systems almost like organisms, they emerge from patterns. This is based on ideas from Christopher Alexander since the 1970s that inform our design philosophy.

These are our own two guiding lights that we’re basing our approach on.

We also believe very strongly that the Team—or the Organization—equals the Product. And that’s called Conway’s law, which says that whatever patterns are present in your organization then those will be reflected directly in your product. So any advantages, opportunities, disadvantages, quirks, politics you have in the company will translate directly into the product you’re building.

At creative.ai, we’re firm believes of this Law, but also taken the opposite approach too, which is called the Inverse Conway Manoeuvre. It means designing the product you want to build then structuring your organization accordingly so the product emerges accordingly as designed. We’re very mindful about working on both of these and designing them accordingly, levelling them up side-by-side.

We’d like to take you through a short history of creative.ai, it’s a very young company relatively speaking, but we’ve been through many different mindsets that are reminiscent from The Industrial Age, then The Information Age, and a more humanist age coming up that we call The Creative Age.

For each of these mindsets, everything affects the whole company the whole product. There’s AI and design involved everywhere here, from the culture to legal, including messaging communication. Everything is inter-related so each one of these things affects the other. This article will dig into each of these…

When we began as a startup, we didn’t think like a big corporation; that’s the advantage of a small company. But we inherited lots of external pressure and expectation, so we fell into these traps of the corporate mindset which were not necessarily beneficial for us.

Looking at it from a legal perspective, ownership is control. As founders, we initially had all of the ownership with some allocated to the team. Then over time the investors buy incrementally buy over more and more of the company as the company grows and goes through multiple rounds of funding. As shareholders, whether founders or investors, we’re basically a small group of people that have control over the company.

We don’t think this structure, even when you have a board of directors, is intrinsically ethical. There are few people in control of what our A.I. technology can do and what the impact is on society. We’ve talked a lot about addressing this, for example with ethics committees, but since this ownership and control structure is not ethical in itself, we think it’s a bit like putting lipstick onto a pig. (We’ve been redesign this accordingly, as you’ll see below.)

The ownership structure reflects itself in the team directly too, where it’s usually setup as a command and control structure. The investors love to hear what your roles are, who is Chief Creative Officer? Chief of Technology? Science Officer? We had these different labels that we gave ourselves, even though we were a small company, they were expected from us. In the future maybe we’ll have a Chief AI Officer with an army of Machine Learning engineers reporting to them ;-)

This setup creates a lot of tensions, both within the hierarchy top and down between “leadership” and the team itself, as well as between separate branches of the hierarchy—in particular if you separate the design and technology!

People tend to be frustrated and dissatisfied when working in these kinds of environments, no matter the company size. It’s coming from a place of power and control, which affects the people on the team and their mindsets. Everyone spends more time in internal company politics (we call this “Job 2”, the one you’re not hired for) not being able to affect things outside of their small area of influence in the hierarchy.

What does design do? Is it a separate branch from technology? Do designers just write documents as large specifications and throw them over the fence? This isn’t a very participatory culture; not only does this produce worse results, with fewer people bought in to the designs, but also causes a lot of sadness…

Looking at the statistics globally for the U.S., 18% of employees are actively sabotaging the projects they are on. 52%, that’s over half of employees, are retired on the job! Are these the kinds of emotions you want in people building or using the next generation of A.I. systems?

These are direct consequences of the power structure that goes on in almost all companies today, as a reflection of the ownership structure and the hierarchy that goes with it.

As creative.ai, we wanted to change all this but we inherited a lot of this baggage from the outside. People defined us as a team of engineers solving creativity. This very much a trans-humanist message where we’re coming in on a shining white horse trying to save humanity as a whole by solving creativity.

Actually, there’s nothing to be solved; creativity is a process and doesn’t need a solution to be found… and engineering the least important part of it.

As a product, we talked about generative pipelines which resonated with some investors and big companies as well. The idea is that there’s an overall A.I. system that controls the generation of things end-to-end, then we put human-shaped holes inside this pipeline wherever there were problems we couldn’t “solve” with technology or where we needed more data.

This was a hybrid human/machine system, but it was not a humanist system. The people were basically cogs in a larger process that we were just using to gather data. It’s a form of centralization of power, just like the ownership and the hierarchy, but in this case it’s in the machine. Words like “cloud computing” and “platform” remind us about this power dynamic…

On the technology side, everyone talks about Big Data and Deep Learning. For the same reasons, they’re not the kinds of techniques we pursue primarily because every single individual’s is less important. You sacrifice every individual at the cost of the average; so if you’re not average, you’ll get sub-optimal performance.

But nobody is average. In the creative space, everyone wants to stand out! These techniques, as defined today, didn’t work for us. We couldn’t rely on them as our primary technology to empower individual creativity and agency. We have lots of data but we don’t think of it this way. Even the concept of a Server Farm is also a centralization of power, but you get the idea…

As a team, we started out firmly in The Information Age with a more distributed mindset, which affected how we approached things internally.

As you saw above, the ownership structure has a strong impact on the entire company and its culture. However, from a legal perspective, we were not “distributed” in the sense that we were not a cooperative with equal ownership—except at the very start when the company was founded. After that, we inherited a lot of legal structures and expectations.

Trying to keep things as fair as possible, we give out a lot of shares to employees and have deterministic salary models too. We also wrote exit clauses that specifies constraints on our technology should something out of our control happen to the company...

The thing we benefited from the most was having a distributed team from all around Europe. We were able to hire a dream team of programmers, designers, architects, in A.I. and M.L., frontend / backend—each that fit well with our company culture so they could onboard quickly and efficiently.

We found that consultants and contractors were the most keen to join forces with us. But we became very aware of this power tension between the company and the individuals, and we didn’t want to turn into a Gig Economy or the “Uber for AI”. We were very mindful of this and drifted out of this model.

When you have this power relationship between the company and its team members, the tension is like a competition and also affects the individuals themselves. Conflicts can emerge between the design and the technology side of building the same AI.

We got around this by using popular self-management techniques, and more transparency which helps a lot! But these are remnants of this Industrial Age mindset where the hierarchy is flattened out and the network must police itself: trying to control everything going on, making sure everyone shares all the time. There’s no trust there.

With people having more agency in a network, being able to decide what they can work on and who they can work with, it creates confusion and uncertainty about what they should be working on. Should I work on this new feature of the AI product or design the UI for its frontend? Should I collaborate with this team or this one over there?

So instead of being angry and frustrated with the hierarchy and lack of freedom, team members become more afraid of making missteps. From school and work life, most people have inherited a mindset of being in a hierarchy and when that structure is gone, most don’t know what to do with the freedom.

This fragmentation and fear also translates in the way we communicated about things. It was about our software vs. established tools, about our humanist approach vs. others that want to automate creativity. For example, “if you use our tool then we’ll use more augmentation.” We defined ourselves with this message to stick together to face the uncertainty of the future together.

The product in a distributed mindset is very fragmented; you have lots of applications on your mobile phone for style transfer or photo manipulation, etc. We fell into this trap too by considering many applications to build on top of our platform. Passing data between these different applications is difficult and doesn’t make for a very good experience. These are side effects of having a distributed culture, for example in a company where product teams don’t work as an integrated whole to collaborate together towards a bigger vision.

On the technology side, we’re building what we call the Generative Web where you can run algorithms on the server on behalf of designers, or on their own computers or mobiles as appropriate. Wherever the computation is most appropriate it will happen there, and the data will get passed around accordingly.

This kind of infrastructure is one of the biggest benefits of The Information Age mindset, making the most of the internet and the many hybrid devices that are connected together.

As a company, we’ve found ourselves most recently moving into this more collective mindset, which is new for us. We pulled together lots of different approaches about this, and are very curious to see where it will take us in the future. (We call this The Creative Age.)

In this mindset, it all starts with the people. We assume that everyone on the team is a mature, rational, responsible and has a purpose. Team members know why they’re in the company and know what they want to achieve in life. It also means they have an “Internal Locus Of Control” and feel like they have agency over their environment.

This changes the internal culture significantly. It makes a culture that’s based on Trust, since you can rely on each other to behave like mature rational adults. You don’t need radical transparency to make sure nobody’s making mistakes, but can rely on team members asking for help when they need it, sharing often because they know it helps everyone learn. If someone does make a mistake, there’s a culture of Respect—or maybe even, Love—to make sure that problem gets resolved in an appropriate fashion.

This culture also affects the way the company also addresses the team too, moving more into coaching to help everyone grow, support and encouragement of personal alignment, and lifelong employment as long as no foundational rules are broken. This helps create a space that’s conducive not only to growth and learning, but being more creative!

This is a great environment to build innovative A.I.-based products because it lets the team self-organize dynamically to tackle difficult problems that would be difficult to solve with a rigid structure. The kind of things we’re exploring nobody else has before, and we need the space for self-organization for everyone to come together to complement each other’s skills.

We picture it like a band. As soon as there’s a tune for a specific milestone or deadline, demo or client project, presentation that needs to be prepared, people with the necessary skills can just come together like band—then go back to regular jamming or practicing or join another project.

This approach often makes things faster to self-organize and it’s very dynamic. It’s not a fixed network structure or hierarchy, it’s more like a living organism that’s pulsating and vibrating to a common beat. It’s quicker to respond to external changes too.

We also think of the legal side for the control of the company as a shared performance too, with in this case four or five different voices: the team, the community, the clients/partners, and the investors. Each have their own style, tone and message to bring into this band that helps steer the company as a whole.

This makes it a much more ethical structure when there’s not a single mindset (profit) that’s dominating the Board of Directors. Instead, it’s the shared performance of a very diverse set of voices that come together in unison—in this case in favor of the common good. (This structure is sometimes called Fair Shares, and we’re currently looking into this with our partners.)

As we move into this more humanist phase, this also affects how the product works. The product is no longer a rigid tool that we hit things with, it’s an instrument that you play. Since it’s an A.I.-amplified instrument that learns how you play, it becomes a feedback loop in the cybernetic sense.

You work perfectly fine on your own as a creator, and this instrument can also do stuff on its own. But when you put the two together, the sum is bigger than the individual parts on their own. This is the core theme of augmentation.

It also reflects in the way we communicate about our product too. The messaging is about taking our A.I. instruments and putting them into groups to augment them, to improve them. This helps empower everyone on those teams to provide more agency, and a stronger sense of mastery over their creative process.

The technology is equally fascinating, and worth a whole other article! It’s all about interaction; how do creators interact together? How do we build our technology into instruments that are easy to interact with? How can these instruments plug together modularly and come together in these jamming sessions and creative environments where everyone contributes their own perspectives and skills.

To conclude, we think there’s a direct relationship between how you structure your organization and the product. If you want to have a positive impact on society you should start at home with the organization itself and rethink everything from the ground up. We’ve done so very mindfully by taking the organization’s principles and levelling up the product accordingly, but sometimes the product was ahead of the organization and we had to level up the team as well.

By doing this, we’ve likely gone out of the comfort zone of most managers and leaders, but just seeing how the team and potential future hires resonate with our culture, we know that in the future the best performers and the best teams will be the ones that really engage with these ideas and resonate with the projects they’re working on.

With all that said, if this message resonates with you and you’d like to work with us in any way, both on creative topics or AI-related topics, we’d love to hear from you!

• If you’re a visual designer, we’d love to get your feedback.
• If you’re a developer, then consider joining the team!

If you’ve read everything this far, we’re sure our paths will cross again on our journeys into The Creative Age. See you soon… #⚘

Watch the video recording of our invited talk here. (11 minutes)

These questions motivate everything we do at creative.ai. How is artificial intelligence changing design work? What does it mean to be creative in a digital world? Why should we put more humanism into our tools and their interfaces? How can we make the workplace more playful for everyone?

This article is based on our #SLUSH17 keynote delivered in Helsinki recently, that elaborates on all these points and outlines our driving principles.

Watch the video recording of our keynote here. (11 minutes)

At creative.ai, we’re in the process of re-imagining work in The Creative Age (a goal that’s as lofty as it sounds) but also redefining what it means to be in stealth! During the day, we quietly and diligently collaborate with the top graphic design agencies worldwide — bringing their creative workflows to life with Artificial Intelligence, and helping manage this transition in the least disruptive way possible.

In the evenings and during weekends, everyone on the team transforms into a casual creator in one way or another. We become musicians, poets, visual artists, comedians, game designers, performers, storytellers, writers, composers… We do it because it’s who we are and because it’s fun and messy, sometimes even beautiful — but mostly just messy!

Frankly, despite each working in the field for decades, we find today’s creative tools rather frustrating. Whether it’s off-the-shelf software or cutting-edge A.I. algorithms, it takes a lot of mechanical skill to use these tools today. The learning curves are more like cliffs, and even when you build up some expertise there’s a lot of mental strain for getting things done. It’s rewarding to be creative and innovative, but the frustration is always there…

The fact is, machines have already taken over the creative process, whether we like it or not. We create our tools, then our tools format us. The interactions and interfaces we use today were designed in the late 60s, and the software we use is based on designs from the late 80s. Our notions and entire beliefs about digital creativity have been shaped by these early decisions. Nobody wants to be formatted like this, and especially not by tools that have been around for so many decades!

Over the past year at creative.ai, we went back to the drawing board and asked ourselves some fundamental questions:

• What does it mean to be creative?
• What defines being human, even?
• How can we design interfaces and tools on our own terms?

Here’s what we learned… (You can probably guess what happened next!)

The first thing we discovered is that everything we do in the workplace today is wrong—at least from a creativity perspective. From our sitting postures to our office layouts, to the working hours and stress levels, the research and the studies are clear: all this makes us less creative.

If you had to redesign the workplace to promote healthy design practices, innovation and creativity, you’d come up with something that’s the opposite of what we have now!

The second thing we realized is that everyone seems to have forgotten what it means to be human. Our tools have formatted us for so many years that our entire self-identity has atrophied. Many people in the field are worried about AI because they define their skills based on the tools they use: they think of themselves as “Photoshop technicians” and somehow sideline everything else of value they bring to their studios.

At creative.ai, we believe that human creators will always have essential roles in the creative process that machines would struggle to do or not be able to do at all. This mindset carries into our design process that aims to amplify human creativity. (It’s good news for all human readers here!)

Assuming there will be humans around in the future, we should probably ask ourselves the question: how do we re-design our interfaces and software based on humanist principles? For us, this has meant starting with the three universal factors that motivate and engage every person on the planet, based on Self-Determination Theory.

We’ve taken each of these three motivators (autonomy, mastery, relatedness) and turned them into corresponding principles of design:

1. Empowerment (for autonomy)
2. Expressiveness (for mastery)
3. Collaboration (for relatedness)

These inspire what we do at creative.ai, but they’re also principles we believe are appropriate in all other software too.

EMPOWERMENT

Part 1. Let’s look at the situation from a perspective of autonomy first. This is arguably the topic most passionately discussed at our creative.ai team retreats!

We believe that most software built around Deep Learning is on the dis-empowering side of the spectrum. This automobile production line is a great visualization of the process: data comes in and data comes out. Everything is built around the machine, so labor is also structured entirely how the machine works: there are human-shaped holes at regular intervals. There are people to clean the data, annotate the inputs, label the outputs.

These aren’t the most empowering roles, autonomy is low because the system dictates what you do, and so it’s not necessarily very creative either. Yet this industrial age mindset is the most common today when people design their software around “Artificial Intelligence”. What would it take to fix this?

Another alternative metaphor for software is inspired by the Information Age. There are large quantities of data being gathered from sensors, cameras, trackers, monitors and it’s all presented to the users. At the same time, there are many controls available as a collection of input fields and sliders, pages of buttons and tick-boxes, as well as other actuators.

It not only requires a lot of expertise to learn to use the systems, but the bulk of information and overwhelming choice creates cognitive overload. This also has a tendency to reduce creativity too. How can we reverse this effect?

We think it’s time to flip this relationship around and make it more playful. Imagine a child playing with a toy, full of joy to the point of forgetting the world around, so deeply engulfed in the creative flow. Creators should be the ones to have the power, putting them in charge of the system as if they were its original creator. That’s how we think about the software we’re building, to make it even more empowering.

At creative.ai, the metaphor we like to use is AI as an instrument. You can pick it up, perform, experiment, be creative, produce something, then put it down again when you’re done. It’s yours and you’re in control at every stage of the process!

Who says the approach you use today for your specific problem is the best choice for you tomorrow? Should anyone else make these decisions for you? Only you can determine what’s most appropriate for each situation, especially if you’re aiming to be creative!

We believe the best way to do this with software is using a modular approach where you can assemble your own tools in a do-it-yourself (DIY) fashion. This takes us back to more constructivist languages and frameworks like Smalltalk and Squeak that make it easy to modify running systems, remix ideas, and remake tools on-the-fly as you create.

When creators are in charge and can build up their own tools, it becomes much easier for them to offload menial tasks they see fit to the system. We want people to be able to dance around the automation line, for example:

• Ask the system to generate dozens of unexplored design variations when you go for a run to review when you get back.
• Ask the generator to make multiple variations of any particular design so it can be used in many different contexts.
• Ask the software to perform an exhaustive review of other work in the field overnight and give you a report.

These are the good kinds of automation!

EXPRESSIVENESS

Part 2. Now let’s look at creative software from the perspective of mastery, another one of the core drives from Self-Determination Theory.

Software today is rarely self-explanatory. There’s often a learning cliff that few people overcome. Both the interfaces and the features themselves are hard to understand, and you need clunky books or awkward video tutorials to understand what’s going on!

When you do get over the learning curve, there’s a chance you’ll hit the limitations of the software. The software is usually made of complex algorithms that must be built beforehand by the original developers. At best you can hire a professional to extend your tools for you, but the software is not expressive enough to extend itself.

Algorithms for Artificial Intelligence or Machine Learning are not very self-explanatory or expressive either. Have you tried looking inside a deep neural network lately? These algorithms are built to operate efficiently on their own, but not designed to work well as components in a larger hybrid system that involves real people. With recent EU regulations on the right to explanations, this is going to affect the algorithms we use, as well as their integration into our workflows.

We suggest approaching the problem differently. At creative.ai we’re designing a language for creativity that lets you express what you want to do with patterns, properties, symbols… This language can apply to architecture, music composition, visual design, storytelling, or any other creative process. All interactions are then expressed within this language!

By using combinations of patterns, properties, or symbols (whichever name you prefer for these concepts), it’s easier to see what’s happening within the system. It’s no longer a black box that takes a Ph.D. and a server farm to analyze. You know at first glance that a pattern breaks up into other patterns. Each property has a clear label and is described within the system itself.

Naturally, you can assign your own labels to these properties or symbols, those you use yourself or within your team. You can create new patterns that combine other patterns together, then start using them as part of your own dialect that you use to create things!

When you build up your own toolset and its language in a do-it-yourself fashion, what you’ve built becomes a reflection of what you’ve learned. AI then becomes a tool for self-reflection too. How do I approach this problem? What actions do I use most often while creating? What concepts am I missing that I could add to my palette?

COLLABORATION

Part 3. This brings us smoothly into the topic of relatedness in creative software, the last core drive from Self-Determination theory.

The workplace today often puts people next to each other, but it isn’t necessarily the most collaborative nor the most creative environments. The overload of information doesn’t help collaboration, the fact that we’re often seated in static positions, indoors with little fresh air, in boring and regular environments, working long hours to a schedule. The research shows these settings are not conducive to creativity.

Mobile technology doesn’t necessarily help either, even when it’s outside of the workplace. Not only are applications are robbing us from our attention, they’re preventing us from having piece of mind. It’s those moments of tranquility and calm that are most conducive to creativity. We need boredom to have the opportunity to be creative! It’s from these peaceful and tranquil moments that sparks fly. When technology gets out of the way, both personal inspiration and collaboration can happen inhibited.

What would human-computer interactions look like if we redesigned them? When you’re on a walk outdoors, why not share your ideas or suggestions immediately with the system through your mobile? When you’re in a business room, why not use the whole space and project things on all surfaces? When you’re brainstorming, why not use a more tactile approach using physical objects, scribble things down on paper and have the system read your sketches?

Thanks to progress in computer hardware and software, we can use modern AI and machine learning to help redesign these interfaces. With computer vision algorithms, voice recognition, motion tracking we can have the system operate in the environments where we are the most creative!

We can also use Artificial Intelligence to shift the burden of getting over the learning cliff over to the system too. The tools we use should be able to model us too, not just requiring us to do all the work. This makes the relationship bi-directional and moving into co-creative workflows.

Taking the next step, we can also redesign our interfaces around the idea of empowering and enlightening creators, letting them ask for help about any topic they are interested in. We can create coaches that help the creators build-up their expertise in the most constructive way. We can create experts that can explain how things are being done, for example visual design.

By using AI agents that collaborate with us, we can also build a network that includes both human creators and AI agents. This makes sharing concepts, goals, processes, histories much easier thanks to the underlying creative language that’s shared. It becomes a generative web that can produce creative artifacts!

CONCLUSION

You may have been expecting it, but there’s no product screenshot in this post. We’re not trying to sell you anything! This is about something much more important: it’s about making a choice about the kinds of systems you want to build and use in the future.

We have to assume there will be machines in the future, and for the sake of those reading this that are human, we should assume there will be humans too! ;-) The question is how do we design for the interactions? It’s all about those interactions! There are three major ways we can approach this.

We can take Industrial Age mindsets and turn humans into digital cogs. Under the guidance of strict software, these are easy roles to fill and are mechanical to perform. We can easily measure outputs and it’s the kind of product that Wall Street would be the most interested in! Of course, it won’t be very fulfilling for anyone in the pipeline, and the outputs won’t necessarily be very creative either…

Alternatively, we can take an Information Age mindset and see everyone as producers and consumers of large quantities of data. All your online tools and applications, even the office environment, is instrumented to capture as much information as possible. Having all this data available and so many choices too can make it overwhelming to the point of making you less creative too. While this approach is very popular in Silicon Valley, these open plan offices (or cubicles) are not much better than the conveyor belts. It’s easy to fall into these patterns too, but we’ve chosen otherwise.

The alternative is to think in more humanist terms, but it’s not easy. In fact, it’s the most difficult thing we’ve done at creative.ai. It requires everyone on the team to level up, be more mindful of every aspect of their work lives—becoming more involved and taking more responsibility. In the long run, we’ll have to change the way companies operate from the ground up to be more humanist too.

The result will be new types of organizations with a supportive culture, that embrace emergence and where team members are like musicians improvising in self-organizing bands. Not only does this require mindful design of company culture, but we also need to design our AI systems as instrument that don’t get in the way — instead empowering the creative process.

As a society, our current trajectory was heading to a dystopian future where the machines we build are either in direct control of our lives (similar to George Orwell’s 1984) or alternatively designed to exploit human nature (rather like Aldous Huxley’s Brave New World). As a side effect, our creativity and playfulness both went out of the window, but if we can recapture those there’s a chance we can build an alternative…

We call this “The Creative Age” and it starts with you! It’s your choice now what kind of systems you want to interact with in the future. Are they more like industrial pipelines or instruments to play with? Will you be mindful about these and take responsibility for their use? Not everyone will choose to craft their own future like this, it will be the hardest thing we’ve done, but there’s hope that we’re in this together!

Watch the video recording of our keynote here. (11 minutes)

Does this post give you hope? Do you resonate with our vision?

• If you’re a visual designer, we’d love to get your feedback.
• If you’re a developer, then consider joining the team!

If you’ve read everything this far, we’re sure our paths will cross again on our journeys into The Creative Age. See you soon! #⚘

CreativeAI.net is a space to share CreativeAI projects. Countless links to CreativeAI Projects are floating around the web. It’s hard to keep track of these gems. CreativeAI.net attempts to solve this problem by offering a collaboratively curated feed of projects. Each project is conveniently presented as Link + Media + Tags + Discussion. People can submit their findings and let the community discover and discuss it. A regular newsletter makes it easy to stay up-to-date on recent advancements. It’s free and open.

http://CreativeAI.net

Final thoughts

There are countless places on the web to share CreativeAI links: Reddit, Twitter etc. The defining factor with all these outlets is the community. CreativeAI.net focuses solely on one topic, CreativeAI. Thus, it attempts to attract a multidisciplinary community of researchers and enthusiasts, sharing knowledge and learning from each other. The goal is to help connect a movement. It is a celebration of creativity and an invitation for play!

Subscribe to our Newsletter

Chapter 01

by Roelof Pieters & Samim Winiger

>>>> Find us at @CreativeDotAI <<<<

Introduction

We live in times, where science fiction authors are struggling to keep up with reality. In recent years, there has been an explosion of research and experiments that deal with creativity and A.I. Almost every week, there is a new bot that paints, writes stories, composes music, designs objects or builds houses: Artificial Intelligence systems performing creative tasks?

Our research started by wondering about this phenomenon and playfully experimenting with it. This lead to an in-depth investigation, of what we call “CreativeAI”. This document is the first chapter of our adventure into CreativeAI, aiming at establishing a backstory and language we can use to talk about this intricate subject. Our initial intuition was, that creativity is a central force throughout human history — and is currently evolving in interesting ways. In our attempt to understand this phenomenon, we think about creativity and technology in a structured way. We focus on emerging creation patterns, Assisted Creation and Generative Creation, and argue that they are leading to the Democratization and Escalation of Creativity.

The goal of this project is to find a set of guiding principles, metaphors and ideas that inform the development of a CreativeAI praxis, new theories, experiments, and applications. To explore this space, we investigate history and technology, construct a narrative and develop a vision for a future where CreativeAI helps us raise the human potential.

INDEX

1. Creativity
2. Assisted Creation
3. Generative Creation
4. Conclusion
5. Authors & Acknowledgments
6. References

>>>> Subscribe to our Newsletter <<<<

Interested to help out bringing AI to studios and agencies in creative industries around the world? We’re hiring at http://join.creative.ai/

1. Creativity

It is central to the human condition and takes many forms in our daily activities, yet defining creativity is challenging. This section provides a selective overview of historical, theoretical and technological metaphors for creativity, relevant for CreativeAI.

Ancient cultures lacked our concept of creativity, including thinkers of Ancient Greece, China, and India [1]. They viewed creativity as a form of discovery. The rejection of creativity in favor of discovery would dominate the west until the Renaissance. By the 18th century, mention of creativity became more frequent, linked with the concept of imagination [2]. In late 19th century, theorists such as Walls, Wertheimer, Helmholtz and Poincaré [3] began to reflect on and publish their creative processes, pioneering the scientific study of creativity.

The scientific study of creativity produced many theories, models and systems throughout the 20th century: philosophical, sociological, historical, technical and practical. While defining creativity in objective terms was and still is challenging, the systematic study of creativity and its enabling factors allowed industries such as advertising, architecture, design, fashion, film and music to adopt creative processes rapidly and reproduce them at scale.

Science, technology and creativity have a long, intertwined history. Selecting which metaphors to explore is an important research decision. We explore three metaphors: Augmented Creativity, Computational Creativity and Creative Systems.

Augmented Creativity

In “As We May Think” (1949), Vannevar Bush imagines the “memex”, a desk-like device where people could search through a library of articles through a series of switches [4]. While entirely mechanical, Bush describes a device that features hyperlinked text, aggregated notes and bookmarks all extending human capacity to research and process information: The web.

The article inspired a young Douglas Engelbart to quit his job and attend graduate school at UC Berkeley [5]. At Berkeley he wrote a paper, published in 1962 titled “Augmenting the Human Intellect: A Conceptual Framework”. In it, Engelbart, influenced by Bush’s memex concept, wrote about a “writing machine [that] would permit you to use a new process of composing text (..) You can integrate your new ideas more easily, and thus harness your creativity more continuously (..) This will probably allow you to devise and use even-more complex procedures to better harness your talents…” [6].

Engelbart did not only provide a vision of interacting with a computer system but he had a guiding philosophy [7]. He believed that computers can be used to create an extension for the ways we do thinking, representation and association in our minds [8]. Engelbart’s vision was not just to automate processes but to multiply the power of people and collaborators by creating systems that augment our intellect, humanity and creativity. His goal was to raise the human potential [9].

Ivan Sutherland, a student of Claude Shannon, who in turn was a student of Vannevar Bush, built a working system inspired by the Memex already in 1963. His seminal PhD project “Sketchpad” [10] is considered to be the ancestor of modern computer-aided design (CAD) programs [11]. It demonstrated the potential of interactive computer graphics for technical and creative purposes.

Only a few years later, Engelbart’s Stanford Augmentation Research Center (ARC), invented a range of technologies, still widely used today. Among them, video conferencing and the mouse [12]. Simultaneously, John McCarthy had founded the Stanford Artificial Intelligence Laboratory (SAIL). McCarthy’s group wasn’t concerned with augmentation, but wanted to reproduce the human intelligence electronically [13]. Engelbart’s Center and McCarthy’s Laboratory brought together Ph.D.s, hardware and software hackers, and high school students, including Steve Wozniak and Steve Jobs [14], to experiment collectively.

When Xerox (a paper company) decided to fund its Palo Alto Research Center (PARC) in 1970 [15], it quickly attracted ARC and SAIL veterans eager to work on personal computing, user interface design and graphics. This facility developed a number of innovations like ethernet and pioneered a new metaphor for doing creative work with computer systems: the Desktop. Soon after Xerox opened its center, a more informal but equally important outlet and movement emerged to explore computers: The Homebrew Computing Club. Homebrew attracted a mix of antiwar activists, makers and computer scientists. Ultimately, Dozens of companies, including Apple and Microsoft, and technologies such as the Personal Computer (PC) would come out the Homebrew movement [16].

Computational Creativity

Already in 1950, Claude Shannon was able to approximate proper English grammar and generate new sentences using computational methods [17]. Such early research in “computational creativity” lead to an interdisciplinary dialog, exploring the use of computational approaches for creative problems.

Starting the early 1960s, researchers at Bell Labs were pioneering the use of computers for creativity. In a series of breakthrough experiments, they generate graphics, animations and art [18] with early computer systems. One of the most active researchers was Michael Noll. In 1970, he made a call to action: “What we really need is a new breed of artist-computer scientist” [19]. Noll’s call was soon echoed by artists and musicians, such as Brian Eno. Already in 1975, Eno was using algorithmic and generative principles to compose music — later describing his work as “using the technology that was invented to make replicas to make originals” [20].

A further milestone was set in 1979 by Benoit Mandelbrot [21] with the discovery of the Mandelbrot set. He was the first to use computer graphics to display fractal geometric images. By doing so, he was able to show how visual complexity can be created from simple rules. Fractals had a profound effect on our perception of creativity and machine. It led many to ask “can a computer/algorithm be creative?” and inspired scientists, artists and engineers to experiment with creativity.

Video games pioneered the industrial application of computational creativity. Around 1978, games started to make extensive use of procedural systems to define game maps and character behaviours [22]. Such methods allowed for the development of complex gameplay without having to spend excessive time creating games. Games such as Simcity [23] by Will Wright developed these concepts further with playful interactive simulations of complex systems.

Since the 1980s, focused research in industry and academia has led to the formalisation of computational creativity as a scientific discipline [24]. At the same time, a wide range of fields — such as computer science, architecture and design — started intensely experimenting with computation creatively. Finding a single definition for computational creativity is challenging, yet many have tried. A currently often cited definition is: “create computations which — if they were made by humans — would be deemed creative” [25].

Today, interest in creativity from an A.I perspective has begun to blossom, with yearly conferences, schools and PhD programs dedicated to computational creativity [26]. A steady surge of ideas and techniques, that are at least computationally creative in intention, have moved into the mainstream: A.I characters, artificial musicians, journalist bots, generative architecture and neural nets that “dream”. While such systems are nowhere near human capabilities, they are actively being used in culture, industry and academia to create outputs that are increasingly met with great curiosity by the public. In many areas, systems are making the leap from experimentation to production, leading to new creative processes and outputs.

Creative Systems

After World War II, the United States enjoyed a period of euphoria. The Allied Powers had triumphed — seemingly through science, technology and systems thinking. In this environment, the Josiah Macy Jr. Foundation organized a series of conferences from 1946 to 1953 “on the workings of the human mind” [27], later titled “Cybernetics”. The aim of the conferences was to promote meaningful communication across scientific disciplines and restore unity to science [28]. It included people like J.C.R. Licklider, Margaret Mead, Heinz von Foerster, John von Neumann, Claude Shannon and Norbert Wiener.

Inspired by the conference, in 1948 Wiener published his seminal work “Cybernetics: or Control and Communication in the Animal and the Machine” [29] and Shannon published “A Mathematical Theory of Communication” [30]. Such works laid the foundation for today’s information age by providing a scientific theory for concepts such as “information”, “communication”, “feedback” and “control”.

Wiener defined cybernetics as the science of adaptive, feedback-based control [31]. The name comes from the ancient greek word for steersman. Cybernetics takes the view that control in complex environments must be conversational. It requires not just action but also listening and adaptation: To steer a boat across a lake, you have to use your tiller and sails to adjust to changing winds and currents. The cybernetic model of control is circular, decisions depend not only on how well people carry out their intentions but also on how the environment responds.

An early link between Cybernetics and Creativity was made in 1968 with the exhibition “Cybernetic Serendipity”, at the Institute of Contemporary Arts in London [32]. The show explored connections between creativity and technology. Artists such as Gordon Pask and Nam June Paik were using systems to generated music, poetry, movies, paintings and computer graphics [33].

This new spirit of creation was addressed by Buckminster Fuller in his notion of the “comprehensive designer”, which he describes as “an emerging synthesis of artist, inventor, mechanic, objective economist and evolutionary strategist” [34].

In the following decades, cybernetic ideas would profoundly impact thinking in fields such as business, politics, art, design and architecture [35]. As Pask noted, “architects are first and foremost systems designers,” but they lack “an underpinning and unifying theory… Cybernetics is a discipline which fills the bill” [36]. By systematically integrating context and relationships, cybernetics pushed creation & design beyond its object-based approach.

While cybernetics went out of fashion in the 1970s, its legacy lives on in fields such as Control Theory and Complex Systems Studies, Interaction Design and Design Thinking [37]. Today holistic approaches, that attempt to combine technological, human and social needs, are cited in many fields. Inspired by cybernetics, creative systems thinking has found “surprising” application in areas such as software (agile, open-source), management (Google 20% time), labour (Uber / Lyft) and resource allocation (algorithmic trading / amazon).

Examples / Media

The following a selection of projects from Augmented Creativity, Computational Creativity and Creative Systems research. The aim is to provide visual context and show progress over time.

1. Computer Interaction Input Device (1968)
2. Mouse — Mass market Input Device (1982)
3. Touch Screen — Input Device (1982)
4. Mass Market Voice Control (2011)
5. Mass Market Virtual Reality Headset (2016)

1. Sketchpad — Computer Aided Design (1963)
2. Autocad — Mass market CAD Tools (1982)
3. Maya — Mass Market 3D CAD
(1998)
4. Generative Bicycle — 3D Printed (2015)
5. Generative Dress — 3D printed (2016)

1. Tetris — Procedural Gameplay (1984)
2. Simcity — Simulation of Complex Systems (1989)
3. Spore — Procedural Game Characters (2008)
4. Minecraft — Procedural 3D Worlds (2011)
5. No Man’s Sky — Universe Simulation (2016)

1. Hypercubes — Computer Graphics/Animation (1968)
2. Fractals — Complexity from Simple Rules (1980)
3. Mandelbulb — 3D Fractals
(2009)
4. DeepDream — Generative Painting (2015)
5. NeuralPatch — Generative Style Transfer — (2016)

— Intermission —

In the previous section, we explored the long, intertwined history of science, technology and creativity. In this process we investigated three metaphors for creativity: Augmented Creativity, Computational Creativity and Creative Systems. In the following sections, we consider how these metaphors have developed further and extrapolate two main categories of activity today: Assisted Creation and Generative Creation.

2. Assisted Creation

Humans have used tools to extend their creative capabilities since the stone age — adapting to changing needs. While mastering creative skills used to be attainable only for few, assistive systems are making creativity more accessible. This section presents three generations of assisted creation systems and explores how they democratise and escalate creativity.

Inspired by Engelbart’s vision from the 1960s, countless scientific papers and experiments explored how to assist humans to perform “creative” tasks — or as researcher Ben Shneiderman defined it, technologies that allow more people “to be more creative more of the time” [1]. Such research, coupled with the emerging PC revolution, allowed companies like Apple and Lotus to build early digital applications for creative tasks. Ultimately, this movement led to the founding of companies such as Autodesk (1979) [2] and Adobe (1982) [3], that exclusively focused on building tools and systems that enable creativity.

Industry pioneered the development of first generation assisted creation systems in the 1980s: Photoshop, Autocad, Pro-Tools, Word and many more. First generation systems mimic analogue tools with digital means [4]. The human’s full attention is required to drive the creative process: Feedback is slow and assistance limited. Yet, such tools allowed expert and non-experts alike to be more creative, which lead to a flood of new creative processes and outputs.

The camera Autofocus, invented by Leica in 1976 [5], is an early example of a second generation assisted creation system. In these systems, humans and machines negotiate the creative process through tight action-feedback loops. The machine is provided with greater agency so control can be shared. Decisions are made collaboratively with the system. Second generation systems are ubiquitous today. They are being used in production across cultures and industries.

Autocorrect, invented in 1991 by Dean Hachamovitch at Microsoft [6], changed how millions of people write — Autotune, invented in 1998 by Andy Hildebrand at Exxon [7], transformed how music is made. The impact such systems had on creativity is hard to measure, yet clearly significant: By lowering the bar of mastery, assisted creation systems empowered experts and non-experts alike to shift their attention to higher level issues, perform complex creative tasks more reliably and experiment quickly. While such systems are not with out their risks and complications [8]- ultimately, they enable us to be more creative, more of the time.

Assisted Creation 3.0

Second generation systems are often limited and limiting: Negotiation for control is blunt and interactions not fine grained. Due to such limitation, widely used tools such as autocomplete have a mixed reputation. A set of new ideas and techniques, coming from diverse research disciplines, promise to overcome previous limitations. We define them as Third generation assisted creation systems (AC 3.0). A shared vision is to design systems that negotiate the creative process in fine-grained conversations, augment creative capabilities and accelerate the skill acquisition time, from novice to expert. Third generation assisted creation principles are finding practical use across an expanding range of creative tasks.

To name a few examples:

• Assisted Drawing helps illustrators to draw, by correcting strokes.
• Assisted Writing helps authors to write, by improving text style.
• Assisted Video helps directors to edit, by fine-tuning movie cuts.
• Assisted Music helps musicians to make music, by suggesting ideas.

Describing the breadth of ongoing research in a few examples is challenging, as there are many ideas and domains to explore. To track assisted creation, we analyzed recent research publications across many organisations with the help of machine learning, graph theory and visualization. Judged on quantitative measures (publications and experiments), assisted creation research and use is on the rise across creative disciplines. Notably, Machine Learning (ML) and Human Computer Interaction (HCI) are contributing a steady stream of research, relevant for the design of assisted creation systems. Together, ML and HCI are providing us with a conceptual framework for machine intelligence in a human context.

Already in 2011 Rebecca Anne Fiebrink, HCI/ML researcher at Goldsmith University, fittingly asked: “Can we find a use for machine learning algorithms in unconventional contexts, such as the support of human creativity and discovery?“ [9]. In the years since, Fiebrink’s call has been taken up by a multidisciplinary community: a wide range of new ideas, theories, experiments, approaches and products are being explored and developed. Ongoing HCI/ML research offer us new possibilities and metaphors for the design of assisted creation systems.

Democratisation and Escalation

By researching Assisted Creation, we recognise emerging trends, with implications for creativity: 1. Assistive Creation Systems are making a wide range of creative skills more accessible. 2. Collaborative platforms, such as Online Video and Open Source, are making it easier to learn new creative skills. As these trends are increasingly converging, they are accelerating the skill acquisition time from novice to expert. This is leading to a phenomenon we have named “the democratisation of creativity”. We explore these trends further and extrapolate a vision.

TREND 1: Creativity is becoming more accessible.

While having a photo studio or music recording studio at home was but a dream for a 1980s creator, in today’s world it’s one click away. Such trends, observable for many creative tasks, are empowering non-experts and experts alike to be more creative, more of the time. One could say, the price of creation is falling. In this trajectory, a key challenge has been the “high barrier to entry” [10] for those without specific skills or talents. Today, Assisted creation systems are increasingly lowering this “high bar” by actively guiding creative processes and bootstrapping the learning of new skills.

TREND 2: Collaboration is becoming more accessible.

Already in the 1960’s, Engelbart’s vision was not only about enhancing individuals: He wanted to augment the collective intelligence and creativity of groups, to improve collaboration and group problem-solving ability. With the rise of collaboration and social software, and a deeper theoretical understanding of how groups can use technology to self-organize and cooperate, systems are emerging that can make groups effectively more creative. A key notion is that creativity is a collective process that can be strengthened through technology, but goes beyond just technological means. Human capabilities and tool capabilities have to be raised in sync.

The escalation of creativity

By projecting these trends into the (near) future, we can start to imagine a scenario we call “the escalation of creativity”: a world where creativity is highly accessible and anyone can write at the level of Shakespeare, compose music on par with Bach, paint in the style of Van Gogh, be a master designer and discover new forms of creative expression. For a person who does not have a particular creative skill, gaining a new capability through assisted creation systems is highly empowering. If creative tasks can be master on-demand and access is democratically shared, age-old notions such as “expert” or “design” are bound to be redefined. Further, this escalation can lead us to scenarios such as using creativity as means of empathic communication — at scale.

Even though such scenarios are currently fiction, thinking about the implications of the democratisation and escalation of creativity, influences today’s design decisions. Creating systems that are respectful of cultural practices, support different types of creativity, are responsive to human needs, provide feedback transparently and are ethically grounded is proving to be highly challenging already now.

Automation or Augmentation

A first question we should ask ourselves when talking about the democratisation and escalation of creativity is “Are we designing tools that empower us or autopilots that replace us?”. Such questions have been negotiated in a global discussion, starting over 5000 years ago with the use of oxen in agriculture [11]. History shows us that any technology has feedback dynamics and momentum, or in the words of Marshall McLuhan: “First we shape our tools, thereafter they shape us” [12]. Nonetheless, we see technology not as a primary force of nature: Human decisions and actions play a key role — negotiated through culture, politics & power.

A second question to address is “what does automation mean?”. Our current understanding of automation is heavily influenced by ideas from the industrial revolution: mass producing goods with mechanical butlers. While concerns about mass unemployment due to certain types of automation have to be taken serious, automation is not inherently bad; some “jobs” might be better left to machines, as they can be inhumane and wasteful of human potential. Shifting the discussion to one of human potential and capability — reflecting on our strengths and weaknesses, needs and dreams — allows us to reframe fears of automation as opportunities for augmentation.

Augmentation is not the same as automation: Where automation promises to “free us from inhumane tasks”, augmentation aims at strengthening our capabilities. It is the notion of raising the collective human potential, not replacing it. To analyse this notion further, we refer to a framework introduced by NASA, for thinking about autonomy: The H-Metaphor [13]. It proposes to view our interactions with systems more as we do with horses, instead of butlers.

Think of a rider on a horse: If a rider uses deliberate movements, the horse follows exactly. As the control becomes vaguer, the horse resorts to familiar behaviour patterns and takes over control. Being able to “loosen or tighten the reins” leads to smooth ebb and flow of control between human and horse, rather than instructions and responses. Considering feedback and control is key.

HCI/ML Researcher Roderick Murray-Smith suggests using the H-Metaphor and control theory when thinking about interface dynamics. He predicts:

“Future devices will be able to sense much more on and around them, offering us more ways to interact. We can use this to let go sometimes and be casual about our interactions” [14].

Having the ability to interact with systems casually, and let go of control at times, promises substantially improved forms of human-machine and human-human collaboration. By combining human intuition with machine intelligence, shared control principles lets us imagine new creative processes, not possible independently by either human or machine. Such principles promise to make creativity more accessible and raise our collective potential.

While a range of dystopian outcomes can easily be imagined, we deliberately choose to explore a vision that focuses on opportunities, not fears. In order to prevent bleak future scenarios, complex metaphorical and ethical questions can not be an afterthought, but are an opportunity for collaborative exploration and a call to action for collective decision making.

Examples / Media

The following is a selective overview of ongoing assisted creation research, experiments and products, across a range of creative tasks / disciplines.

Assisted Photography

1. Assisted Photo Enhancement (2016)
2. Assisted prediction of photo memorability (2015).
3. Assisted Categorisation and Tagging of Photos (2015).
4. Auto Photo Colorisation (2016)
5. Realtime Smile and Emotion Detection (2015).

Assisted Drawing

1. Assisted Handwriting Beautification (2013).
2. Assisted Freehand Drawing with Real-time Guidance (2013).
3. Autocomplete hand-drawn animations (2015).
4. Animating drawings with face recognition (2015).
5. Robotic Handwriting Assistance (2013).

Assisted Read/Write

1. Assisted CV text creation and optimisation (2015).
2. Auto-respond to email (2015)
3. User guided / Automatic summarization of text (2015).
4. Text Style transfer from English to Shakespeare (2015).
5. Word Processor with a Crowd Inside (2010).

Assisted Music

1.Music style and harmony transfer, genre to genre (2014).
2. Composing Music with Augmented Drawing (2009).
3. Assisted Musical Genre Recognition (2013).
4. 909 Drum-machine that learns from behaviour (2015).
5. Assisted Robo Guitarist (2013).

Assisted Design

1.Learning Visual Clothing Style (2015).
2. Assisted Design of 3d models by merging shapes (2015)
3. Learning Perceptual Shape Style Similarity (2015).
4. Parsing Sewing Patterns into 3D Garments (2013).
5. Shape Shifting Table (2015).

Assisted Experiments

1. Wearable Assisted Text-Reading Device (2015).
2. Pain Visualization through patient text (2013).
3. Hair Modeling with DB (2015).
4. Assisted Ethical Decision Making, with a fan (2015).
5. Text Entry for Novice2Expert Transitions (2014).

Assisted Community

1. Real Time video stream of creative processes (2015).
2. Massive Open Course (2012)
3. Large scale Open Source Collaboration (2008).
4. Creative Process Question Answer Sites (2010).
5. Zero Cost Creative Content Distribution (2007).

Assisted Culture

1. Assisted drumming with third robot arm (2016).
2. Assisted Vending, selects drinks based on looks (2016).
3. Computer Ballet (2016).
4. Assisted Karaoke Singing with Face Swap (2016).
5. Pingpong Assistant with AR Glasses (2015).

3. Generative Creation

Our abilities to represent complex creative problems is increasing. A fundamental shift in perspective is allowing us to revisit many creative problems. The following section presents generative creation and explores how it democratises and escalates creativity.

Representation has played a pivotal role throughout human history. Ever more detailed representational systems have allowed us to communicate complex phenomena in understandable terms — to organise information, manage problems and make informed decisions. Since the invention of writing, representational strategies have evolved substantially. From the inclusion of measurement in the early 16th century [1], to the adoption of perspective drawing in the Renaissance [2]: New forms of representation have lead to revolutions in science and technology.

Abstraction strategies, such as drawing and writing, try to represent big ideas with highly limited means. They force humans to keep all the moving parts in their heads. As Matt Jezyk (Autodesk) suggests, such tools were invented in the age of documentation, where the bandwidth to represent problems was low [3]. Jezyk describes the 20th century as the age of optimization: New techniques, such as simulation expanded our representational bandwidth significantly and allowed many disciplines and industries to adopt reproducible abstraction methods at scale.

Historically, the use of simulations were largely isolated in different fields. 20th century studies of systems theory and cybernetics combined with the proliferation of computers led to a more unified, systematic perspective: the age of models. A Model is a high-bandwidth, computational representation of reality. The model represents the system — its characteristics and behaviors — whereas the simulation represents the operation of the system over time. This new representational paradigm does not rely on abstraction methods but tries to make “things that behave like the thing they represent”.

Models give us an infrastructure for representing the overall problem. They help us to understand complex interconnected issues and develop a deeper understanding of inherent logic and relationships of parts. While modeling techniques go back till at least the early 1940s (Nuclear-bomb simulation) [4], it was the exclusive domain of experts and inhibitively expensive. Today, modeling and simulation methods are becoming highly accessible and cheap.

We argue that accessible modeling techniques are allowing us to negotiate a wide range of creative problems from a higher level perspective and create differently. We explore this emerging pattern — culture, technology and implications — and name it the generative age.

The Generative Age

Already in the 1960s, Engelbart pointed to the impact digital technologies have on our representational ability: “We can represent information structures within the computer that will generally be far too complex to study directly” [5] . A conceptual link between these new representational abilities and creativity was made in the early 2000s by designers like Patrik Schumacher, co-founder of Zaha Hadid Architects. He describes an “ontological shift”, from the platonic ideal shapes of the past 5000 years, to new computational “primitives“ [6].

What he was alluding to, is a fundamental shift in perspective, from 3D to nD: While the renaissance age gave us the ability to represent reality from a three dimensional perspective (3D), the generative age enables us to represent (model) complexity and see (infer) reality from a probabilistic, or high dimensional perspective (nD). Inspired by such ideas, influential design manifestos [7], books [8], and software [9] were published, laying the foundation for a new movement: Generative Design.

Michael Hansmeyer, an architect, describes Generative Design as “thinking about designing not the object but a process to generate objects” [10]. He is implying a shift from object to process — from certainty to probability — suggesting that instead of designing one “artefact”, we use computational models to design processes that generate infinite “artefacts”.

Essentially, Generative Design is an umbrella term — describing ongoing research and developments in diverse fields, ranging from design, architecture, industrial design to machine learning. A shared vision is to empower human designers to explore a greater number of design possibilities from a new perspective and lower the time between intention and execution. In the generative age, the cost of creating diversity and complexity is falling. This allows us to create an order of magnitude more elaborate form and function. For example: bicycles that are mass-customizable to people’s individual taste, while using a fraction of the material traditionally required. While generative approaches are not constrained to any particular field, notably architecture and more recently design have been among the first disciplines to systematically take hold of these approaches, as illustrated in the following examples.

While Generative Models have been used for creative applications since the 1970s (procedural game), recent research advances — driven notably by Machine Learning and Deep Learning — are leading to a quantitative and qualitative leap in generative modeling capabilities. Today, new models are released practically every week. Research projects with acronyms such as VAE [11], DRAW [12], VRNN [13], GAN [14], DCGAN [15], LAPGAN [16] and GRAN [17] are allowing us to model complexity with greater resolution and apply modelling techniques to a wider range of creative problems.

The application of generative machine learning models to creative tasks is a recent development, yet it is already leading to the discovery of new primitives for creation: Design building blocks that are applicable across many creative domains. Such creative generative models have been successfully used to generate fashion items, paintings, music, poems, song lyrics, journalistic news articles, furniture, image and video effects, industrial design, comics, illustrations and architecture, to name just a few of its applications. See “Examples” section for a selection of projects.

Generative models let us explore data in unprecedented ways. To give an example, imagine a chair: we can represent its characteristics, such as color, height or style, as dimensions in high-dimensional information spaces. This space can be filled with data about millions of chairs. Chairs with similar characteristics are mapped in vicinity of each other. This creates a chair model, which can be explored and visualized.

Such high-dimensional topologies allow us to easily retrieve information, explore data and ask questions about relationships, logic and meaning: e.g. “Show me all chairs that are red and tall”. Further, they allow us to make predictions and infer new characteristics: e.g. “Show me all chairs, that are similar to chair A and B but unlike chair C”. Finally, we can use high-dimensional spaces to generate new objects: e.g. “Make me a chair that resembles a car, and is comfortable to sit in”.

Generative models invite designers to play with data and generated infinite imaginative variations and solutions to creative problems. By having powerful tools to explore, optimise and test creative design ideas rapidly, we computationally maximise the opportunity for serendipity. While generative models can be used to perform classical creative tasks efficiently, additionally they open up a range of new creative capabilities, incomparable with classical methods.

Artificial Serendipity: Systems that maximise the opportunity for serendipity.

Recent advances in machine learning make it possible to include data from different “modalities” in a single model. It enables us to translate between modalities. The key insight is that all forms of information can be encoded in a shared information space. Early research into multimodality has lead to a set of widely adopted systems: “Auto Translate” [18] lets us translate from one language to another, “Speech2Text” [19] transcribes audio to text. Multi-modal machine learning is allowing for more complex scenarios, which go beyond simple translation of data: Generate Images from Text [20], Text from Videos [21], Music from Movement [22], 3D shapes from shopping data, etc. We call this:

Artificial Synesthesia: Systems that enable inter-sensory experiences.

Democratisation and Escalation

The Generative age gives us a new canvas for creativity, which we have only just started to explore. While it is hard to predict where these developments will take us, emerging trends are worth investigating — as their impact can already be felt. By extrapolating these developments and thinking about their implications, we arrive at a scenario we call “the democratisation and escalation of creativity”.

We explore this notion further and describe four trends:

1. Generative Perspective:
For the first time in human history, we can create from a blended, or generative perspective — as it mixes elements of the collective human perspective, machine perspective and individual perspective. It gives us the ability to transcend creative constraints, such as habit, socialisation and education, and create objects which are altogether new.

2. Generative Predictions:
Take the concept of recommendation, personalization and customisation and apply it to the creative process. The vision is to have systems that suggest potential next “actions”, allow people to casually adjust aspects of designs according to personal needs and histories, and enable us to playfully discover creativity.

3. Generative Markets:
In the future, generative models might be shared in an open collaborative model marketplace (OCMM). While current marketplaces allow us to trade artefacts/products, generative markets will facilitate the sharing of recipes to create unlimited new artefacts. In essence, think of it as GitHub for open-source creativity.

4. Generative Manufacturing: Emerging digital manufacturing techniques, such as 3D printing, combined with generative systems used to create physical objects. Early signs of such trends can be observed in things such as Shapeways, Kickstarter and the “maker movement”. It’s starting to redefine the relationship between creation, production and consumption.

When projecting such scenarios even further into the future, we arrive at the realisation that Generative Creation has profound implications for fields like technology, manufacturing, resource allocation, economics and politics. Already today, Generative Creation methods are leading to the democratisation of creativity in many areas. By lowering the time between intention and realisation, Generative Creation is leading to an escalation of new “artefacts” — forms, functions and aesthetics. It allows us to explore what lies beyond the artefact.

Combined with new manufacturing techniques, generative creation is redefining concepts such as production, consumption, labour and innovation. As current economic models are largely built around the notion of “artefacts”, a renegotiation of fundamentals is foreseeable. While predicting the future of the demo-cratisation and escalation of creativity is impossible, thinking about narrative, opportunities and implications, informs today’s decisions and visions. Or in the words of Robert Anton Wilson:

Examples

The following is a selective overview of generative creation research, experiments and products, across a range of creative tasks / disciplines.

Generative Experiments

1. Generating Flora and Fauna (2015).
2. Generating Chairs, Tables and Cars (2015).
3. Generative Font Design with Neural Networks (2015).
4. Generative Manga Illustration (2015).
5. Generating Faces with Manifold Traversal (2015).

Generative Design

1. Semantic Shape Editing Using Deform Handles (2015).
2. Generative Motorcycle swingarm Design (2015).
3. Generative airplane partition design (2015).
4. Generative Data-Driven Shoe Midsole Design (2015).
5. Generative Jewellery design (2015).

Generative Text

1. Generating Stories about Images (2015).
2. Generating Sentences from a continuous space (2015).
3. Generative Journalism (2010)
4. Generating Cooking recipes with Watson (2015).
5. Generating Clickbait Web content and site (2015).

Generative Serendipity

1. Exploratory Modeling with Collaborative Design (2015).
2. Generative Music Score Composition with RNNs (2015).
3. Messa di Voce, Generative Theater (2003).
4. Generative Image Style Transfer (2015).
5. Interactive Neural Net Hallucinations (2015).

Generative Architecture

1. Generative Columns Design and Manufacturing (2010).
2. Generating House 3d Models with House agents (2012).
3. Heydar Aliyev Center (2012).
4. Generative Biological inspired form (2015).
5. Francis Bitont on 3D printing (2015).

Generative Design

1. Generative strategies for welding (2015)
2. Generative Car Chaise Design (2016).
3. Generative mass customized knitwear (2016)
4. Generative mass customized T-shirt and bags (2016)
5. Generative Lampshade based on room 3D scan (2015).

Generative Games

1. Generative Creation of Universe (2014).
2. Texture Synthesis (2015).
3. Generative Character Controls (2012).
4. Generative Game Map and Characters (2013).
5. Generative enemy manager (2010).

Generative Synesthesia

1. Synesthesia Mask Lets You Smell Colors (2016).
2.Cross-modal Sound Mapping Using ML (2013).
3. Expressing Sequence of Images with Sentences (2015).
4. Generative Graffiti, adapting to Music (2016).
5. Music to 3d Game (2001).

Conclusion

Our research journey began with a series of experiments — playfully exploring the space between creativity and A.I. It led to an in-depth investigation into creativity, which reinforced our initial intuition that creativity is a central, evolving force throughout human history. New metaphors such as Augmented Creativity, Computational Creativity and Creative Systems allowed us to approach creativity from new perspectives and explore how it intersects with technology.

During this journey, we have tried to think about creativity and technology in a structured way. This has allowed us to recognize, analyze and define emerging creation patterns. We focused on two general types: Assisted Creation and Generative Creation. Together, these patterns are leading to a vision we call the democratization and escalation of creativity: A world where creativity is highly accessible, through systems that empower us to create from new perspectives and raise the collective human potential. Through our research, we learned to appreciate creativity as an ever evolving, driving force of humanity and as a wide open frontier for interdisciplinary research and development.

A primary goal of this research project was to find a set of guiding principles, metaphors and ideas, that inform the development of future theories, experiments, and applications. By combining different domains into one narrative, we formulate a new school, or praxis for creativity: CreativeAI. Its desire is to explore and celebrate creativity. Its goal is to develop systems that raise the human potential. Its belief is that addressing the “what” and “why” is as important as the “how”. Its conviction is that complex ethical questions are not an afterthought, but an opportunity to be creative collectively.

Finally, CreativeAI is a question, rather than an answer. Its only demand is more collaboration and creativity. It is an invitation for play!

“The creation of something new is not accomplished by the intellect but by the play instinct acting from inner necessity. The creative mind plays with the objects it loves” — Carl Jung

Interested to help out bringing AI to studios and agencies in creative industries around the world? We’re hiring at:

http://join.creative.ai/

About the Authors

Roelof Pieters
@graphific
roelof@creative.ai

Samim Winiger
@samim
s@artificialexperience.com

Special thanks to: Boris Anthony, Jake Witlen, Hendrik Heuer, Kiran Varanasi, Francis Tseng, Mark Riedl, Jack Clark, Alex J. Champandard, Gene Kogan, David J.Klein, Simone Rebaudengo, Matthieu Cherubini, Saurabh Datta, Max Niederhofer, Dave Ha, Kyle Kastner, Mattias Östmar, Melisande Middleton, Tor Sanden, Mami Ebara.

Further, thanks to people on twitter for inspiration, conversations and helpful suggestions!

>>>> Subscribe to our Newsletter <<<<

>>>> Find us at @CreativeDotAI <<<<

Footnotes

1. Creativity

1. Albert, R. S.; Runco, M. A. (1999) “A History of Research on Creativity”. In Sternberg, R. J. Handbook of Creativity, Cambridge University Press.
2. Tatarkiewicz, W. (1980) A History of Six Ideas: an Essay in Aesthetics. Translated from the Polish by Christopher Kasparek, The Hague: Martinus Nijhoff; Runco, Mark A.; Albert, Robert S. (2010) “Creativity Research” In James C. Kaufman and Robert J. Sternberg, The Cambridge Handbook of Creativity, Cambridge University Press.
3. Wallas, G. (1926) Art of Thought, New York, Harcourt, Brace and Company; Wertheimer, M (1945) Productive Thinking. New York: Harper; Poincaré, H. (1952) [1908] “Mathematical creation”. In ed. Ghiselin, B. The Creative Process: A Symposium. Mentor; Helmholtz, H. v. L. (1896) Vorträge und Reden (5th edition). Friederich Vieweg und Sohn.
4. Bush, V. (1945) As We May Think, Atlantic Monthly 176 (July 1945) pp. 101–108 [pdf].
5. Moggridge, B. (2007) Designing Interactions. Cambridge, MA: The MIT Press. p 30–31.
6. Engelbart, D. C. (1962) Augmenting human intellect: A conceptual framework, Stanford Research Institute, Tech. Rep., October 1962, p 5 [online text]
7. In December 1950, after three years of working at a steady job and getting engaged, Douglas Engelbart realising that there was more to life, asking himself, “How can my career maximize my contribution to mankind?”. Recounted in: Bardini, T. (2000) Bootstrapping: Douglas Engelbart, Coevolution, and the Origins of Personal Computing, Stanford University Press, p 7. And O’Brien, T. (1999) “From the archives: Douglas Engelbart’s lasting legacy”, Mercury News, at [siliconvalley.com]. This same realization is recounted in Markoff, J. (2005) What the Dormouse Said: How the Sixties Counterculture Shaped the Personal Computer Industry, New York: Penguin Group. p 8.
8. This point is eloquently described by Alan Kay at the MIT/Brown Vannevar Bush Symposium, on October 12–13, 1995, to celebrate the 50th anniversary of Vannevar Bush’s seminal article “As We May Think”, [Video] (recounted at: 38m 44s).
9. Gillespie, T. (2014) Media Technologies: Essays on Communication, Materiality, and Society. Mit Press Ltd, pp 213–215.
10. Ivan Sutherland’s PhD thesis was reprinted in 1980 as: Sutherland, Ivan Edward (1980), Sketchpad: A Man-Machine Graphical Communication System, New York: Garland Publishers. An electronic edition was published in 2003 [pdf].
11. Sears, Andrew; Jacko, Julie A. (2007). The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, Second Edition. CRC Press. p 5.
12. Both mouse and video conferencing made their debut in what was come to be known in a demonstration by Doug Engelbart which came to be known as “The Mother of All Demos”: Engelbart, Douglas C.; et al. (1968–12–09). “SRI-ARC. A technical session presentation at the Fall Joint Computer Conference in San Francisco”. NLS demo ’68: The computer mouse debut. Engelbart Collection (Menlo Park, CA: Stanford University Library) [Video recording, annotations, and full transcript].
13. Markoff, What the Dormouse Said, pp 80–109.
14. ibid, pp 254–287.
15. A detailed history of Xerox, PARC, and other intersections of business and technology, and the creation of “Silicon Valley” is: Rao, A. and Scaruffi, P. (2011) A History of Silicon Valley: The Greatest Creation of Wealth in the History of the Planet. Redwood City: Omniware.
16. Stephen Wozniak (1984) “Homebrew and How the Apple Came to Be” In: Steve Ditlea (ed) Digital Deli: The Comprehensive, User-Lovable Menu of Computer Lore, Culture, Lifestyles and Fancy. [text online]. A good history of the Home Computer Club can also be found in John Markoff, What the Dormouse Said. And in: Freiberger, Paul; Swaine, Michael (2000) Fire in the Valley: The Making of the Personal Computer, 2th ed, McGraw-Hill; An adaption of “Fire in the Valley” on the History of Apple and Microsoft led to the documentary “Pirates of Silicon Valley” (1999) by director Martyn Burke.
17. Shannon, C. E. (1951) “Prediction and Entropy of Printed English” In Bell System Technical Journal 30 (1): 50–64 [full text at archive.org].
18. An excellent historical overview is given in: Holbrook, B.D. and Brown, W.S. Computing Science Technical Report No. 99 : A History of Computing Research at Bell Laboratories (1937–1975), [pdf]. A more visually oriented overview all the way to contemporary digital art is given by Paul, C. (2015) Digital Art, 3th ed. Thames & Hudson (pp 15–18 focuses on the period we are mainly interested in).
19. Noll, A.M. (1970) “Art Ex Machina” in IEEE Student Journal, Vol. 8, No. 4, (September 1970), pp. 10–14, [pdf]. Two other important early works by Noll, exploring computer graphics are: Noll, A. Michael (1967) “The Digital Computer as a Creative Medium,” IEEE Spectrum, Vol. 4, No. 10, (October 1967), pp. 89–95; and Noll, A. Michael (1971) “Scanned-Display Computer Graphics,” Communications of the ACM, Vol. 14, No. 3, (March 1971), pp. 145–150.
20. Brian Eno, lecture in Rio de Janeiro, October 20th, 2012 [Video] (citation at: 52m 35s).
21. Benoît Mandelbrot (1924–2010), Nature Obituary, Nature 468, 378 (18 November 2010) [nature].
22. “Beneath Apple Manor”, a rogue-like dungeon crawl released in 1978 for the Apple II, was one of the first games to have procedural generated maps and game objects. Available to play online at [archive.org]
23. The first in the SimCity game series was published in 1989. Available to play online at [archive.org]. Wright was greatly influenced by Jay Forrester’s (1969) work on System Dynamics in his design of the game. Or in the words of Wright, 25 years later: “Most of the simulation is really built up of rather simple rules, if you look under the hood, and it’s really interesting how these simple rules, when they interact with each other, give rise to great complexity,” Wright says. “You can’t even really sit back and engineer it or blueprint it. It’s more like you have to discover it, because the emergent systems are inherently, by definition, unpredictable.” In Doug Bierend, SimCity That I Used to Know: On the game’s 25th birthday, a devotee talks with creator Will Wright, Medium, Oct 17, 2014, [medium.com].
24. Work defining creativity in computational terms is as early as: Newell, Allen, Shaw, J. G., and Simon, Herbert A. (1963), “The process of creative thinking” in Gruber, H. E., Terrell, G. and Wertheimer, M. Contemporary Approaches to Creative Thinking, pp 63–119. New York: Atherton Press; Early foundational work on the categorization of creativity are: Boden, M. (1990), The Creative Mind: Myths and Mechanisms, London: Weidenfeld and Nicholson; Boden, M. (1999) Computational models of creativity, Handbook of Creativity, pp 351–373. This has been further explored and formalized by: Wiggins, G.A. (2006), A Preliminary Framework for Description, Analysis and Comparison of Creative Systems, Journal of Knowledge Based Systems 19(7), pp. 449–458. A good overview of the field of computational creativity is given in: Pereira, F. C. (2007). “Creativity and Artificial Intelligence: A Conceptual Blending Approach”. Applications of Cognitive Linguistics series, Mouton de Gruyter. Similar, but more multi-disciplinary oriented, works are: Veale, T., Feyaerts, K. and Forceville, C. (2013) Creativity and the Agile Mind: A Multi-Disciplinary Study of a Multifaceted Phenomenon, Walter de Gruyter; Besold, T.R., Schorlemmer, M. and Smaill, A. (2015) Computational Creativity Research: Towards Creative Machines, Atlantis Press.
25. Wiggins, G. A. (2006) A Preliminary Framework for Description, Analysis and Comparison of Creative Systems, Knowledge-Based Systems, Volume 19 Issue 7, pp 449–458.
26. The International Conference on Computational Creativity (ICCC) occurs annually since 2010, organized by The Association for Computational Creativity. Before that, a more informal collective of researchers held a dedicated workshop, the International Joint Workshop on Computational Creativity, every year since 2004. Today also few specialized departments or research groups exist (notably Godsmiths London, UK) and research cooperation with industry (Autodesk, Adobe, Sony, etc.) is common practice.
27. Von Foerster, H. “A Note from the Editors,” in Cybernetics: Circular Causal and Feedback Mechanisms in Biological and Social Systems, Transactions of the Eighth Conference (New York: Josiah Macy Jr. Foundation, 1952), 321.
28. Von Foerster, H., Mead, M., & Teuber, H. L. (Eds.). (1951). Cybernetics: Circular causal and feedback mechanisms in biological and social systems. Transactions of the seventh conference. New York: Josiah Macy, Jr. Foundation, p. vii; Fremont-Smith, F. (1960). The Macy Foundation conference plan. In M. Capes (Ed.), Communication or conflict: Conferences, their nature, dynamics and planning. New York: Association Press, pp. 218–19.
29. Norbert Wiener, Cybernetics: or Control and Communication in the Animal and the Machine (New York: Wiley, 1948), 11 [pdf].
30. Shannon, C. E. (1948). “A Mathematical Theory of Communication”. Bell System Technical Journal 27 (3): 379–423. July 1948; Shannon, C. E. (1948). “A Mathematical Theory of Communication”. Bell System Technical Journal 27 (4): 623–656. October 1948. Later in 1949 published together as what would become the text book of information theory, titled The Mathematical Theory of Communication by University of Illinois Press. It introduced concepts as information entropy and redundancy, and introduced the term “bit”.
31. Wiener, N. (1948). Cybernetics, or Control and Communication in the Animal and the Machine. Cambridge: MIT Press.
32. Cybernetic Serendipity, 1968, Institute of Contemporary Arts (ICA), London, UK, from 2 August to 20 October 1968 [http://cyberneticserendipity.net]. Exhibition catalogue “Cybernetic Serendipity: the computer and the arts”, Edited by Jasia Reichardt, Studio International Special Issue, London, 1968 [pdf].
33. Gordon Pask, “The Colloquy of Mobiles”, ICA London 1968; Nam June Paik, “Robot K-456”, ICA London 1968; Peter Zinovieff, “Music Computer”, ICA London 1968. More info at the cybernetics serendipity archive and catalogue in [32].
34. Fuller, B. (1969) Ideas and Integrities: A Spontaneous Autobiographical Disclosure (New York: Macmillan). For an analysis see: Turner, F. (2010) From Counterculture to Cyberculture: Stewart Brand, the Whole Earth Network, and the Rise of Digital Utopianism, University of Chicago Press, pp 56–58.
35. For an extensive history on the impact of cybernetics on fields as diverse as psychiatry, engineering, management, politics, music, architecture, education, etc. see: Pickering A. (2011) The Cybernetic Brain: Sketches of Another Future, University Of Chicago Press. Another historical work grounded in the belief that there is no way to separate cyberculture from counterculture is: Turner, F. (2006) From Counterculture to Cyberculture: Stewart Brand, the Whole Earth Network, and the Rise of Digital Utopianism, Chicago: University of Chicago Press. On the bridge between cybernetics and design see: Dubberly, H. & Pangaro, P. (2015). How cybernetics connects computing, counterculture, and design. In Hippie modernism: The struggle for utopia (pp. 126–141). Minneapolis, MN: Walker Art Center, And: Glanville, R. (2014). How design and cybernetics respect each other, RSD3 Relating Systems Thinking and Design 2014 working paper, [pdf]. Influential to architecture and cybernetics was Cedric Price’s Fun Palace going well beyond then established notions of architecture. See: Matthews, S. (2005) The Fun Palace: Cedric Price’s experiment in architecture and technology, Technoetic Arts Vol 3 Nr 2, [pdf]. An important legacy of cybernetic thinking can also be found in MIT Media Lab, founded by Nicolas Negroponte, to merge the “old media” (print and broadcasting) with the “new media” of personal computing. A foundational work on its vision is: Brand, S. (1988) The Media Lab: Inventing the Future at M.I.T., Penguin Books.
36. Gordon Pask (1969) “The Architectural Relevance of Cybernetics,” Architectural Design, p 494.
37. Dubberly, H. & Pangaro, P. (2015) How cybernetics connects computing, counterculture, and design [dubberly.com].

2. Assisted Creation

1. Shneiderman, B. (2000) Creating creativity: user interfaces for supporting innovation, ACM Transactions on Computer-Human Interaction (TOCHI) — Special issue on human-computer interaction in the new millennium, Part 1, Volume 7 Issue 1, March 2000, pp 114–138.
2. The founding and development of Autodesk (and CAD systems) is well described in David E. Weisberg (2006) The Engineering Design Revolution: The People, Companies and Computer Systems That Changed Forever the Practice of Engineering, chapter 8: Autodesk and AutoCAD [cadhistory.net].
3. The history of Adobe Systems is most extensively (and perhaps a bit biased as published by Adobe Press) described in Pfiffner, P. (2002) Inside the Publishing Revolution: The Adobe Story, Adobe Press. Another interesting review is given in the short documentary by Terry Hemphill (also of Adobe) “The Adobe Illustrator Story”, [Vimeo].
4. We intend the skeuomorphic meaning here: designing representations of objects to resemble their real-world counterpart.
5. Leica (then Ernst Leitz Wetzlar GmbH) invented its autofocus knowledge and patented it between 1960 and 1973 to sell the technology to Minolta who produced the first autofocus camera: the Minolta Maxxum 7000. Reference patent US 4047022 A, Holle, W.H. (1977) Auto focus with spatial filtering and pairwise interrogation of photoelectric diodes”.
6. Lewis-Kraus, G. (2014) “The Fasinatng … Frustrating … Fascinating History of Autocorrect” in Wired Magazine, 22/7/2014, [wired.com]; Original patent 23155063 Rayson et. al. (1994) “Autocorrecting text typed into a word processing document”.
7. Crockett, Z. (2015) “The Inventor of Auto-Tune”, at [priceonomics.com]; Feature with Andy Hildebrand in “Auto-Tune”, NOVA scienceNOW, PBS TV, June 30, 2009 [Video].
8. Case in point is the example of Auto-tune, which was both celebrated by some and despised by others. Critical voices come from people as music producer Rick Rubin: “Right now, if you listen to pop, everything is in perfect pitch, perfect time and perfect tune”; On the other side, auto-tune — as we wrote — lowers the barriers for creative participation, mainly for less professionals, but also opens up new possibilities in music. A funny example here comes from the band The Gregory Brothers in their series Songify the News, “songifying” serious news broadcasts, as with their “Bed Intruder Song” video, which became the most-watched YouTube video of 2010.
9. Fiebrink, Rebecca Anne (2011) Real-time Human Interaction with Supervised Learning Algorithms for Music Composition and Performance, Dissertation, p 2 [pdf].
10. Nicholas Davis, Alexander Zook, Brian O’Neill, Brandon Headrick, Mark Riedl, Ashton Grosz, and Michael Nitsche (2013) Creativity support for novice digital filmmaking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘13). pp 651–660 [pdf]; Unlocking human creativity: Mark Riedl at TEDxPeachtree, [Video].
11. Needham, Joseph (1986). Science and Civilization in China: Volume 4, Physics and Physical Technology, Part 2, Mechanical Engineering. Taipei: Caves Books, Ltd. p 89, 110; A A Harms, B W Baetz, R R Volti (2004) Engineering in Time: The Systematics of Engineering History and Its Contemporary Context, Imperial College Press, p 49.
12. The quote “First we shape our tools, thereafter they shape us” seems to be mistakenly attributed to Marshall McLuhan. While consistent with his ideas on technology, the quote is actually from John M. Culkin in an article about McLuhan: Culkin, J.M. (1967) A schoolman’s guide to Marshall McLuhan. Saturday Review (March 18, 1967) pp. 51–53, 71–72. This point is made by Alex Kuskis (2013) “We shape our tools and thereafter our tools shape us” [mcluhangalaxy.wordpress.com].
13. Flemisch, Frank .O., Adams, Catherine A., Conway, Sheila R., Palmer, Michael T. and Schutt, Paul C. (2003) The H-Metaphor as a Guideline for Vehicle Automation and Interaction, NASA/TM — 2003- 212672 [pdf].
14. Roderick Murray-Smith (2013) Casual interaction with computers: music applications, presentation at Digital Audio, Copenhagen, 20th June 2013, p 54 [slides]; For an application of this “focused–casual continuum” approach also: Henning Pohl and Roderick Murray-Smith (2013) Focused and Casual Interactions: Allowing Users to Vary Their Level of Engagement [pdf].

3. Generative Creation

1. Human scale measurements and proportions became commonplace, most famously in Leonardo Da Vinci studies, ie The Vitruvian Man, in ca. 1492. Studies of anatomy, light and the landscape by Leonardo see specifically “A Treatise on Painting” by Leonardo da Vinci, assembled by his pupil Francesco Melzi and published in France and Italy in 1651 and Germany in 1724 [full text at archive.org].
2. Renaissance painters Brunelleschi and Alberti developed theories of perspective to represent three-dimensional objects on a two-dimensional surface. Again it was Leonardo Da Vinci who improved on these theories through his scientific approach to art. See again: Leonardo da Vinci “A Treatise on Painting”
3. Matt Jezyk (2015) Design Computation Symposium Part 1, AS12577, Autodesk University, Las Vegas [video at autodesk.com].
4. Early applications of computer simulations were almost all military. Arguably the first massive computer simulation was during the Manhattan Project in World War II to model the detonation process of a nuclear bomb. See also: Masco, J. (2006) The Nuclear Borderlands: The Manhattan Project in Post-Cold War New Mexico. Princeton University Press.
5. Doug Engelbart (1986) “The Augmented Knowledge Workshop” at ACM Conference on the History of Personal Workstations, held at Rickey’s Hyatt House in Palo Alto, California, January 9–10, 1986. [Video], [Transcription].
6. Schumacher, P. (2010) The Parametricist Epoch: Let the Style Wars Begin, In: AJ — The Architects’ Journal, Number 16, Volume 231, 06. May 2010 [text at patrikschumacher.com]; Schumacher, P. (2014) The Historical Pertinence of Parametricism and the Prospect of a Free Market Urban Order, In: Poole, M. and Shvartzberg, M. The Politics of Parametricism, Digital Technologies in Architecture, Bloomsbury Academic, New York 2015 [text at patrikschumacher.com].
7. Schumacher, P. (2008) Parametricism as Style — Parametricist Manifesto, London, Presented and discussed at the Dark Side Club , 11th Architecture Biennale, Venice 2008 [text at patrikschumacher.com].
8. Bohnacker, H., Gross, B.,. Laub, J., Lazzeroni, C. (2009) Generative Gestaltung: Entwerfen, Programmieren, Visualisieren. Schmidt, Mainz [book & info at generative-gestaltung.de].
9. ie Rhinoceros (typically abbreviated Rhino, or Rhino3D), a popular 3D computer graphics and CAD application, used for many generative design tasks (ie 3D printing) because its geometric model is based on the NURBS mathematical model, which focuses on producing mathematically precise representations. A popular visual scripting language add-on is Grasshopper, used extensively under generative artists and designers.
10. Michael Hansmeyer: Building unimaginable shapes, TEDGlobal 2012, Jun 2011 [Ted Talk] (citation at 10m12s).
11. Diederik P Kingma, Max Welling (2013) Auto-Encoding Variational Bayes. Arxiv, GitXiv (Code & Article); Danilo Jimenez Rezende, Shakir Mohamed, Daan Wierstra (2014) Stochastic Backpropagation and Approximate Inference in Deep Generative Models. [Arxiv].
12. Gregor, K., Danihelka, I., Graves, A., Rezende, D.J., Wierstra, D., (2015) DRAW: A Recurrent Neural Network For Image Generation. [Arxiv], [GitXiv] (Code & Article).
13. Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A., Bengio Y., (2015) A Recurrent Latent Variable Model for Sequential Data. Arxiv, GitXiv (Code & Article)
14. Goodfellow, I.J.,Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y. (2014) Generative Adversarial Networks. [Arxiv], [GitXiv] (Code & Article).
15. Radford, A, Metz, L., Chintala, S. (2015) Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. [Arxiv], [GitXiv] (Code & Article).
16. Denton, E., Chintala, S., Szlam, A., Fergus, R. (2015) Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks. [Arxiv], [GitXiv] (Code & Article).
17. Im, D.J., Kim, C.D., Jiang, H., Memisevic, R. (2016) Generating images with recurrent adversarial networks. [Arxiv].
18. “Auto Translate”, most famously implemented in Google Translate.
19. A flurry of Speech-to-text APIs and service has become available over the past few years, ie Google’s speech-api (api key needed).
20. Mansimov, E., Parisotto, E., Ba, J.L., Salakhutdinov, R. (2015) Generating Images from Captions with Attention. [Arxiv].
21. Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H. and Courville, A. (2015) Describing videos by exploiting temporal structure. In ICCV15. [Arxiv], [GitXiv] (Code & Article); Venugopalan, A., Xu, H., Donahue, J., Rohrbach, M., Mooney, R. and Saenko, K. (2015) Translating videos to natural language using deep recurrent neural networks. In NAACL15. Arxiv; Srivastava, N., Mansimov, E. and Salakhutdinov, R. (2015) Unsupervised learning of video representations using LSTMs. In ICML15. [Arxiv], [GitXiv] (Code & Article); Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T. and Saenko, K. (2015) Sequence to Sequence — Video to Text. In ICCV15. [Arxiv], [GitXiv] (Code & Article); Simonyan, K. and Zisserman, A. (2014) Two-stream convolutional networks for action recognition in videos. In NIPS14. at Arxiv; Sharma, S., Kiros, R. and Salakhutdinov, R. (2015) Action Recognition using Visual Attention. [Arxiv], [GitXiv] (Code & Article)
22. Fiebrink, R.; Trueman, D. and Cook, P. R. (2011) The Wekinator: Software for using machine learning to build real-time interactive systems. Demo at MusicTechFest London (2015), [Video], [Software];
    Vik, C. (2012) Astral Body, Demonstration at PACT Theatre, Sydney, 18 February 2012 [Video].