The first few years of my career were a time of intense learning.

I encountered the realities of being a software engineer and had to acquire many skills that I didn’t know I needed. Looking back, it would sure have been nice to know the things I know now.

So, I wrote this guide to help others based on the experiences of developers I mentored in their first few years as professionals, and those of myself and some of my colleagues.

I will cover:

• How to make the best out of interviews,
• How to survive (and thrive) in your work as a software engineer,
• And what resources to look into when considering continuous improvement.

Interviews

As you start your career in Software Engineering, you’ll have to face one indisputable fact. Interviews suck.

They can be awful for everybody involved. Having been both an interviewer and an interviewee, I can attest that interviews are a big time sink, extremely stressful and a really bad indicator of future job performance. Nevertheless, they are a necessary evil that you and your résumé better be prepared for.

Preparing for Battle

If you’re considering a career in Software Engineering, make sure to learn some of the most commonly asked programming interview questions, such as ‘FizzBuzz’:

“Write a program that prints the numbers from 1 to 100. But for multiples of three print ‘Fizz’ instead of the number and for the multiples of five print ‘Buzz’. For numbers which are multiples of both three and five print ‘FizzBuzz’.”

(Coding Horror)

Sounds simple enough, right?

Well, the vast majority of interviewees fail this simple test, let alone its more complex variants.

I’ve personally seen many candidates for senior positions fail this test while having full internet access. So make sure that if a programming language is listed on your résumé, you know how to do at least FizzBuzz in it. Otherwise, you’re just wasting everybody’s time, including yours.

Of course, you’ll need to know more than just FizzBuzz to survive your interviews. You also need to make sure that you know:

• Basic data structures and algorithms: such as linked lists, arrays, trees and sorts.
• Common “gotchas” in your language of choice, such as whether strings are immutable, and how memory is managed.
• Object Oriented Programming concepts like class versus object, and inheritance.

At the beginning of your career, you’ll need to shine on these sorts of questions, since you don’t have the experiences to prove that you will be good at the job. There are two resources that I always recommend when preparing for interviews:

• “Cracking the Coding Interview”, a fantastic book that includes a lot of coding problems and their solutions, as well as summaries of what you need to know to solve them
• CodeWars, a website that has a large collection of coding problems that you can solve in the browser using a wide selection of languages. The most useful part is seeing how other users solved the same problem. You’ll get to see different approaches to the same problem and learn new tools in the language of your choice.

Give yourself that extra edge

There are several things you can do that will give you that little something extra.

First, learn to communicate your experiences. You should have an elevator pitch that summarizes your resume into a coherent and engaging narrative.

In addition, know your own résumé! It sounds silly, but I’ve seen a lot of interviewees struggle to explain a particular item on their résumé. You should be able to answer questions about any experience that you list on your résumé and explain how it’s made you a better candidate for the job.

Next, have code samples to show on GitHub (or another public repository).

Seeing is believing, and interviewers being able to see your code will do wonders. Plus, it shows you have an understanding of version control systems.

The code samples don’t have to be anything too complex, but they do need to be clean and show good coding practices. This is your chance to show how you code without the time-pressure of a coding interview.

Once you’ve done all of the above, it’s time to consider participating in an open source project. It will show you can work in an existing code base and collaborate with other programmers.

This will be the closest you can get to programming in an industry environment without actually being in an industry environment. This is by far the hardest and time-consuming item so far, so reserve it until you’ve completed the lower hanging fruit I’ve discussed above.

Interviewing your interviewer

In the rush and stress of the job hunt, many candidates forget that interviewing is a two-way street. As the company is trying to figure out whether you’re the right person for the job, you should be figuring out if the company is the right fit for you.

Make sure that you get to ask some of the questions below, even if it’s in a follow-up email. Be aware that often times, companies might try to spin not following best engineering practices as a perk, so read between the lines.

Here are some example questions you could ask:

“What would a typical workday look like for me?”

It’s important to know what to expect from a particular position because Software Engineering jobs vary quite a bit. You might be expected to upkeep servers or talk to clients directly, for example.

Red flag: “I’m not sure.” → Means that the people interviewing you won’t be on your team, or they don’t have a clear idea why they’re hiring you.

“How do you test your software?”

Ideally, a combination of unit testing, manual testing, and automated testing should be used to verify the quality of the code.

Red flag: “We just don’t write bugs, haha.” → Those people are exactly the ones writing bugs.

“What version control system do you use?”

Version control systems are extremely useful for collaboration and there are zero reasons to not use one in a professional setting.

Red flag #1: “Uh, version control system?” → Run far, far away.

Red flag #2: “<insert obscure or custom VCS>” → Indicates they’re most likely not keeping up with the times and haven’t updated their infrastructure in a long time.

“Do you do peer reviews?”

Peer reviews, or having someone else look at your code before it goes into the code base, is a fantastic way to spot silly mistakes and is a vital training opportunity when starting your career.

Red flag: “We just trust each other!” → Very likely that the senior developers are very protective of their code and not great at receiving feedback.

“What programs do you have for continuous education?”

Being a software engineer means constantly learning as technologies appear, mature and become out-dated at a dizzying rate. As such, many companies have a training budget that they use to pay for university and online classes, conferences, or in-house talks.

Red flag: “You mean reading stuff online in your free time?” → The company is either strapped for cash or sees developers as replaceable and not as long-term investments.

“What’s the software development process that you use?”

Process is vital to software engineering, regardless of the actual details. The particulars of what constitutes optimal process are subject to intense debate, but the mere existence of an agreed upon way of working on a project minimizes chaos and ensures everybody is on the same page.

Red flag: “Our process is inspired by free-form jazz.” → Most likely the entire department is in fire-fighting mode, jumping from emergency to emergency without any clear goal.

“How do you tackle technical debt?”

Technical debt is an accumulation of outdated technologies and quick-but-dirty solutions in the code base. Addressing it is important to the long-term health of the code and should be done on a continuous basis.

Red flag: “We’re exclusively focused on new features.” → Their code base is a mess or it will be soon.

“What’s your company culture like?”

Company culture might be a very vague concept, but even small things like an open office versus cubicles will change your day-to-day interaction with colleagues in significant ways. There are no general red flags, but make sure their answer is something you can live with for 40+ hours a week for years.

Working as a Software Engineer

At this stage, if you performed well in your interviews and liked how the interviewers answered your questions, you’ll likely be hired.

Congrats, You’re Officially an Engineer!

Now what? Well, it’s time to relearn a lot of things about coding and working. And since we’re programmers, let’s start by discussing code.

Good Industry Code

Good industry code has the following properties, in that order:

• Readable, because code is read and maintained more often than it is written. The intent of the code must be clear to other developers years after you’ve written it.
• Defensive, as in following best practices of defensive coding. Defensive coding is a topic all on its own, but the gist of it is: You have to ensure that improper use of classes and methods you’ve written won’t lead to your code crashing the software.
• Optimized, which is last on this list because most of the time, you won’t really need to worry about it. That doesn’t mean you should write bad code that does something in O(n³) when a linear solution exists. But developers are generally eager to try and over-optimize when there’s no need for it, often at the detriment of the readability and defensibility of the code. You should always be able to prove that a certain optimization that sacrifices those properties is actually needed.

Now that you know how to write good industry code:

You Won’t be Doing Much Coding

It may come as a surprise, but most of the time you won’t be writing new code, but instead, you’ll be:

• Debugging
• Reading existing code
• In meetings or writing emails
• Researching what to do so you don’t write code

Therefore skills other than coding will be just as vital for your career.

Debugging and Reading Code

• You’ll need a lot more than debugging using print statements. All widely-used languages and tech stacks have a variety of powerful tools. Learn to use them as they’ll make debugging a breeze and save you countless hours.
• Understand the code base. Most tech stacks have some sort of code graph generation tools that will help you understand the structure of the code base. Enterprise IDEs generally have that functionality built in. You can also explore the code using tools such as ReSharper, grep or Sourcegraph.
• Understand the product. You’ll be surprised how many developers don’t know how the software is supposed to work before they try to “fix” it. Just read the documentation.

Organize Your Thoughts

Since a lot of your time will be spent in communication, research and multi-tasking, you need some tools to help keep everything in order.

• TODO lists / Tasking: Your company should already have tasking software of some kind, but it helps to have a personal system as well. Use post-it notes or software like Trello or Todoist.
• Notes: Always take notes in meetings, work to improve existing documentation and create a personal knowledge base. Use Evernote, OneNote, or a notebook, like in the olden days. It might seem like overkill, but you’ll be thanking yourself a year later when you’re revisiting that obscure build process which took you 3 days to figure out the first time. I’ve never met a good Software Engineer who didn’t take extensive notes.
• Charts/Visualizations: Humans are visual creatures and creating charts of processes and architectures will help you and others understand complex topics. Diagrams are particularly useful when communicating with non-technical colleagues. Use Lucidchart, Visio or a plain whiteboard.

Know When to Use Libraries

Short answer: Almost all the time.

Long answer: 99% of the time, you shouldn’t reinvent the wheel. In most Software Engineering positions, implementing a particular kind of sort is a complete waste of time. That doesn’t mean you shouldn’t know how the algorithms and data structures you use work, since that will help you decide what to use and when.

In order to be an efficient Software Engineer, you need to understand the libraries that you have at your disposal. The standard libraries of most popular languages are extremely useful and are larger than what you’d expect. In addition, the code base might also utilize additional, specialized libraries. Read their documentation and know when to use them.

You should also not be afraid to suggest additional libraries if they will save time. However, you need to ensure that you pick a good library for industry use. A good library is:

• Open source, so you can verify the quality of code yourself and potentially fix bugs that are critical to your application.
• Licensed under a permissive license such as MIT and BSD, so your company doesn’t run into any issues by using it. Be careful with GPL, lest you open source your entire code base by accident.
• Mature, i.e. it has been out for some time and has a rich set of features.
• Maintained, with new releases coming out often.
• Used by other companies or projects, which acts as a stamp of approval and ensures it has industry support for continued maintenance.

Continuous Improvement

In addition to learning the skills that will make you better at your day-to-day job, you’ll also need to continuously improve your skills and learn new ones, in order to create new career opportunities for yourself.

The opportunities to learn are many and a lot of them are quite affordable:

• Online Courses: The opportunity to learn from the best professors in the field in a flexible format shouldn’t be missed. Check out Coursera, Udacity, and edX (among many) for courses that can supplement your existing skills.
• Online Master’s Degrees: A recent trend among top-ranked universities, online Master’s Degrees are a flexible way to continue your formal education. They are also generally less expensive thank on-campus degrees, with most programs costing ~$10,000 for the entire degree. Georgia Tech, UT, and UC San Diego are some of the universities offering such degrees. I personally recommend Georgia Tech’s Online Master’s which I graduated from this year.
• Blogs: Blogs are an important part of the developer community (no surprise here, as you’re reading one right now). Blogs such as Coding Horror, Joel on Software, or even more humorous websites such as The Daily WTF can give you a good idea of what and what not to do as a Software Engineer. Browsing Medium, r/programming, HackerNews or other feeds will also lead you to good articles and blogs.
• Conferences: Last, but not least, conferences are an amazing learning opportunity and you should definitely take advantage of your company’s training budget by going to them. A very incomplete list of good conferences to check out (alongside their topic): GOTO; (General), Strange Loop (General), PyCon (Python), CPPCon (C++), DEF CON (Security), Fluent (Web dev). All of these also have videos of (most) talks on YouTube so you will be able to learn something even if you can’t attend!

Hopefully, this article has armed you with the knowledge of what to expect from the beginning of your career as a Software Engineer and given you the tools to perform well on this exciting journey! Thanks for reading! If you have any questions or suggestions, please leave a comment or tweet @AlexievValeri.

A Software Engineering survival guide was originally published in We’ve moved to freeCodeCamp.org/news on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this short tutorial, I’ll walk you through training a Keras model for image classification and then using that model in a web app by utilizing TensorFlow.js. The problem we’ll be solving is Not Hotdog: given an image, our model will have to correctly classify the object as a hotdog or not a hotdog. This classification task is not particularly exciting, but for this tutorial, we’ll be focusing more on the process of using a pre-trained Keras model using Tensorflow.js.

Training a Keras Model

Let’s begin by building our dataset. I used the Google images download utility, but you can use whatever you prefer. Instabot is another good option. Just make sure you have a few hundred images for both classes and you split them into training, validation and test sets in the format that Keras expects:

Trending AI Articles:

1. Building a Django POST face-detection API using OpenCV and Haar Cascades

2. Learning from mistakes with Hindsight Experience Replay

3. How to Prevent Bias in Machine Learning

4. AI & NLP Workshop

Next, we’ll build a simple deep net to train on the dataset that we have. The neural network I used is composed of 3 chunks of convolutions with ReLU activations and maxpool layers after them. On top, we have two fully connected layers with a ReLU activation, a dropout layer and a sigmoid for binary classification.

We’ll use binary cross-entropy as the loss function and use RMSProp as the optimization algorithm. We train for 50 epoch to achieve ~93% accuracy on the validation set, which is good enough for the purposes of this tutorial. To train the model yourself or play around with the code, check out the notebook here. The code is largely based on the first parts of this Keras tutorial.

Once we have a trained model, we need to make sure we save it to disk before we proceed to the next section:

model.save('simplemodel.h5')

Be sure to use the correct method for saving the model. Keras provides several different methods for saving models:

• model.save_weights('<filename>') will save just the weights of the model;
• model.to_json()/ model.to_yaml() will save the architecture to json or yaml format;
• model.save('<filename'>) will save the weights, architecture and even the optimizer state so training can be resumed;

We need to use the last method because, unsurprisingly, TensorFlow.js needs both the weights and architecture of our model before it can utilize it.

Converting a Keras Model into a TensorFlow.js Model

Now that we have the model saved, install the tensorflowjs Python package and run the following command:

tensorflowjs --input_format keras <path-to-h5-file> <path-to-dir>

Alternatively, we could have used the tensorflowjs Python API to save the model directly as a TensorFlow.js model:

tensorflowjs.converters.save_keras_model(model, model_dir)

In either case, we should now have several files in our model directory: a model.json file and several weight files in binary format. It’s important to note that these conversions are only supported for standard Keras classes and methods. Custom layers or metrics cannot be safely converted from Python to JavaScript.

Using a Pre-trained Model in TensorFlow.js

Once we have the model converted, let’s use it in a small web application. On the HTML side of the things, we’ll simply have an image_upload file select element, an image element to show the selected image, and a result div to show the model’s classification.

The JavaScript side of things is a bit more complicated. Let’s take a look at the code and then we can step through it:

First, we begin by loading our model and we ensure that we actually wait for the operation to finish by using await:

const model = await tf.loadModel('./model/model.json');

Next, we need to set an event handler that responds to the file selector being used. We’ll use the FileReader API by setting another callback when an image is loaded and trigger the actual loading of the image using readAsDataURL(...).

document.getElementById('image_upload').onchange = function(ev) {
    var f = ev.target.files[0];
    var fr = new FileReader();

    var makePrediction = async function(img) { ... };
    var fileReadComplete = function(ev2) { ... };

    fr.onload = fileReadComplete;
    
    fr.readAsDataURL(f);
}

Once the file has been read, we’ll show the image on our page and then we’ll create an Image object that will be passed to the actual prediction function:

var fileReadComplete = function(ev2) {
    document.getElementById('image').src = ev2.target.result;
    var img = new Image();
    img.src = ev2.target.result;
    makePrediction(img);
};

At this point, we have to ensure that the Image object is ready or the rest of the code will not be happy. That’s why we’ll use the wait lambda that we defined at the top of our code, to ensure that the function waits until the image is ready to be used.

Then, we have to convert our Image object into a tensor with the correct formatting. We’ll use the fromPixels(...) method to transform the image to a tensor, resize it to what our model expect using resizeNearestNeighbor(...), convert it to floating point values using toFloat(), and then use expandDims() to insert another dimension in our tensor so that it fits the batched input format our model was trained on.

var makePrediction = async function(img) {
    while(!img.complete) {
        await wait(100);
    }

    var tensor = tf.fromPixels(img)
                   .resizeNearestNeighbor([150,150])
                   .toFloat().expandDims();

    const prediction = model.predict(tensor);

    var data = prediction.dataSync();
    document.getElementById('result').innerHTML = 
       data[0] == 0 ? "Now, that's a hotdog! :)" : "Not hotdog! :(";
}

After we have pre-processed our image, we can pass it into our model using the predict(...) method and get a prediction. In order to get the actual data out of the prediction tensor, we’ll use the dataSync() method. At this point, you can do whatever you need with the prediction. In this case, we’ll add a simple message to our web page that answers the age-old question: “Is this a hotdog?” We truly live in the future.

Thanks for reading! If you have any questions or suggestions, please leave a comment or tweet @AlexievValeri.

Not Hotdog with Keras and TensorFlow.js was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Classical neural networks are extremely flexible, but there are certain tasks they are not well suited for. Some arithmetic operations, in particular, prove challenging to neural networks. That’s why Trask et al in their paper Neural Arithmetic Logic Units introduce two new modules that are meant to perform well on certain arithmetic tasks. In this blog post, I’ll describe the problem that the paper aims to solve, the proposed solution and discuss implementation details and results reproduction using PyTorch.

So you’d like to learn the identity function?

That sounds like a simple enough task for a neural network, right? And it is, if our test set is in the same range as our training set. However, as soon as our test data goes outside the training range, most neural networks fail spectacularly. This failure reveals an uncomfortable truth: a multi-layer perceptron is theoretically capable of representing any continuous function, but choices of architecture, training data and learning schedule will severely bias what function the MLP ends up learning. Trask et al effectively illustrate this failure with the following graph:

In order to overcome these limitations, the paper suggests using two new modules: the Neural Accumulator (NAC) and the Neural Arithmetic Logic Unit (NALU). Let’s look at what they are and the math behind them.

NAC and NALU

Let’s start with NAC:

As we can see above, NAC is similar to a linear transformation, but the weight matrix we use is the result of two learned matrices on which we apply tanh and sigmoid before we element-wise multiply them:

Why go through this trouble? And how does this help our networks to learn arithmetic functions properly? Let’s take a look at the graph below:

We can see that the values of the weight vector are biased towards 0, 1, and -1. This means that the outputs of a NAC are additions and subtractions of the input vector, rather than scalings. This allows NACs to be stacked together indefinitely and still keep consistent scaling of the input representations. This is clearly helpful for learning arithmetic operations dealing with addition and subtraction, but it won’t cut it for multiplication or division.

That’s where the NALU comes in:

The NALU uses a NAC as a subunit and adds two additional flows of information. The first uses the NAC to operate in log-space and the second acts as a learned gate that calculates a weighted sum between the standard NAC and the log-space NAC. This should theoretically allow the NALU to learn functions such as multiplication and division, as well as addition and subtraction.

Enough chit-chat, let’s look at the code!

The implementation of NAC and NALU is fairly straightforward. You can take a look at the GitHub repo, but I’ll be going over the important bits here. First, we’ll start with NAC:

class NAC(Module):    
  def __init__(self, n_in, n_out):
    super().__init__()
    self.W_hat = Parameter(torch.Tensor(n_out, n_in))
    self.M_hat = Parameter(torch.Tensor(n_out, n_in))
    self.reset_parameters()

  def reset_parameters(self):
    init.kaiming_uniform_(self.W_hat)         
    init.kaiming_uniform_(self.M_hat)

  def forward(self, input):
    weights = torch.tanh(self.W_hat) * torch.sigmoid(self.M_hat)
    return F.linear(input, weights)

The code should be fairly self-explanatory. We start by subclassing PyTorch’s Module. In the initialization step, we define W_hat and M_hat, the two learned matrices, as Parameters and initialize them using He initialization. During a forward step, we pass W_hat through a tanh and M_hat through a sigmoid before we multiply them. Then, we take the resulting matrix and perform a linear transformation using it and the input to the cell. And there you have it: a fully-functional NAC module!

Before we move to the NALU implementation, it’s important to highlight the fact that there are only two learned matrices in the above code. The matrix that gets passed into the linear transformation does not get trained and is just the result of the mathematical operation of the actual learned matrices W_hat and M_hat. When the NALU paper first came out, many of the PyTorch implementations had that final weight as a parameter, which turned their NACs into simple linear layers. These FauxNALUs, as I called them, became an accidental ablation study and I’ll discuss their significance in the next section.

Before we get there though, we still need to take a look at the NALU implementation:

class NALU(Module):    
  def __init__(self, n_in, n_out):
    super().__init__()        
    self.NAC = NAC(n_in, n_out)        
    self.G = Parameter(torch.Tensor(1, n_in))        
    self.eps = 1e-6        
    self.reset_parameters()        

  def reset_parameters(self):        
    init.kaiming_uniform_(self.G)        

  def forward(self, input):        
    g = torch.sigmoid(F.linear(input, self.G))        
    y1 = g * self.NAC(input)        
    y2 = (1 - g) * 
         torch.exp(self.NAC(torch.log(torch.abs(input) + self.eps)))
    return y1 + y2

We begin by defining the NAC and the learned gate matrix and initialize them. In the forward step, we first compute the gate value by doing a linear transform of the input and the gate matrix G. Then we compute the standard NAC output and the log-space NAC output before we return their weighted sum. It’s important to note that we need to pass a small epsilon value to the log function when computing the log-space NAC, in order to avoid any undefined behavior.

Time for math class

So now that we know the theory behind NALUs and how to implement them in PyTorch, all that’s left is to actually see them in action. We’ll attempt to reproduce the results of the function learning part of the paper using scripts based on kevinzakka’s code that I’ve modified to include FauxNALU here. The idea of the function learning test is that we’ll provide randomly generated input data that conforms to a particular function (addition, subtraction, division, square root, etc) and then test whether our MLP can learn that arithmetic function properly:

There are several takeaways from these results:

• The NAC performs amazingly at addition and subtraction but is horrible at multiplication, division and power functions. This makes sense because the same thing that makes it powerful for addition (no arbitrary scaling of inputs) is limiting it greatly when it has to learn other functions.
• The NALU performs consistently well at everything except subtraction. Given the fact that both NAC and FauxNALU perform well on subtraction, this indicates that somehow the log gate and the NAC interfere with each other in the subtraction use case.
• The results of the FauxNALU are the most interesting. It performs about as well as the None MLP (the one using just a linearity), which indicates that the learned gate, by itself, does not help with learning these functions. NALU beats FauxNALU decisively when it comes to learning division and square roots, suggesting that’s where the combination between a log-space NAC and a standard NAC really shine.

That’s all, folks!

NAC and NALU are certainly interesting modules, even though their performance on the toy problem we looked at was mixed. Only time and further experiments will tell whether these specialized modules will become part of the regular toolbox of machine learning practitioners. One thing is for certain: the idea that we can design modules with a particular bias in mind through the clever definition of weight matrices and the use of gates is here to stay and we’ll be seeing a lot of other modules that leverage it.

Thanks for reading! If you have any questions or suggestions, please leave a comment or tweet @AlexievValeri.

A Quick Introduction to Neural Arithmetic Logic Units was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

“A cook is cracking a raw egg against a glass bowl. Properly performed, the impact of the egg against the edge of the bowl will crack the eggshell in half. Holding the egg over the bowl, the cook will separate the two halves of the shell with his fingers, enlarging the crack, and the contents of the egg will fall gently into the bowl. The end result is that the entire contents of the egg will be in the bowl, with the egg unbroken, and that the two halves of the shell are in the cook’s fingers.” (Thomason)

This problem is one of the so-called benchmark problems for common sense reasoning. Such problems are notoriously hard for computers to handle, because there is no easy way to formalize them. This is because we can consider many complications: What if the cook breaks the egg too quickly, or too slowly? What if he tries the procedure with an M&M or a coconut? What if the bowl is smaller than the egg or upside down? Such considerations are easy for humans to make, but AI systems need some sort of formalized system to handle this. Several researchers have solved the egg-cracking problem, such as Morgenstern (2001) and Shanahan (2004). However, their solutions are too long to elaborate on in this paper. One thing is evident: there are many similarities in the solutions, which are top-down and use versions of Situation Calculus or Event Calculus. The axiomatizations are modular and focus on different aspects of the problem, including movement, material properties, relevant actions, etc. As in most top-down approaches, all relevant aspects must be explicitly stated. Therefore, needless to say, while the simple case is solved, many of the elaborations of the problem are not considered, though the axiomatic systems are built in such a way as to make such elaborations possible (Thomason).

While there are several attempts at formalizing common sense, this paper is going to focus on the works of Andrew S. Gordon and Jerry Hobbs. They have been working on an ambitious top-down formalization of common sense psychology. Their project started with a simple question: What concepts are necessary for the common strategies that people use in achieving their goal? Gordon and Hobbs investigated ten different domains, including politics, personal relationships, artistic performance and warfare, and collected more than 350 strategies. They identified a vocabulary of concepts and expanded it using methods from computational linguistics (Gordon, Hobbs, 2003). They categorized these concepts into 48 representational areas. 18 of those areas included topics such as naive physics, which have been investigated before in great detail. Gordon and Hobbs decided to focus on the remaining 30 areas that are connected with common sense psychology (see Figure 1). Currently, they have completed about half of those representational areas (Hobbs, Gordon, 2005A).

In this paper, I’m going to focus primarily on the smallest representational area: memory. This area deals with storing and retrieving concepts, forgetting, repressing, reminding and other actions that an agent can do with its memory. First, we need the definitions of a mind and of the entities that have minds:

mind(x, p) ⊃ agent(p)
person(p)  ⊃ agent(p)
person(p)  ⊃ (∃x)mind(x, p)

That is if p has mind x, then p is an agent. Persons are agents and all persons have minds. Note, however, that these axioms do not claim that every agent has a mind (Gordon, Hobbs, 2003). In fact, an interesting feature of this theory is that nations, organizations, companies and other groups of agents can be considered agents in their own right (Hobbs, Gordon, 2005A). For example, we can think of the company Coca Cola to have selling soda as its goal and think of it as planning in order to achieve that goal. This makes the theory very flexible as to what constitutes an agent, which reflects how humans typically view the world as well.

A serious problem in most AI theories of belief is logical omniscience. Put simply, this means that if we enable the agent to draw conclusions from his beliefs, then he automatically believes all the logical consequences of those beliefs. This, of course, isn’t the case. If humans were logically omniscient, it would mean that knowing the axioms of set theory would be equivalent to knowing all of mathematics (Gordon, Hobbs, 2004). Hobbs and Gordon work around this by limiting the beliefs available for inference to only those that are in the current focus of attention. This solves the problem of logical omniscience in a practical way consistent with our experiences. After all, who hasn’t failed to solve a problem because they didn’t think to apply a theorem they know? Thus, we need to define the mind’s component parts:

mind(x, p) ⊃ (∃f)[focus(f,p)^part(f,x)]
mind(x, p) ⊃ (∃m)[memory(m,p)^part(m,x)]

Simply put, a mind has at least two parts: focus and memory. Focus is the part of mind where concepts are when a person is actively thinking about them, and memory is the part of the mind that the person has all concepts he knows, but is not necessarily thinking about at the moment (Gordon, Hobbs, 2003). After defining the parts of the mind, Gordon and Hobbs define what entities need to be in those parts. They define the predicate concept very vaguely. It is a general term that covers any entity that is in a person’s mind, including beliefs, facts, and memories. Even visual and audio information perceived by the senses can be referred to as a concept (Gordon, Hobbs, 2004).

Next, in order to effectively talk about concepts being in parts of the mind, Hobbs and Gordon define a predicate inm, meaning “in-mental-state”:

inm(c, x, t) ⊃ concept(c) ^ temporal-entity(t)
inm(c, x, t) ⊃ (∃p, m)[agent(p)^mind(m, p)^part(x, m)]

That is, inm(c, x, t) means that concept c is in part of the mind x at time t. The predicate temporal-entity(t) can indicate either an interval or an instant (Gordon, Hobbs, 2003).

In the system of Gordon and Hobbs, people can perform two main actions with their memory: store and retrieve. Storing is defined as follows:

agent(p) ^ focus(f, p) ^ memory(m, p)
 ⊃ (∀ c, t2)[
  store(p, c, t2) ≡
   (∃t1)change(
    inm(c, f, t1) ^ ┑inm(c, m, t1),
    inm(c, m, t2), 
    t2
 )
]

Or in other words, a person storing a concept c means that his mind changes from having c in focus, but not in memory, to having c in memory. Note that this axiom does not specify whether the concept is in focus after storing. Similarly, for retrieving:

agent(p) ^ focus(f, p) ^ memory(m, p)
 ⊃ (∀ c, t2)[
  retrieve(p, c, t2) ≡ 
  (∃t1)change(
    inm(c, m, t1) ^ ┑inm(c, f, t1), 
    inm(c, f, t2) ^ inm(c, m, t2), 
    t2
  )
 ]

The person retrieving the concept c changes his mind from having c in memory, but not in focus, to having it in both focus and memory. The predicate change is defined in a theory of changes, that’s also connected to a theory of time. The use of the predicate change on formulas is a shorthand for a more complex representation that involves reifying inm and other relations, that is, transforming them so that they are quantifiable variables (Gordon, Hobbs, 2003).

Gordon and Hobbs point out that in most other theories of belief there are no limitations for accessing beliefs. This, however, does not represent how humans truly operate. We forget things all the time, remembering things too late or desperately trying to recall others (Hobbs, Gordon, 2005B). That’s why they introduce the notion of accessibility, which is a partial ordering of concepts for a given person’s memory:

memory(m, p) ^ inm(c, m, t) ⊃ (∃a)a = accessibility(c, m, t)

Every concept in a person’s memory has a specific accessibility at time t. The higher the accessibility, the easier it is to retrieve the concept. We also define a memory threshold for a person:

a0 = mthreshold(p) ≡
 (∃m)[
  memory(m, p) ^ 
  (∀a1, c, t)[
   a1 = accessibility(c, m, t) ^ a1 < a0 ⊃ ┑retrieve(p, c, t)
  ]
 ]

Simply put, if the accessibility of a concept is below the memory threshold, then that concept cannot be retrieved (Gordon, Hobbs, 2003). Accessibility, however, can be changed. After all, in order to remember concepts, people use different mnemonics all the time, from tying strings on their fingers to setting alarms. In order to accommodate for these actions, Gordon and Hobbs introduce the action remind:

remind(c1, p, c2, t) ≡ 
 (∃ f, m)[
  focus(f, p)  ^ 
  memory(m, p) ^ 
  cause(inm(c1, f, t), retrieve(p, c2, t), t)
 ]

That is, concept c1 reminds person p of concept c2 at time t, if and only if the fact that c1 is in person p’s focus f at time t causes p to retrieve c2 (Gordon, Hobbs, 2003). The predicate cause defines a causal relationship and is defined in a theory of causality. The use of logical expressions inside the predicates is again a shorthand that could be eliminated if we reify inm and retrieve. One concept can remind a person of another concept, if those concepts are associated in that person’s mind as expressed in the Axiom of Accessibility by Association:

associated(c1, c2, p) 
 ⊃ (∃ f)[
  focus(f, p) ^ 
  (∀ t1, t2)(∃ a1, a2, t3)[
   cause(
    change(┑inm(c1, f, t), inm(c1, f, t2), t2), 
    change(
     a1 = accessibility(c2, p, t2), 
     a2 = accessibility(c2, p, t3), t3)
   ) ^ 
   a1 < a2
  ]
 ]

This axiom states that if c1 is associated with c2, that means that if c1 comes into a person’s focus that will cause the accessibility of c2 to increase. If c2 was previously below the memory threshold, this change of accessibility may make it retrievable again (Gordon, Hobbs, 2004).

One serious problem in Hobbs and Gordon’s theory is the fact that the only way for a concept to get into an agent’s memory is for it to be stored:

inm(c, m, t) ^ memory(m, p) ⊃ (∃t1)[t1 < t ^ store(p, c, t1)]

This axiom rules out pre-existing Platonic ideas in the memory (Gordon, Hobbs, 2003). However, it also introduces some problems. For example, what if a person stores a concept, but, after a while, that concept degrades in his memory, and changes. Technically, it is a new, different concept that has appeared in the person’s memory without being stored. It can be easily seen that in Hobbs and Gordon’s theory, concepts in memory are quite rigid, in contrast to our experience of memories that have slightly changed details or are altogether false.

This can be solved by applying a theory of “graded” belief, which Gordon and Hobbs have just sketched out for the moment. An agent can believe a proposition to a certain degree. Key properties of this system is that the degree of belief does not diminish under modus ponens, that the degrees of a proposition and its negation vary inversely, and that multiple independent supports for a proposition increase its degree of belief. Graded beliefs over some threshold become full-fledged beliefs, in a way similar to how the memory threshold acts (Hobbs, Gordon, 2005B). No more details on this graded belief system are available at the moment. It seems similar to fuzzy logic and Hobbs and Gordon claim that it accommodates nonmonotonic reasoning, even though they admit they have axiomatized the graded belief “in a very abstract and noncommittal manner” (Hobbs, Gordon, 2005B). Even without the details, it’s clear that we can apply graded beliefs to concepts in memory. The degree of belief can indicate how certain the agent is that a given concept has not changed since it was stored in memory. The degree can then change depending on external evidence, other stored concepts, and how much the agent trusts its memory at a given time. This slight change would make Hobbs and Gordon’s theory of memory better reflect the fluid nature of memories.

Andrew Gordon and Jerry Hobbs have been working on this formalization attempt for several years, but there is still a lot of work to be done. As noted before, only about half of the representational areas are completed. This can create many problems. Let’s take for example another action in the Memory area, repression:

repressed(c, p, t) ^ memory(m, p)
 ⊃ accessibility(c, m, t) < mthreshold(p)

repressed(c, p, t) ⊃ unpleasant(c, p)

repressed’(e, c, p, t) ⊃ partially-cause(unpleasant(c, p), e, t)

A repressed memory has accessibility below the memory threshold and repressing a memory is at least partially caused by that concept being unpleasant to the person at time t. These axioms use two predicates that are outside of the Memory area: unpleasant and partially-cause. If those predicates are not yet defined, we would run into problems trying to make any sort of inference about repression. Hobbs and Gordon seem to understand that this is a problem, but since so many of the areas are tightly connected, it seems impossible that there is a solution (Gordon, Hobbs, 2003). While the final version of the theory strives to be able to make deep human-like inferences about common sense problems, the current incomplete version does not support even the simplest of inferences because of how interconnected the representational areas are. In addition, actually trying to use the system for inferences doesn’t seem to be one of the priorities of Hobbs and Gordon. Currently, they are trying to finish the 30 representational areas. This is reasonable, but it means that the system will be tested only after it is completed. That is, at the moment, we don’t know whether any part of this theory will be useful in any way.

This system tries to capture and formalize the important concepts of common sense psychology in such a way as to allow AI agents to be able to reason about everyday problems and interactions with other intelligent agents. One main problem with Hobbs and Gordon’s approach is pointed out by Thomason, who notes that even when attempted on a moderate scale the formalization of common sense results in sets of axioms that put considerable pressure on even the most powerful logical systems. This applies to Hobbs and Gordon’s system as well, which is a rather massive undertaking, much bigger than most other such systems. In an effort to create a powerful system that covers all aspects of common sense psychology, they might have created something that’s too complex for any practical application. In addition, by covering all the aspects of common sense psychology, such a system would not only be capable of correct common sense reasoning, but it would also fall victim to many common fallacies as well.

Hobbs and Gordon address the previous criticisms by discussing the philosophy that led them to develop their system in such a way. They discuss how it’s important that a formalized commonsense theory should have a high degree of competency and coverage. Researchers usually focus on generating competency theories that can answer problems that humans are able to solve. Competency theories have a focus on the function of axiomatic theories rather than their form. Hobbs and Gordon note the fact that such axiomatic theories should also have a wide breadth and cover many of the concepts needed for common sense reasoning. They argue that without addressing the problem of coverage first, competency theories would be resistant to extensions and would be impossible to integrate with each other or with larger cognitive models (Gordon, Hobbs, 2003). The fact that some of the problems with this theory exist because of decisions that were consciously made at the beginning of the process is certainly a positive sign. It also means that Gordon and Hobbs are aware of these problems and that as more representational areas are completed, the impact of those problems will lessen. Their system is so complex and massive, because they believe that, otherwise, it will simply fail to scale to bigger problems or integrate with other theories.

It can be argued that the complexity issue is not unique to the effort of Gordon and Hobbs. Any axiomatic system for common sense reasoning might be too difficult to comprehend and highly unpredictable, as well as extremely hard to create. Similarly, there are currently no methods or criteria for evaluating work in this field, which makes it impossible to know whether a given system is on the right track. This is not much of a hurdle for small formalization problems, where we can just treat formalization as an art, rather than a discipline with well-defined methodology. However, the current trends show that researchers are beginning to focus on larger systems and more complicated common sense reasoning problems, which would mean that field would require carefully thought out and tested methodology. Thomason proposes that the field needs something similar to the software engineering practices in computer science that were developed in order to handle the increasing complexity of programs (Thomason).

Maybe more importantly than the previous concerns, it’s quite possible that trying to axiomatize common sense might be the wrong approach to this issue altogether. The AI community at large is somewhat skeptical about such formalization projects (Thomason). This might be due to the general trend in recent years towards bottom-up approaches in AI, since axiomatic systems are decidedly top-down. A major criticism of top-down approaches is the fact that babies are not born with an understanding of common sense reasoning and do not learn about it by memorizing an explicit set of axioms. Instead, they learn from their experiences to create an informal system for common sense reasoning. Similarly, it might be too much to expect that robots and other AI agents will start their existences able to apply commonsense reasoning. A more reasonable view is that such agents will learn common sense by interacting with their environment and other agents. For example, Eureqa is a Cornell software project that detects mathematical relationships and laws in data (Keim). It’s easy to see how its abilities could be easily modified to approximate our common sense understanding of physics, or naive physics, by gathering information from the environment, much like a human infant would. Similarly, it could learn other important common sense concepts such as causality, time, memory, planning, etc, without a human having to program them all in. Such an approach has the benefit of being a lot more flexible and robust than a static axiomatic system, that could fail to work if one of the axioms doesn’t capture common sense reasoning properly.

In conclusion, while Hobbs and Gordon’s massive attempt at formalizing common sense is interesting and very ambitious, its scope is also the main reason why it fails. Even after years of work, the system is not yet completed, and the end result might be too complex and unwieldy for any practical application. However, I do believe that such formalization efforts can be useful when applied on smaller domains that produce manageable axiomatic systems. In addition, working on large scale formalization systems might produce unexpected insights and developments in the field by motivating the creation of a more strict methodology.

Thanks for reading! If you have any questions or suggestions, please leave a comment!

Note: I wrote this paper as part of Dr. Curtis Brown’s Non-Classical Logics course at Trinity University during the spring semester of 2012

Bibliography:

Gordon, Andrew S., and Jerry R. Hobbs. “Formalizations Of Commonsense Psychology.” AI Magazine 25.4 (2004): 49–62. Applied Science & Technology Full Text (H.W. Wilson). Web. 14 Apr. 2012

Gordon, Andrew, and Jerry R. Hobbs, 2003.”Coverage and Competency in Formal Theories: A Commonsense Theory of Memory’’, Proceedings, AAAI Spring Symposium on Logical Formalizations of Commonsense Reasoning, Stanford, California, March 2003.

Hobbs, Jerry R., and Andrew S. Gordon, 2005. “Encoding Knowledge of Commonsense Psychology”, Proceedings, 7th International Symposium on Logical Formalizations of Commonsense Reasoning, Corfu, Greece, pp. 107–114, May 2005.

Hobbs, Jerry R., and Andrew S. Gordon, 2005. “Toward a Large-Scale Formal Theory of Commonsense Psychology for Metacognition”, Proceedings, AAAI Spring Symposium on Metacognition in Computation, Stanford, CA, pp. 49–54, March 2005.

Hobbs, Jerry R., and Andrew Gordon, 2010. “Goals in a Formal Theory of Commonsense Psychology”, in A. Galton and R. Mizoguchi (eds.), Formal Ontology in Information Systems: Proceedings of the Sixth International Conference (FOIS 2010), IOS Press, Amsterdam, pp. 59–72

Keim, Brandon. “Download Your Own Robot Scientist.” Wired. N.p., 3 Dec. 2009. Web. 22 Apr. 2012. <http://www.wired.com/wiredscience/2009/12/download-robot-scientist/>.

Morgenstern, Leora, 2001, “Mid-Sized Axiomatizations of Commonsense Problems: A Case Study in Egg Cracking”, Studia Logica, 67(3):333–384.

Shanahan, Murray, 2004, “An attempt to formalise a non-trivial benchmark problem in common sense reasoning”, Artificial Intelligence, 153(1–2): 141–165

Thomason, Richmond. “Logic and Artificial Intelligence.” Stanford Encyclopedia of Philosophy. N.p., 9 May 2009. Web. 22 Apr. 2012. <http://plato.stanford.edu/entries/logic-ai/>.

This paper aims to provide a rounded defense of the idea of pancomputationalism, which I will define thusly:

If every program that a computer runs is a computation, then every physical process is a computation.

I try to do this by both showing how the above is the logical consequence of a straight-forward premise and by responding to common criticisms of the idea of pancomputationalism. I start with what I believe to be a reasonable assertion and then go on to show how it logically leads to increasingly broad definitions of computation. I also respond to some criticisms to pancomputationalism covered in Piccinini’s “Computational Modeling vs Computational Explanation”. Then I show that pancomputationalism isn’t only a reasonable view of computation, but is also a useful way to look at the universe that can inform modern physics and other branches of science.

1. Running Any Program on a Computer is Computation

I’ll start with the claim that running any program on a computer (or any TM-equivalent) is computation. This seems straightforward enough since the notion of computation began with exploring what human computers do and formalizing it, which resulted in the notion of Turing Machines. Thus, running any Turing Machine should be considered a computation, regardless of the complexity of the program. This means that programs that consist of empty infinite loops or programs that return the same output for any input are computations.

One criticism to this view would be that this definition of computation will include many processes that wouldn’t be considered useful or interesting computations. I consider this to be a rather anthropocentric view of the problem. We do not hold many other natural phenomena towards the same “usefulness” principle. For example, the slingshot effect is a phenomenon in which an object approaching a much larger object can use the larger object’s gravitational well in order to accelerate. Scientists have used this effect to accelerate spacecraft such as Voyager I, but the same thing has been documented to happen when supermassive black holes eject entire stars from their system. Yet, we do not distinguish between the useful instances of the slingshot effect and the purely accidental ones. The same could be said about many other phenomena and there’s no reason that computation should be treated any differently.

That being said, a notion of “useful computation” is needed, as there are many practical and philosophical implications of such types of computations. However, limiting the extent of our definition of computation is not the only way to create a notion of “useful” computations. For example, it could be achieved by creating a hierarchy of types of computations, in the same vein to what Piccinini and Scarantino do in their paper “Information Processing, Computation, and Cognition”. Instead of defining the set of computations so as to exclude all “uninteresting” computations, we can just define a set of useful computations that is a subset of the much larger set of general computations.

2. Baking a Cake is Computation

Recipes are often likened to algorithms for a reason, and I would argue they achieve a certain relation between what their outputs and inputs represent. In the case of cake, the output represents a delicious dessert. Even if we don’t consider a dessert to be a proper output of a computation, we can still envision how varying some of the parameters of the cooking process (ex. temperature and length of baking) will result in a change of output (such as amount of moisture in the resulting cake). Whether we can easily achieve any useful computations using this method is questionable, but as I pointed out in Section 1 that should not be a basis for disqualifying the process from being considered a computation. In fact, some successful research has been done on chemical computers, which use concentrations of chemicals to represent data and chemical reactions to perform the computations. This shows that chemical reactions can be used to implement algorithms and, thus, following the logic in Section 1 means that even the simplest of chemical reactions can be viewed as computations.

3. Weather is Computation

In “Information Processing, Computation, and Cognition”, Piccinini and Scarantino claim that they reject at least a version of pancomputationalism because “it conflates computational modeling with computational explanation.” Certainly, I agree that a rough computational model is not enough to merit being used as a computational explanation. However, I disagree with the generality of this view because I’d argue that a sufficiently detailed simulation is indistinguishable from reality for all intents and purposes. That is, a simulated storm does thunder (if the simulation is detailed enough, for an observer within that simulation). If we plug in a consciousness to witness a well-simulated storm, it will not be able to detect the difference between a simulated one and a real one. Thus, claiming that a storm is not computation is a relative rather than an absolute viewpoint. In addition, we could theoretically use a weather pattern to compute (though we obviously currently lack the technology to do so), but that further strengthens the idea that a storm is a computation, albeit a rather uninteresting one.

4. Stone is Computation

This should be an simple logical conclusion from the points argued in the previous section, since weather and stones are the same type of general physical phenomenon. One difference might be the complicated chaotic behavior of weather and other such systems, but stone is not really static either, since at the very least atoms always move at temperatures above 0 K. Thus, it could potentially be used as to compute certain functions by noticing how its properties change in response to changes in environment. I think it is clear that if we can consider even stones to be computation, then that claim could be easily extended to include all physical processes.

5. Response to Piccinini

This section will focus on Piccinini’s arguments against pancomputationalism presented in his paper “Computational Modeling vs Computational Explanation”. As the paper name suggests, one of the main arguments is that computational modeling and computational explanation are inherently different and this difference is the reason why the thesis of pancomputationalism is false. To make his point, Piccinini gives an example of computational explanation by focusing on a calculator computing the square root of a number: “the behavior of the calculator is explained by the fact that it performs a square root computation” (6). Here Piccinini seems to be begging the question by claiming that the calculator’s behavior can be computationally explained because it performs computations. But can’t we apply this logic to all other processes, if we consider them computational? For example, if we consider evolution as a process that finds a set of genes that ensure the survival of an individual, then we can explain the behavior of systems that are undergoing evolution by the fact that they are performing a computation that finds a good set of genes for the given environment. The same linguistic gymnastics could be performed for many other, less complex physical phenomena. For example, bubbles always taking the shape of a sphere can be explained computationally by the fact that the bubble system is computing an optimal shape that tries to maximize volume, while minimizing surface area.

Another one of Piccinini’s criticisms of computational modeling as a valid explanation of physical phenomena stems from the fact that “[i]n modern science, dynamical descriptions are usually given not by means of computing systems but by systems of differential equations, which determine a continuous state space, which assigns an uncountable number of possible states and state space trajectories” (12). He claims that because Turing Machines can only have an enumerable amount of states, they would fail at capturing the exact essence of the system, unlike differential equations. Here Piccinini makes an assumption about how well differential equations describe reality. While the math used in modern physics does determine continuous state space, isn’t that also an approximation of how the physical universe works? Modern physics has concepts such as Planck length and Planck time, that are defined as the smallest possible units (of distance and time respectively) that make any physical sense. This means that in some sense the universe might not be continuous, but discrete. Therefore, systems of differential equations might not be an exact descriptions (just fairly accurate ones), while a discrete computational system might be better. However, Piccinini seems to accept that differential equations are somehow inherently better than computational systems at describing physical phenomena. He mentions that “there is no independent evidence that the universe is fundamentally discrete” and that “no one knows what the most fundamental physical level is” (13). However, this logic goes both ways, as there is no definite proof that the Universe is continuous either. In addition, most differential equations used in physics, especially for mechanics, are known to be approximations and not accurate descriptions of the physical world, since most of them break down when applied on very small scales and high energies. Thus, even though they are used to great success, they do not accurately reflect the nature of the Universe and cannot be claimed to be inherently more accurate descriptions of reality than computations.

Piccinini also claims that a flaw of the thesis of pancomputationalism is the fact that for many computing systems it would be impossible to miscompute. It is certainly true that it would not make sense to talk about a storm miscomputing. Talking about miscomputing does not make sense in the case of pancomputationalism, but only when we apply a different definition of computing, such as a semantic one. However, why should the ability to miscompute be an important feature of computing systems? Especially from a theoretical standpoint, the ability of a computing system to miscompute is never discussed. After all, the tape of an abstract Turing Machine never breaks. The ability of our physical computing systems to miscompute stems from our faulty implementation, rather than some inherent feature of computation.

One of the biggest flaws of Piccinini’s paper is that he bases some of his arguments on what seem to be dogmatic ideas, rather than rational explanations. When arguing about how computation should be a more restricted notion than a pure functional approach, he writes:

“ If we like, we may start calling every function of every system a computation. As a result, computation will acquire explanatory force by piggybacking on the explanatory force of functional analysis. But this also turns every system that is subject to functional analysis into a computing system. Functional analysis applies to artifacts, organisms, and their components, including stomachs and other paradigmatic examples of non-computing systems” (24).

Here, Piccinini seems to refuse that stomachs can be computations solely because they’re a paradigmatic example of non-computing systems, rather than some other reason. However, a claim being paradigmatic is not enough to establish its truth, since paradigms, especially in science, get overturned often. It seems that a lot of philosophers who oppose pancomputationalism start their idea of computation in a negative way, by excluding certain systems that they do not believe are computation. I, instead, try to build my definition of computation in the opposite way, by starting with something that I believe is widely considered a computation and then following that definition to its logical end.

6. Advantages of Pancomputationalism

At this point of the paper a reader might have agreed with every statement that I’ve made and, yet, still have the question: “Why does this matter?” I believe this is where the strongest argument for pancomputationalism comes in. It is not only the logical extension of a simple claim, but it is also a very powerful and useful concept. This is because we can apply several ideas from computer science to augment our understanding of the universe or to show useful lines of research.

One strength of the pancomputationalist view is that it creates a strong connection between computer science and physics. For example, Lloyd and Ng show in their paper “Black Hole Computers”, that using a pancomputationalist approach in cosmology can lead to some very interesting ideas in both physics and computer science. For instance, they show that, using approaches from information theory and computer science, one can calculate the computational speed of the universe. Additionally, they use this approach to propose that it is possible to use astronomical objects, such as black holes and neutron stars, to perform computations. Following this approach, they are even able to identify details about these theoretical machines, such as their speed and the type of computation they perform. For example, they point out that because of their physics, black holes can be used as incredibly fast serialized computers, while a box consisting of hot plasma might offer a parallelizable alternative. While a farfetched notion at the moment, with the ever-advancing rate of technology, it might be possible one day to put these ideas into fruition.

There are other strengths to using pancomputationalism to look at our Universe. We could use pancomputationalism to draw parallels between diverse fields that could give us insights as to how all those fields can be unified. For example, a pancomputationalist could argue that there is a connection between the behavior of chaotic systems in physics and the halting problem in computer science. Chaos theory deals with complex systems, in which the tiniest change of initial conditions can lead to wildly different results. Many such systems are well known to science and include weather systems and star systems. The halting problem is a famous proof in theoretical computer science that states that there is no general approach to finding out whether or not a computation will ever halt. Similarly, in chaos theory, there’s no shortcut to determining whether a chaotic system will ever settle in a stable configuration or continue its chaotic behavior indefinitely.

Furthermore, connecting computer science and physics will also make it possible to see parallels from other closely related fields, such as logic. For example, we can argue that there are physical examples of Gödel’s Incompleteness Theorem. Because the speed of light is limited and the Universe continuously expands, there is a giant event horizon around Earth beyond which lie portions of the Universe that we’ll never be able to reach or interact with. Thus, there are certain statements about our Universe that might be true, but they can never be proven because of the limitations that our current understanding of physics imposes. While that does not seem to be too practical of a parallel, I would argue that it shows that there are many phenomena in these diverse fields that can be seen as just different sides of the same idea. I think this shows us a glimpse of what a unified theory of science might look like: diverse issues that can all be explained by the same general principles.

Another area that will benefit from a pancomputationalist approach is the philosophy of mind. Currently, there are a lot of debates as to whether the mind is a computation or not. Given the premise that everything is a computation, it would follow trivially that the mind is also a computation of sorts. This is seen as a problem by some scholars. Piccinini writes in his paper “Computational Modeling vs Computational Explanation”: “[I]f everything is a computing system, it is unclear how computation could have anything interesting to do with inference, rationality, executing instructions, following rules, or anything else specific to explaining mental phenomena” (4). I’d argue that computation, inherently, is not the source of any such type of phenomena. After all, both the simplest calculator and the most advanced AI system that we have are computations, yet the latter is vastly more complicated than the former and can perform at least some of the mental phenomena mentioned. Thus, we know that computation, by itself, is not responsible for such complex intellectual abilities. Computation isn’t some sort of magical spark that gives minds consciousness or any of the other mental capacities we have. If that were the case, we would have achieved truly conscious machines a long time ago. What’s important are the algorithms that the minds are running. For example, what was groundbreaking about Marr’s book about vision was not so much the claim that vision is a computational process, but the fact that he identified the specific algorithms that made vision possible in animals. By accepting that the brain is trivially a computing system as a consequence of pancomputationalism, philosophers would be able to focus on the issue of the exact nature of the computations that the mind performs, which I consider to be a vastly more interesting problem than whether the brain computes at all. After all, if everything is a computation, that brings up the very intriguing question of what is so special about the algorithms that brains are running that give them the multitude of cognitive capacities that we struggle to implement in our state-of-the-art computing systems. This discussion would hopefully not only be philosophically elucidating, but will also inform AI researchers and help them with designing more intelligent systems and maybe, one day, even true AI.

7 . Conclusion

The idea of pancomputationalism presented in this paper is based on a simple, straight-forward claim: if we consider all programs run on computers to be computations, then everything is a computation. Some opponents to such ideas of pancomputationalism present arguments that I believe to be too anthropocentric and, thus, flawed. I try to present a definition of computation that makes it an objective, absolute property. Other critics seemingly construct their definitions of computations by having preconceived notions of what isn’t a computation, which invariably leads them to conclude that pancomputationalism is a wrong approach. I have presented what I believe to be a more constructive approach to defining computation that begins not by excluding certain processes, but defining a core of what I consider to be obvious computations and expanding on it. I believe that, while the idea of pancomputationalism might seem foreign and unintuitive to many, in time, it will prove to be both illuminating and extremely useful in many areas of science and philosophy.

Thanks for reading! If you have any questions or suggestions, please leave a comment!

Note: This paper was written as part of a Philosophy of Computation seminar led by Dr. Curtis Brown during the spring semester of 2013 at Trinity University. While not typical of the kind of work I’ll be posting on this blog, I think that my stance in regards to pancomputationalism is important because it necessarily leads me to certain conclusions about the nature of consciousness and how it should be recreated using AI.

References:

“Chemical computer”, http://en.wikipedia.org/wiki/Chemical_computer

“Event Horizon”, http://en.wikipedia.org/wiki/Event_horizon

“Gravitational slingshot”, http://en.wikipedia.org/wiki/Gravitational_slingshot

Gualtiero Piccinini, Andrea Scarantino, “Information Processing, Computation, and Cognition” (2011)

Gualtiero Piccinini, “Computational Modeling vs. Computational Explanation: Is Everything a Turing Machine, and Does It Matter to the Philosophy of Mind?” (2007)

Lloyd & Ng, “Black Hole Computers,” Scientific American, 2004

Ray Villard, “Hyperfast Stars Point to Black Hole Slingshot”, http://news.discovery.com/space/hyperfast-stars-point-to-black-hole-slingshot-121024.htm