<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Hazem A. A. Nomer on Medium]]></title>
        <description><![CDATA[Stories by Hazem A. A. Nomer on Medium]]></description>
        <link>https://medium.com/@hazemaanomer?source=rss-72b659e6531f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*2QgSctOG8TrU1Yvr7w9oag.jpeg</url>
            <title>Stories by Hazem A. A. Nomer on Medium</title>
            <link>https://medium.com/@hazemaanomer?source=rss-72b659e6531f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 14:32:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hazemaanomer/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why strong AI might be impossible]]></title>
            <link>https://medium.com/@hazemaanomer/why-strong-ai-might-be-impossible-b8e59b73d1fb?source=rss-72b659e6531f------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8e59b73d1fb</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Hazem A. A. Nomer]]></dc:creator>
            <pubDate>Thu, 07 Apr 2016 01:07:25 GMT</pubDate>
            <atom:updated>2016-04-07T15:39:13.898Z</atom:updated>
            <content:encoded><![CDATA[<p>First of all, I am a strong believer in general artificial intelligence. Here I try to list possible reasons that make “strong AI” hard. These reasons will show how the hype of “AI will kill us one day” is not quite right, and if you really understand the fundamentals of artificial intelligence, you will be not so convinced with this hype.</p><p>We don’t have a very rigid definition of what intelligence is, but we know what is its artificial form. AI is how to devise algorithms that act like humans. This already happens in many fields as computer vision, natural language processing, speech understanding,etc… We have neural networks that see pictures,understand speech, understand text and even generate new things. It seems like the problem is solved, but these are not the only things we do. We think, feel, act, plan,dream and most importantly we evolve to be better. We are aware of ourselves and the environment surrounding us. We ask questions and seek answers. <br>We simply didn’t reach the AI we want yet. Deep learning might be promising but not the radical solution.</p><p>Let’s assume we are going to build an AI agent that passes <a href="https://en.wikipedia.org/wiki/Turing_test">Turing test</a>, we call it “Boho”. Boho will be entirely built from different types of neural networks. Each network or a group of networks will be responsible for some function of the brain. Boho can run continuously forever and it was never trained before. Boho has a module called “Trainer” which contains a lot of training algorithms, Boho can call “Trainer” any time it thinks it is necessary to train all or part of its neural networks. Boho only deals with text, images from a camera and some sensor data. You can talk with Boho with a text interface. <br>For each <a href="https://thebeautyofml.wordpress.com/2016/03/25/in-nutshell-neural-networks/">neural network</a> Boho has we will define two things: the model of neural network (how it is organized) and the search space (the possible weights of the network).</p><p>Remember Boho has the ability to train itself when it need, modify the code of the “Trainer” and change the structure and the topology of its neural networks. <br>Now we describe problems we may encounter when building Boho.</p><ol><li>The Halting Problem:<br>Imagine that you work as a software tester and your supervisor gives you a large file with thousands of lines of code. You want to go home as soon as possible and you can’t because of this extra task your supervisor just gave you. You are a very smart geek who believes in automation and decided to write a script that will automatically test your code given an input (let’s assume you don’t know the output). You are stuck, Alan Turing has said that you will never be able to write this code. The halting problem is that you can’t write an algorithm that determines from a program and a given input on this program whether it will halt (finish execution of the last instruction) or will loop forever. Alan Turing proved that such an algorithm can not exist. <br>Boho now can not know if the code it modified will halt or not. Of course, Boho will have the ability to find a modified code that is better from the point of view of its goal (objective function) and will have the ability to proof that is correct (although somehow hard) but it doesn&#39;t necessarily mean that it will halt for all possible inputs. We may need a different computational model from Turing machines, perhaps a model that sees the outcome of computations ahead for several time steps. One promising model which is quite similar to Boho is <a href="http://people.idsia.ch/~juergen/goedelmachine.html">Gödel Machine</a>.</li><li>P vs. NP: <br>It is a very well know problem in computer science, given a problem if you have an input and an output for this problem you can check that the output is correct in polynomial time, Can you find the output in polynomial time ? <br>The problem can be reformulated as: “Can you find an algorithm for an NP (nondeterministic polynomial time .. it takes the age of the universe to solve it for a large input) problem that runs in polynomial time ?” <br>No one knows the answer to P vs. NP problem, although many computer scientists believe that P does not equal NP. <br>Almost all artificial intelligence problems from the perspective of theoretical computer science are NP-complete (the hardest problems in NP) or NP-hard (you may not find an algorithm to solve it). As a machine learners (or using other techniques) we are competing to find the best approximate algorithms that solve these problems. <br>Boho networks search spaces are large thus requires faster search algorithms to find the optimal weights for its connections. Even if we have the fastest computer on earth (aka quantum computer), it will not solve the problem. Boho still doesn&#39;t know whether its neural networks structures are good or not. Boho also -if it is truly intelligent- should know whether a neural network is good or it has to use another model or perhaps a different form of neural network. There are some promising techniques as <a href="http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf">neuroevolution</a> for finding optimal topologies. <br>The problem does not lie in the search space, the problem is the model. We are sure that if our brain is organized differently, we will not be the same. The strengths of connections between neurons in your brain are probably gained by years of evolution, and neurons fire at the same rate for healthy persons. The learning rule is not as important as the structure of the brain.</li><li>It is about statistics<br>Boho should be non-deterministic. Once it becomes non-deterministic, it exhibiting a sign of intelligence. Today&#39;s machine learning methods (even deep learning) are about statistics. Boho neural networks will try to capture the distributions of the data continuously fed to them. All machine learning methods depend on the data distribution, so it is possible using some analysis to predict the behavior of a learning algorithm on some data before execution and thus predicting the behavior of the model.<br>Boho may not be able to become non-deterministic. Humans themselves can be predicted. Boho self-modification of his own code is one trick that will make him unpredictable for humans. Every time you get readings from Boho behavior, he changes his behavior next time.</li></ol><p>Finally, AI agents (say robots) if become intelligent they will not necessarily be smarter than us, and if it happens humans who are in control and we define them as “smart” already use their intelligence in destruction so robots will not be doing something new.</p><blockquote><em>I would like to thank</em><a href="http://ayounes.page.tl/"><em> Dr. Ahmed Younes</em></a><em> for his wonderful explanation of the halting problem last term, which contributed a lot to this article.</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8e59b73d1fb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[In a Nutshell: Neural Networks]]></title>
            <link>https://medium.com/@hazemaanomer/in-a-nutshell-neural-networks-fc1f66240804?source=rss-72b659e6531f------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc1f66240804</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Hazem A. A. Nomer]]></dc:creator>
            <pubDate>Fri, 25 Mar 2016 23:42:43 GMT</pubDate>
            <atom:updated>2016-04-29T19:57:44.452Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong><em>I will try not to make these posts just a replication of what is already on the web. These posts will discuss machine learning, neural networks, and deep learning in a different fashion as much as possible. You will sometimes find these posts discuss theoretical and philosophical aspects of these fields. Nutshell posts goal is to take baby steps towards these fields and great leaps in understanding the core concepts.</em></strong></blockquote><p>Artificial Neural Networks (ANNs) are very powerful tools (devices) in machine learning. A neural network is a simplified model of the networks that exist in your brain. That is why it is a revolutionary idea because it is (and many other variants of it) inspired by the human brain.</p><p>To describe neural networks first, we have to describe its building block the “neuron”. You can imagine a neuron as a binary (yes or no / true or false / 0 or 1) decision maker, you want to take this decision based on some indicators (predicators). To take this decision you have to weight these indicators, these weights should show how important each indicator in your decision. Let’s say you want to buy a new mobile phone, your indicators will be the price, the CPU type, the battery life and the screen size. Your budget is low and you want something cheap and have a long battery life other indicators are not so important. Your weights are between 0 and 1.<br>The neuron work as follows you input your indicators (as numbers) and it sums the multiplication of each indicator by its corresponding weight, pass this through a function called activation function. Let’s assume this function will make the output either 0 or 1 and if the sum is too high it will output 1 and vice-versa.<br>Now if the weight of the price is high the probability that the neuron decision will be 1 is high which is incorrect you want a low price also, you want a long battery life so the weight corresponding to the battery life will be high. Other factors (CPU and screen size) may have low weights indicating that they are not important in your decision process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*GDLYbftdzFbG1wBs." /></figure><p>The above figure shows a neuron, h here is a function, the arrows that go into the neuron here represent the weights the (+1) is called a bias. Think of bias as a way to make your neuron not biased to a certain output (you simply multiply (+1) and bias weight)</p><p>A network of neurons is called a neural network, the neurons are organized in layers. Outputs of neurons in layer 1 are inputs to neurons in layer 2 and so on. Each neuron in layer (<em>L+1</em>) takes all the outputs of all neurons in layer (<em>L</em>) as its input, so you have a fully connected neural network. Neurons of the same layer are not connected to each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jzmFiUBZdXEkBs1i.png" /></figure><p>The figure shows a neural network with three layers: input layer (which is not actually a layer), hidden layer and output layer. You can see that, activations (output of neuron after passing the weighted sum of inputs through an activation function) of the second layer act as inputs to the next layer, the edges represent weights.</p><h3>The Math of Neural Networks</h3><p>It is easier to understand neural networks in terms of matrices and vectors. Your input is a vector X = [x1,x2,x3, …, xn]. The weights of a layer are the elements of a matrix W, the number of rows is the number of neurons in this layer and the number of columns is the number of neurons in the previous layer. Each row in W represents the weights associated with the input to the neuron in the layer. The bias b is a vector, you can think of a bias as a neuron in the layer that always output (+1). The vector holds bias terms in the layer.</p><p>To compute the output of the neural network h(x), you first need to compute a1, a2, a3, and to compute these values you simply multiply X (input vector) by W and add the bias vector b. The result is a vector of size (1 x number of neurons in the layer). The activation function then acts on each element in this vector, and the result is again a vector of the same size. This vector acts as an input to the next layer which in turn is multiplied by the weights of the layer and added to the bias vector then passes through the activation function.<br>Why we use this notation of matrices and vectors ? Because actually it how the neural network is implemented. Neural networks are not implemented as graphs because the neurons themselves are not important, but the weights and the biases are the core of the neural networks. A learning algorithm is a way to find weights and biases. You compute the output of a neural network using the weights and biases.<br>The matrix equations for the neural network (layer 2 &amp; output layer) in the figure is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/205/0*_jS1ULfbpTsJzewh." /></figure><h3>Neural Networks and Universal Approximation</h3><p>You might see the similarity between neurons and linear regression. Linear regression equation of multi-variables is: <strong>Y = WX+b</strong>. An algorithm as gradient descent finds the parameters of this model (W and b). The difference between the two is that neurons are non-linear models if the activation function is non-linear.<br> It is proved that a neural network with two hidden layers can approximate any function, it is a universal approximator. From the point of view of algorithm analysis, an algorithm is a function of its inputs. This means that a neural network can approximate an algorithm which is interesting. If we do not know the algorithm for a problem we can now train a neural network to have a behavior that nearly equal to that of the correct algorithm. This means that the neural network will not always output the correct result. The approximate algorithm that we have now in the form of the weights and biases of the neural network is optimal (just the complexity of the matrix multiplication algorithm times the number of layers).</p><p>This is a very good news now we can find approximate algorithms for NP problems, simply by training a neural network to give an output with some pretty good accuracy. All the previous assertations are true because almost all artificial intelligence problems, and now neural networks already solve some with high accuracy, are NP-complete problems.<br>The bad news that neural networks themselves are hard to train, but you can get around this problem. The real bad news is that two hidden layer neural networks are not enough to solve all problems. The second problem is that feed forward neural networks lack a very important part of computation which is a memory, it can not remember. This means that feed forward neural networks can not solve a problem that a Turing machine can solve. It can not even solve the problems that a push-down automaton can solve (both models have a memory).<br>The real catastrophic thing is that we do not know how many neurons in each hidden layer we should have to solve a problem or even the number of layers enough.<br>The bottom line is the model we have just described is simple even with two hidden layers.</p><h3>What hidden layers do ?</h3><p>A hidden layer in a neural network computes things that we do not know about the input. If you use a neural network as a black box (you don’t care about its internals), you really do not know what is happening inside. Each hidden layer tells something about the input that we do not know but is helpful in computing the final output. We say that each hidden layer computes (if the weights are assigned properly) features about the data. If your data is the pixel of an image, the first hidden layer detect edges, the second layer detects lines, the third learns the human eye shape and so on. You can think of each vector of activations for each hidden layer as a feature vector after passing a filter on your image. The same concept can be applied to any type of data.<br>How does this happen ? The magic lies in the learning algorithm. What the learning algorithm do is adjusting weights to get the correct output from the neural network using many data examples (dataset). To get the correct output the activations from the previous layers should be correct too. The algorithm only sees the input and the correct output.</p><blockquote><em>Originally published at </em><a href="https://thebeautyofml.wordpress.com/2016/03/25/in-nutshell-neural-networks/"><em>thebeautyofml.wordpress.com</em></a><em> on March 25, 2016.</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc1f66240804" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>