<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arjeus A. Guevarra on Medium]]></title>
        <description><![CDATA[Stories by Arjeus A. Guevarra on Medium]]></description>
        <link>https://medium.com/@arjeusaguevarra?source=rss-db0332ebf22e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*B0gTOtTQdksop1vG.</url>
            <title>Stories by Arjeus A. Guevarra on Medium</title>
            <link>https://medium.com/@arjeusaguevarra?source=rss-db0332ebf22e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 11:52:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@arjeusaguevarra/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Updates as of July 2021 (Dynamic Boolean Network)]]></title>
            <link>https://medium.com/machine-learning-based-on-boolean-logic/updates-as-of-july-2021-27426e175d72?source=rss-db0332ebf22e------2</link>
            <guid isPermaLink="false">https://medium.com/p/27426e175d72</guid>
            <category><![CDATA[boolean-logic]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Arjeus A. Guevarra]]></dc:creator>
            <pubDate>Thu, 22 Jul 2021 05:42:02 GMT</pubDate>
            <atom:updated>2022-02-06T03:26:59.097Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*03leEHVPn5nSfKn16SpMNw.png" /><figcaption>From Flaticon. Designed by Freepik</figcaption></figure><ol><li>Since the previous work is only for expository purposes, the static network, which causes termination, is now <em>dynamic</em>, consisting of 2 steps per iteration:<br> Recall that once a “<em>backward hierarchical fixing</em>” fails, the following two methods are added:<br> a. <em>Simplification</em>, where neurons are removed or pruned while maintaining exact equivalence of the unmodified network. <br> b. <em>Augmentation</em>, where the network adds neurons and still maintaining equivalence of the unmodified network.</li><li>Allowing <em>architectural search</em> poses challenge to achieving high accuracy and also constrained by using lower Boolean variables per node.</li><li>A simple training scheme is developed by preserving last model and augmenting new neurons into it.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27426e175d72" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-based-on-boolean-logic/updates-as-of-july-2021-27426e175d72">Updates as of July 2021 (Dynamic Boolean Network)</a> was originally published in <a href="https://medium.com/machine-learning-based-on-boolean-logic">Machine learning based on boolean logic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Boolean Logic Networks with hidden layers]]></title>
            <link>https://medium.com/@arjeusaguevarra/boolean-logic-networks-with-hidden-layers-fa80b1a0501d?source=rss-db0332ebf22e------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa80b1a0501d</guid>
            <category><![CDATA[expert-systems]]></category>
            <category><![CDATA[symbolic]]></category>
            <category><![CDATA[logic]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Arjeus A. Guevarra]]></dc:creator>
            <pubDate>Sun, 29 Dec 2019 14:18:07 GMT</pubDate>
            <atom:updated>2019-12-29T14:18:07.110Z</atom:updated>
            <content:encoded><![CDATA[<p>Naively adding layers of Boolean in leads to halting or non-termination.</p><p><strong>Introduction</strong></p><p>On my previous <a href="https://medium.com/machine-learning-based-on-boolean-logic/machine-learning-through-logic-simplification-on-predicting-isf-2d5a88fec847">post</a>, I introduced the feature of Boolean Logic Simplification as a form as Machine Learning. Here, we’ll try to make the networks deep, which is similar to multilayer perceptrons, to introduce the <em>hierarchy of logical formulas</em>.</p><p><strong>Objective</strong></p><p>Naively create a hidden layer to the logic network and draw insights from its performance.</p><p><strong>Experimentation</strong></p><p>Code can be found at: <a href="https://github.com/arjay55/Logic_learn_12202019">https://github.com/arjay55/Logic_learn_12202019</a></p><p>A <em>dense hidden layer</em> is added this time. Each <em>node </em>is an object consisting of the ff:</p><ol><li>logical formula</li><li>Truth table evaluated as 1</li><li>Truth table evaluated as 0</li><li>Truth table of don’t cares</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/958/1*_5y_qs4aaIude5lYk5bzDw.png" /><figcaption>A 2 layer Boolean network. Each node has an SOP of the inputs.</figcaption></figure><p>As you can see from the figure above, the network is essentially like a typical dense neural network, but the description of the node is essentially a <em>logic formula in SOP or </em><strong><em>Sum of Products</em></strong><em> form</em>.</p><p>The errors, where the <em>node(j,k )</em> XOR <em>node(j+1,k)</em> == 1, the set correct value is obtained from the inputs. If the inputs are from the set of don’t cares <em>node(j,k), </em>the formula is then recomputed with the new truth table. Otherwise, the next value from don’t cares dataset are used as correct values from <em>node(j,k ) </em>thus training the affected nodes using connections at <em>node(j- 1,k)</em>. This is analogous to “<em>backpropagation” </em>in multilayer perceptrons. As defined from the previous post, the <em>updated truth table</em> is part of the node’s attributes for further updates coming from the<em> don’t cares set</em>.</p><p><strong>Training algorithm:</strong></p><p>The training algorithm is analogy of a “<em>test-driven development</em>” in software engineering. If the inference commits an error, then a training is sample is obtained for training. The process repeats until 100% accuracy is achieved. The goal is to achieve 100% where the data used to train is<em> less than 100%.</em></p><p><strong>Experiments on 1 layer</strong>:</p><ol><li>As expected, the training is successful, achieving 100% logic equivalence. However, this resulted to the <em>immediate intractability (NP-hard)</em> nature of Boolean logic generalization. You can run the earlier version of the code () to review the training of the dataset</li></ol><p>As a dense network is naively added, some issues may occur:</p><ol><li>In this experiment, the “backpropagation” technique together with “test-driven” learning gets stuck in infinite loop, due to same error when forward propagated, causing failure to learn. This may be due to any of the following:</li></ol><p>a. Excessive connectivities result to incorrect assumptions, as demonstrated by <em>Occam’s Razor</em>. This is analogous to <em>overfitting </em>in other machine learning algorithms.</p><p>b. The failure to learn is analogous to only getting optimized to <em>local optimum.</em></p><p><strong>Conclusions:</strong></p><p>The network of Boolean nodes(nodes that contain logical formulas), require further research into trainable architectures with generalization properties. Comments on this research are welcome.</p><p><strong>Further work:</strong></p><p><em>There is an ongoing research on a new hypothesis in creating trainable Boolean Logic Network, which may serve as continuation of this article/post.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa80b1a0501d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multivariable time series forecasting using Stateless Neural Networks]]></title>
            <link>https://medium.com/data-science/multivariable-time-series-forecasting-using-stateless-neural-networks-e88afdd5cd82?source=rss-db0332ebf22e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e88afdd5cd82</guid>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[multivariate]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Arjeus A. Guevarra]]></dc:creator>
            <pubDate>Tue, 27 Aug 2019 14:39:07 GMT</pubDate>
            <atom:updated>2019-08-27T14:39:07.205Z</atom:updated>
            <content:encoded><![CDATA[<p>Forecasting with multiple variables that aids a target variable with stateless deep learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ClAOhtPIWe5gTk3M" /><figcaption>Photo by <a href="https://unsplash.com/@jontyson?utm_source=medium&amp;utm_medium=referral">Jon Tyson</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Time series forecasting with assisting variables</h3><p>Time series. Datasets that have a time element with them. Such data allow us to think about the combination of <strong><em>2</em></strong><em> properties</em> of time series:</p><ol><li><strong>Seasonality </strong>— Patterns in data that tends to repeat over and over at a specific length of time.</li><li><strong>Trend </strong>— This is similar to regression, where we are capturing the global pattern of the series.</li><li>The relevance of data tends to <em>center at</em> <em>present time, </em>meaning past and data close to present time is of greater influence and accuracy of future predictions is better when closer to present data (principle of <em>entropy</em>).</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*ciU5nqeSKJtTJ0PtBBU3Mw.png" /></figure><p>Such a combination is what makes time series special. The data at a point in time depends on the previous data. Intuitively, this is the <em>cause and effect </em>nature of events, such as the words that you’re reading makes sense based on the previous words.</p><p>Normally, machine learning only captures the trend, which makes training a typical machine learning model over time series expensive.</p><h3>Forecasting a single variable with supporting variables</h3><p>Normally forecasting a <em>single variable</em> involves creating a model that is a function of previous time data. This is known as <em>autoregression of single variable.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/153/1*HltwJpgMUFSiUi0yBFl_0w.png" /><figcaption>[1]Autoregression formula. (Wikipedia)</figcaption></figure><p>The image above is a sample of an autoregression formula. [1]</p><p>We will not dive into the details of the formula, but notice the ff variables:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/72/1*hFuE2LmHzqTp9wg4V5GRIg.png" /></figure><p>→<strong>dependent variable</strong>. The value of data at present time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/72/1*ahEL06MssH3N73drWqN6sg.png" /></figure><p>→<strong>independent variable</strong>. Note that <em>previous data</em> are given values to the variables.</p><p>The rest are additional parameters and variables that refine the accuracy of the model.</p><p>But what if we have additional hypothetical variables that are correlated with the target variable? These could increase the accuracy of the target variable that we need through proper modelling.</p><h3>Algorithms and Models</h3><p>When you think of time series, you immediately think of LSTMs, GRUs and other <em>recurrent neural networks</em>. But in this case we refuse to do this and instead rely on <em>feedforward neural networks</em>. Why is it? Let’s illustrate the two and why I chose to follow the latter:</p><ol><li><strong>Recurrent neural networks — </strong>These are graphs whose output is fed back to the input. As inference is being done, the output of the network are stored as states, ready to be inferred on the next time step.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fDV9INjZlKJeFkU5OUsRSw.png" /><figcaption>RNN and its states through time. Source: <a href="https://upload.wikimedia.org/wikipedia/commons/e/ee/Unfold_through_time.png">https://upload.wikimedia.org/wikipedia/commons/e/ee/Unfold_through_time.png</a></figcaption></figure><p>The recurrent network attempts to predict the next timestep through the following:</p><ol><li><em>At </em>→ Actual input value at time <em>t.</em></li><li><em>Xt</em> → The previous state of the RNN or the modelled prediction</li></ol><p>With the variables above, the network achieves its predictive power by minimizing the error between the <em>modelled data</em> and <em>actual data. </em>However, notice that as the RNN passes through different temporal data, the error of predictive model increases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7BE1dq9G55dA1Nt0MxalLg.png" /><figcaption>Continuous state transformation worsens the transformation through time.</figcaption></figure><p>LSTMs and GRUs appear to solve this problem, as it contains gates that accepts or forgets previous data that are considered irrelevant. However, these gates and data transformations cannot specifically be controlled into a critical aspect of time series analysis: how various <em>timestates </em>relate to one another.</p><p>In the image below, the timestep becomes a vector to a cell with an activation function. Thus, the relationship of the timesteps are captured vs the target future step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/630/1*Quj2tPSm2dt5LM-coa8I2Q.png" /><figcaption>Feedforward network that relates various timesteps vs the next timestep</figcaption></figure><p>We can then move to the implementation proper for the specifics of the pipeline and the model designed.</p><h3><strong>IMPLEMENTATION</strong></h3><p>The complete code implementation is located on Github at <a href="https://github.com/arjay55/time_series_stateless.git">https://github.com/arjay55/time_series_stateless.git</a></p><p><strong>I. Libraries</strong></p><p>The libraries mainly consist of the ff:</p><ol><li><strong>pandas → </strong>for data preparation</li><li><strong>Scikit-learn (sklearn) and scipy</strong>→ for external loss function and for scaling</li><li><strong>seaborn → </strong>for heatmap visualization</li><li><strong>skopt</strong> → an optimization module which includes Bayesian optimization. We will use this for hyperparameter tuning during training.</li><li><strong>TensorFlow </strong>→ an open-source library for deep learning. We used the latest version, <em>tensorflow-beta1.</em></li></ol><p><strong>II. Data Preparation</strong></p><p>We will only load the normalized dataset for data privacy.</p><p>First, we load the normalized dataset.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8f30db74de9c5b2a8900a310427c15db/href">https://medium.com/media/8f30db74de9c5b2a8900a310427c15db/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/416/1*1ms5uXdi0_LeMMSRJubycA.png" /><figcaption>Snip of the Dataframe. Note the missing data on <strong><em>inflation </em></strong>column</figcaption></figure><p>Upon examining the DataFrame, we have 3 columns<br>1. <strong>mutual_unit</strong> → this is the trust fund units, convertible to the currency of investment. The higher its value, the more returns you have on your investment.<br>2. <strong>stock_exchange</strong> → this is the index for an aggregate of stock prices from various companies in a company.<br>3. <strong>inflation </strong>→ inflation rate of a given country.</p><p>These variables are chosen as they are representative characteristics of financial status of a given country.</p><p>Note that the <em>inflation </em>column has a lot empty or null values. One technique is to fill the data with assumed values:<br>1. Linear interpolation in the upper and lowermost indices of the dataset.<br>2. ARIMA based extrapolation beyond the furthest indices. Why? It is easier to model the outer portions using 2 forecasting steps only.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/887a1d696245d71299caeb4ba2c50d9c/href">https://medium.com/media/887a1d696245d71299caeb4ba2c50d9c/href</a></iframe><p>We saved the file to a csv for consumption of another programming language, R. I chose to use this as its <em>“auto-arima”</em> function is popular and mature compared to Python equivalent.</p><p>We will have 2 outputs:<br>1. backasted[2] values to <strong><em>x </em></strong>steps → “<em>backasted</em>” means forecasting backwards, or simply reversing the time series and performing forecasting.<br>2. forecasted values to <strong><em>y</em></strong> steps → the usual forecasting.</p><p>The <em>x</em> and <em>y</em> values are determined by function <strong>get_outernan </strong>and <strong>fill_data </strong>fills the missing data with assumed values.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c2f4bf5fac0da553d8a1535dad99010a/href">https://medium.com/media/c2f4bf5fac0da553d8a1535dad99010a/href</a></iframe><p>In <em>Rstudio</em>, we load the interpolated inflation dataset and perform simple ARIMA:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/84fbc1ee589fa6ace70c455694ea1280/href">https://medium.com/media/84fbc1ee589fa6ace70c455694ea1280/href</a></iframe><ul><li>In <em>forecasting, </em>regular ARIMA is done</li><li>In <em>“backcasting”, </em>notice the use <em>rev()</em> function prior to timeseries <em>ts() </em>function to transform data to timeseries.</li><li>The get <strong>get_outernan </strong>function returned values <em>(6,15), </em>where it is the number of steps to <em>backcast </em>and <em>forecast </em>respectively.</li></ul><p><strong>III. Data Exploration</strong></p><p>Now that we have prepared the data for analysis, we can start exploring them to examine its data quality and simplify the data using domain expertise.</p><p>First, let’s refine our objective. Remember that we are intending to forecast a single variable. That variable is the <strong>mutual_unit</strong>. We have 3 variables</p><p>Our intent is to <em>forecast the </em><strong><em>mutual_unit </em></strong><em>variable</em> with the aid of <em>helper variables, </em>which are <em>stock_exchange</em> and <em>inflation.</em></p><p>Let’s plot the three variables:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/1*lWrI8Bck4bmyq5ZRvGz2bg.png" /><figcaption>Line plots of the three variables</figcaption></figure><p>Upon examining the graphs, there may be some correlation between the mutual_unit and stock_exchange. For inflation, the values have a pivot from <em>going up</em> to <em>going down</em>.</p><p>As an additional analysis let’s perform a timestep vs timestep <em>correlation analysis. </em>We’ll use a <em>heatmap </em>to review the correlations.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0bc4ee0248eb580c57f1a7c0abdeeaf7/href">https://medium.com/media/0bc4ee0248eb580c57f1a7c0abdeeaf7/href</a></iframe><p>On the first snip (<em>correl.py</em>), it produces a heatmap.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*_9eyFt-XxgPIzR528YykDQ.png" /></figure><p>We can see that <em>mutual_unit </em>has high levels of correlation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/377/1*VDMiGSzjw6Q11PI4FG5quw.png" /><figcaption>Correlation of mutual_unit to other variables</figcaption></figure><p><em>Note:</em> As of the time of this writing, it appears that <em>inflation </em>has negative correlation with the <em>mutual_unit. </em>This could be that the unit goes higher to recover from the increase in inflation. You could try reducing the time window of interest and it may increase the model accuracy.</p><p>In our model, we need to specify the <em>weights </em>of the variables. This is the 2nd part of the code (<em>correl2</em>). Since we are using <em>stock_exchange </em>and <em>inflation </em>as helper variables, their weight will have these effects:</p><ol><li><em>Higher weights </em>mean higher correlation with the target variable.</li><li><em>Lower weights</em> is the opposite of the above. If higher weights are specified, then the noise of the variable will overfit or make the target variable less accurate.</li></ol><p>To be set as weights for our model. The correlations are scaled as follows:</p><p>ABS(VARIABLEx)/SUM(ABS(VARIABLES))</p><p>where VARIABLEx is the correlation coeff. divided by the sum of the variable’s correlation coeffs. Note that <em>absolute values</em> are taken prior to other computations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/359/1*8Qf6kiN2f2CXbBc-mwJNgw.png" /><figcaption>computed initial weights of the variables</figcaption></figure><p>Note that the three have about the same proportion as they have correlations versus mutual_unit.</p><h3><strong>IV. Prediction Pipeline</strong></h3><p>This is the part where we apply machine learning to make predictions! This is not magical of course, the process is still stochastic and mathematical. It is just its information processing techniques that make its prediction impressive.</p><p>The type of prediction we make is <strong>regression</strong>, where we are predicting continuous values. In <em>regression</em>, we model <em>continuous values</em>. In <em>classification</em>, we model discrete values. In the image below, a certain metric is represented in two types. In discrete or <em>classification</em>, the values are <em>mild</em>, <em>moderate </em>or <em>severe</em>. In the right axis, the metric is represented as continuous values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/958/1*CUSD4JUf1QUCNCfw1X5W3w.png" /><figcaption>Comparison between regression and classification.</figcaption></figure><h3>Temporal processing of datasets</h3><p>Our data is a time series, therefore, apart from its feature values, there is a forward moving time. While our dataset contains trend and seasonality, it is important to truncate the dataset into a rolling window. Why? Because using the entire history may overfit the model.</p><p>In our case, <em>1 month of window may suffice, or 4 business weeks</em>. This assumption means that most of the properties can be modeled using this window.</p><p><em>Note: From this point I will mention the function that are used in the implementations below.</em></p><p>We also need to define the <em>scope of rolling window</em>. In our case, we will assume <em>3 months of training data</em> to characterize the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/1*WfXaGvoBOiyBe-9j3E_XrA.png" /><figcaption>Illustration period of training scope and model scope. The right side of the illustration is the recommended dataset.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*sh60bhp1ClyOhWZGzJ7utQ.png" /><figcaption>Dataset creation using rolling window. Photo credit and source from <a href="https://www.researchgate.net/profile/Huaiqing_Wang">Huaiqing Wang</a>, <a href="https://www.researchgate.net/figure/Diagram-of-building-up-the-dataset-Stock-time-series-segmentation-is-made-by-20-width_fig2_276890244">Diagram of building up the dataset</a></figcaption></figure><p>We’ll now split the train and test datasets:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8412cfa51e451d38a21f07480eff015c/href">https://medium.com/media/8412cfa51e451d38a21f07480eff015c/href</a></iframe><p>The code above shows the creation of <em>train</em> and <em>test dataset</em>. The result is a <em>TimeseriesGenerator </em>that already creates the rolling window datasets.</p><p>We are splitting the train/test dataset by 80%/20%. Note that the test data has batch size of only 1, where as we will encounter later, the test set will be compared <em>1 by 1</em> to form the <em>validation error.</em></p><p>The training window size is <strong>20</strong>.</p><h3><strong>Model Architecture</strong></h3><p>The model is of <em>Sequential</em> in nature, meaning the layers are stacked into a <em>single input/single output </em>without any branching.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/1*Q3U0YY3HALA06Q7Rp1nfxQ.png" /><figcaption>Model architecture from <strong>TensorBoard</strong></figcaption></figure><p>The model starts from <em>“dense” </em>then ending at the <em>“dense_2” </em>layer.</p><ol><li>The 1st layer (<em>“dense”</em>) is simply a <em>dot multiplication</em>, which is equivalent to linear <em>vector autoregression </em>(VAR), with adjustable number of parameters for added complexity. It does not contain stochastic parameters.</li><li>The 2nd layer (<em>“dense_1”</em>) is an additional layer to learn further nonlinearities.</li><li>The Next layers are namely, <br><em>a. Dropout </em>layer —layer that randomly disconnects certain weights during training. This helps the network learn nonlinearities in logical forms while reducing overfitting.<em><br>b. Flatten </em>layer — layer that permutes the <em>previous hidden layer</em> into the <em>output </em>layer. The <em>output dimension</em> is equal to the <em>target variable’s dimension</em>.</li></ol><pre>Function: train_evaluate()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/630/1*gangHck1d6H780CqqbZK-w.png" /><figcaption>Dropout illustration. Source: Srivastava Nitish, et. al, Dropout: <a href="http://jmlr.org/papers/volume15/srivastava14a.old/srivastava14a.pdf">A Simple Way to Prevent Neural Networks from Overfitting</a> (2014), Journal of Machine Learning Research 15</figcaption></figure><h3>Adding temporal training techniques on the input layer</h3><p>Recall that <em>recurrent networks </em>only model the current and the next timestep. What if we only want an <em>autoregressive model</em> that characterizes the <em>next timestep</em> with the previous timesteps? (See Figure above, “<em>Feedforward network that relates various timesteps vs the next timestep</em>”). We’ll then a stateless network as it simplifies the model.</p><p>Another important characteristic is <em>input masking</em>. Recall that our input shape has 20 timesteps. We will <em>not use these </em>shape as input layer shape:</p><ol><li>Using all timesteps may introduce overfitting.</li><li>As we are training as <em>rolling window </em>to capture the seasonality, we will use input skips to capture the seasonality at different scales.</li><li>Further timesteps have lesser relevance to present value.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/298/1*R1kNuRdr-fUOqs25xMpDmg.png" /><figcaption>Masking the inputs. The timesteps highlighted as <strong>gray </strong>are masked.</figcaption></figure><p>With these, we only have <em>11 inputs</em> at input layer. You can try different combinations to mask.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c37f5acfc96a0a571cd328135b76dbd2/href">https://medium.com/media/c37f5acfc96a0a571cd328135b76dbd2/href</a></iframe><h3>Hyperparameter tuning</h3><p><em>Hyperparameters </em>are mainly the adjustable arguments when instantiating or training a model.</p><p>As an example, the gist above instantiates a <em>“Dense” </em>layer. Those that are parts of the <em>kwargs </em>dictionary are hyperparameters:</p><ol><li>The <em>‘input_layer’ </em>is the number of neurons created. Adjusting these will directly affect the model’s capacity to adjust to nonlinearities (underfit or overfit).</li><li>The ‘<em>uni_reg’ </em>applies L1 regularization to that layer. Higher parameter values will adjust more slowly, reducing the risk of <em>overfitting</em>.</li></ol><p>Here are the model’s hyperparameters:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0c8fd23a5df9bcdf51266dde5fe392b1/href">https://medium.com/media/0c8fd23a5df9bcdf51266dde5fe392b1/href</a></iframe><p>The gist above is created using the library called “<em>skopt.space</em>”, where you specify the ranges and categories to optimize.</p><p>Afterwards, we wrap the entire pipeline into a function and run the optimization function command. This is the reason we created a large function called</p><pre>train_evaluate(normalized_data, est_period,loss_func,param_record, **kwargs)</pre><p>It contains model creation, validation process, etc. Feel free to study it.</p><p>Now, to the hyperparameter optimization command.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cde0bed06712b30142a7165a6ec659d6/href">https://medium.com/media/cde0bed06712b30142a7165a6ec659d6/href</a></iframe><p>We are optimizing the parameters using <em>Bayesian optimization of Gaussian Probabilities</em>. It essentially maps a <em>probabilistic model</em> of the optimization space. An assumption on navigating space between <em>exploitation </em>(suspected value) and <em>exploration</em>(creating real data points) [4]</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*a-pPzw9C8dVx4Jh1KohQdQ.png" /><figcaption>Continuous improvement using Bayesian optimization. Source: Gilles Louppe, Manoj Kumar, <a href="https://scikit-optimize.github.io/notebooks/bayesian-optimization.html">Bayesian optimization with </a><a href="https://scikit-optimize.github.io/notebooks/bayesian-optimization.html">skopt</a> (2016)</figcaption></figure><h3>Test/Validation</h3><p>Testing this autoregressive problem has a lot of components:</p><ol><li><em>Test sets during training</em></li></ol><ul><li>these are the test values where features at X(t+1) are predicted</li></ul><p>2. <em>Iterative prediction for multi-step prediction</em></p><p>First, let’s remind ourselves of our target variable, <strong><em>mutual_unit </em></strong><em>.</em></p><p>In order to perform the validation of this variable, the 3 variables need to be predicted at X(t+1). The input data are pushed to the left, then the latest timestep data are replaced with the predicted data. The process are iterated to achieve the iteration at <em>multi-steps.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/518/1*Ce-tHYE0F-ib_JPjkLkfow.png" /><figcaption>Process on how the model forecasts in multi-steps. <strong>Xn</strong> is real data point where <strong>Hn</strong> is a predicted value</figcaption></figure><p>The function that does this is</p><pre><strong>validation()</strong></pre><p>The <em>validation </em>function does the iterative forecast and is compared with the corresponding test data. Note that only <em>mutual_unit</em> is only held out and compared.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/492d073b28b6a8aebe2c9a15fa9af7ec/href">https://medium.com/media/492d073b28b6a8aebe2c9a15fa9af7ec/href</a></iframe><p>Now, the vectors <em>y_pred</em> and <em>y_true </em>are future values. Entropy or unpredictability increases as time steps are forecasted further. Therefore we will introduce a <em>discount factor</em> to these values. In our application, the discount factor is <em>0.9.</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8add523aaa2f7cae0c3ccc1b0a47a2cf/href">https://medium.com/media/8add523aaa2f7cae0c3ccc1b0a47a2cf/href</a></iframe><p>After this, the <em>mean absolute error </em>is computed<em>. </em>This will be the main validation metric as measure of the model’s performance.</p><p>This <em>MAE </em>will serve as metric for <em>Early stopping</em>.</p><p>Early stopping is applied to our validation metric (<em>MAE</em>) to mitigate the risk <em>overfitting. </em>It stops once the validation error begins to <em>increase</em>.However, it has the risk of not full reaching full optimality as the validation value is assumed to be <em>convex</em>, as once the inflection point of the error is founded, training is halted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/312/1*9Fo_BKE5JjkkJBQbbUTiaA.png" /><figcaption>Early stopping illustration. Source: Shubham Jain, <a href="https://www.analyticsvidhya.com/blog/2018/04/fundamentals-deep-learning-regularization-techniques/">An Overview of Regularization Techniques in Deep Learning (with Python code)</a> (2018),</figcaption></figure><p>I used a hand-coded Early stopper and not an API from TensorFlow due to a customized validation computation plus it is easier to code it rather to find an appropriate API. The object name is</p><pre>class CustomEarlyStopper:<br>	def __init__(self, patience=3, delta=1e-4):<br>	...</pre><p>It has similar parameters like <em>patience </em>and <em>delta, </em>which acts as a tolerance to increases in validation error.</p><p>Let’s take a look between predicted and actual values. Notice the predicted values have a positive offset above the predict value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*k3EptW3dPP_HummSR6U_pg.png" /><figcaption>Comparison between the predicted(‘mutual_unit_forecast’) and actual(‘mutual_unit’)</figcaption></figure><h3>Finalization</h3><p>We are now preparing the model for inferential use.</p><ol><li>Using the final parameters, we reoptimize the model to reach the same validation error during hyperparameter tuning. These can be obtained through loading the optimization object.</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3817613988c7d2a611b2635d1ebb5f3a/href">https://medium.com/media/3817613988c7d2a611b2635d1ebb5f3a/href</a></iframe><p>In the notebook, you can notice the functions<em> jump()</em> and <em>load() </em>to make <em>persistent</em>(save to disk). The functions are placed to facilitate usage of objects upon system interruption.</p><p><em>Note: You can refactor the code so that the model optimized with the best hyperparameters are made persistent(user) so as to avoid re-training as above.</em></p><p>2. After this, capture the last <em>training loss </em>of the last optimal training. Note that this is the <strong>training loss</strong> to apply and NOT validation loss. This technique can also be refactored.</p><p>3. Finally, we have to train the model using the entire dataset. To reduce the risk of overfitting, we’ll get the <em>previous training loss</em> from the last trained model. We’ll <em>stop </em>the model training once the target training loss is achieved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*sWuNPP-w_94M0SHYJD5AKw.png" /></figure><h3>Final Remarks</h3><p>Thank you for reading and hope you learned about the basics of time series forecasting, from data preparation to modelling. I had fun sharing this experience in as much as having fun learning, discovering and applying deep learning on simple time series forecasting.</p><h3>References</h3><p>[1] <a href="https://en.wikipedia.org/wiki/Autoregressive_model#Characteristic_polynomial">Autoregressive model</a>, Wikipedia</p><p>[2] “Rob J Hyndman”, <a href="https://robjhyndman.com/hyndsight/backcasting/">Backcasting in R (2014)</a></p><p>[3] <a href="https://en.wikipedia.org/wiki/Vector_autoregression">Vector autoregression</a>, Wikipedia</p><p>[4] Gilles Louppe, Manoj Kumar, <a href="https://scikit-optimize.github.io/notebooks/bayesian-optimization.html">Bayesian optimization with </a><a href="https://scikit-optimize.github.io/notebooks/bayesian-optimization.html">skopt</a> (2016)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e88afdd5cd82" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/multivariable-time-series-forecasting-using-stateless-neural-networks-e88afdd5cd82">Multivariable time series forecasting using Stateless Neural Networks</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine learning and reasoning through Boolean logic]]></title>
            <link>https://medium.com/machine-learning-based-on-boolean-logic/machine-learning-through-logic-simplification-on-predicting-isf-2d5a88fec847?source=rss-db0332ebf22e------2</link>
            <guid isPermaLink="false">https://medium.com/p/2d5a88fec847</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[occams-razor]]></category>
            <dc:creator><![CDATA[Arjeus A. Guevarra]]></dc:creator>
            <pubDate>Tue, 23 Jan 2018 23:36:04 GMT</pubDate>
            <atom:updated>2019-05-20T16:31:34.091Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>“Among competing hypotheses, the one with the fewest assumptions should be selected” Occam’s Razor</em></h4><h4>Introduction of Boolean-based machine learning through simplification algorithms and <em>don’t cares</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WBj1XYypxT6j5IJEK7CoaA.png" /><figcaption>Image by <a href="https://pixabay.com/users/Clker-Free-Vector-Images-3736/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=23330">Clker-Free-Vector-Images</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=23330">Pixabay</a></figcaption></figure><p><strong>I.</strong> <strong>Introduction:</strong></p><p>What is knowledge? As defined by a google search of the term, it means</p><p><em>“facts, information, and skills acquired by a person through experience or education; the theoretical or practical understanding of a subject.”</em></p><p>Thus, to gain knowledge, we need to transform incoming stimulus to a certain <em>representation</em>. In order to represent it, we need to <em>“compress or simplify”</em> the incoming stimuli as we have limited resources in our brain.</p><p>This article is a revision and updated article that I previously posted with greater clarity and explanation. The previous post is located <a href="https://arjayvisionsthoughts.wordpress.com/machine-learning-through-logic-simplification-on-predicting-isf/">here</a>.</p><p>The purpose of this is to expose my learnings towards a broader audience and to add to the pool of knowledge on the advancement of artificial intelligence.</p><p>Initially, I experimented on Boolean Logic based Machine Learning using a commercial synthesis tool specifically for creation of digital ICs. The methodology involves the following:</p><p>a. Verilog-HDL code</p><p>b. Case statements to define input and output values</p><p>c. The use of x or dontcares in the case statement to represent missing values</p><p>d. Commercial logic synthesis tool</p><p>What I did now is to implement the simplification methodology on a <em>software programming paradigm</em>. I also did more metrics in evaluating the machine learning performance based on <em>predictive power</em> and <em>runtime</em>.</p><p><strong>II. Related literature</strong></p><p>My intent for this is to give the reader existing technologies on about the concept I’m about to share. It was after I published my article on Wordpress about the technology of an analytics company called <a href="https://www.rulex.ai/">Rulex Analytics</a>. They are already using the use of logic simplification techniques with the core algorithm of “Switching Neural Networks” and fast logic simplification algorithms like “Shadow Clustering”. The advantage of using rule-based (or logic based) machine learning is that the model is not black box. It produces logical statements that can be easily transformed into human comprehensible sentences. Visit their website to check out their services and their <a href="https://www.rulex.ai/technology/technical-papers/">publications </a>as well.</p><p>While this may be the case, there’s a new publication in Quanta Magazine about how popular deep learning ‘black box’ characteristic can be understood. See the article <a href="https://www.quantamagazine.org/new-theory-cracks-open-the-black-box-of-deep-learning-20170921/">here</a>.</p><p><strong>III. Core algorithms</strong></p><p>Before I discuss the core algorithms used in this research, let’s define the motivating factor for my pursuit in simplifying combinational logic:</p><p>1. Simplifying combination logic has a direct impact on an algorithm’s performance. Once the logic is simplified, it can be converted to a bytecode or delete unneeded logic representations as the number of terms in a simplified logic is at a minimum. <br>2. Simplified logic in it’s simplest form aligns with <strong>Occam’s Razor</strong>. Occam’s Razor states that <em>among competing hypotheses, the one with the fewest assumptions should be selected</em>. This is the foundational concept that I’ve been applying for my research, and I believe that the human brain seeks to simplify its stored information in order to accommodate new information and to easily form more complex concepts and models.</p><p>Mathematically and in statistics, Occam’s razor can be restated as follows:<br>i. <em>By definition, all assumptions introduce possibilities for error; if an assumption does not improve the accuracy of a theory, its only effect is to increase the probability that the overall theory is wrong.</em></p><p>The statement above is similar to the concept of overfitting, that, if given the opportunity, is to simplify your predictive model rather than introduce “magnitude-based” types regularization.</p><p>On the previous evaluation, logic synthesis tools are used to simplify the logic with don’t cares. It has been known that the algorithms used for such simplification utilize the <em>espresso algorithm</em>. The <em>espresso algorithm</em> is a heuristic of logic simplification that is able to reach a sufficiently simplified logic in ASIC designs. While its logic is not totally simplified, the runtime increase of the algorithm is much faster compared to <em>Quine McCluskey</em> method, which is an exhaustive but completely simplified form.</p><p>However due to the fact that espresso algorithms do not return a completely simplified logic, it’s predictive power may <em>not be maximized</em>. I haven’t dug deep into how the algorithm works, but for the purpose of proving the predictive power of simplest information representation, I have resorted to Quine McCluskey algorithm where it produces the simplest Boolean representation to a function.</p><p><strong>IV. Test Setup</strong></p><p>The test will be again interpolating a function with incomplete input data, or <em>Incompletely Satisfiable Function(ISF)</em>. This time the function will be learning is squaring the input function.</p><p><strong>Input values:</strong></p><p><em>X ⊆ [1,∞], where X is an integer represented in binary notation.</em></p><p><strong>Output values:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/518/1*dKQgnQTBjZtMYYq7PWIMxQ.jpeg" /></figure><p><strong>A. Libraries and language used</strong><br>a. <em>Python </em>is the language of choice for this experiment due to it’s syntactical simplicity and faster development. <br>b. Libraries used:</p><p>i. <em>Pickle →</em> used to directly save variables stored in RAM as a file in hard disk<br>ii. <em>Pandas →</em> Used to analyze results and performance quickly through its data analysis features<br>iii. Timeit →used to monitor the time elapsed per learning iteration<br>iv. Matplotlib → used to plot values from data analysis (pandas)<br>v. Sympy → Symbolic library which contains Boolean algorithms like Quine-McCluskey algorithm</p><p><strong>B. Basic algorithm</strong><br>The <em>Quine McCluskey</em> algorithm can simplify logic expressions to their simplest POS(Product of sums) or SOP(Sum of products) form. Unlike the simplification done in logic synthesis, it uses espresso algorithms and other process related mapping and optimizations to achieve the best logic circuit in terms of power, area or timing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/852/1*8IVnHzHoRjrX7kEvxAgbcg.jpeg" /></figure><p><strong>Initialization of data:</strong><br>1. The untrained samples will be all input combinations, from 1 to infinity. (In this case, it is just finite. The specific values will be identified later). These values will be part of the dontcares list.<br>2. The list of minterms are empty.</p><p><strong>Scope of verification:</strong><br>Logic simplification can only produce an output of 1 bit, and therefore, we will focus only on certain bits to verify and interpret. These are bits (from LSB):</p><p>A. 11th bit<br>B. 14th bit<br><em>The algorithm does the ff to introduce learning:</em><br>1. The tool starts learning from input values and squared output values going beginning from no. 1.<br>2. The learning starts from and ascends by increments of 1.<br>3. The tool simplifies the value of 1 at a certain output bit Y.<br>4. The simplified logic on bit Y is verified from 1 in ascending order:<br> a. If value is incorrect, the list of dontcares and minterms are updated based from the verified values and logic simplification is done.<br> b. If value is correct, the loop continues to the next digit for verification. The list of dontcares and minterms are also updated based from the verified values. <br>The machine learning algorithm assumes perfect memory experience, that is, the results of verification are immediately captured by logic simplification algorithm by updating the list of input dontcares and minterms.<br>5. Every time a correct value is achieved, the runtime and the list of correct values are stored in a database for further analysis.</p><p><strong>V. Analysis and Presentation of results</strong></p><p>1. <em>11th bit results</em>:To begin with, the verification and input value range is finite. The initial range or training data has a range from 1 to 1023. This is enough to produce substantial results for the 11th bit:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/573/1*-l0qSYwl_hFvCa7aL_zKMQ.png" /><figcaption>Figure 2. The figure below shows the plot between last correctly verified value and corresponding runtimes.</figcaption></figure><p>In order to plot a continuous graph despite the last correct values and its runtimes are recorded, the correct the value is computed as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/123/1*nHsC_w8OUvyYxehr32SKIg.png" /></figure><p>where LCNx is the last correct value, LCNX-1 is the previous last correct value, and</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/29/1*juluxAYoEcw5k20FNkDjQQ.png" /></figure><p>is the range of the two LCN. <em>This will then include that machine learning capacity will be also a function of the runtime LCN vs runtime plot.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*l75JwooToGEL3b1KeKgRCQ.png" /><figcaption><strong>Figure 3</strong>. The figure below shows the same plot as Figure 2, but this time, the rolling mean is computed in order to filter out noise in the data to verify reduction of runtimes as more data is acquired. The window size is 50 samples.</figcaption></figure><p>With the rolling mean filter being done on the data, it already showed a decrease in runtime as the number of samples increases. Also note that at sample 500 (input==501), the simplified logic is already correct at up to the last sample, which is as of this range, is input==1023. <br>The result is a manifestation of the simplifying power of Quine-Mccluskey method:<br>1. simplifies logic for computing efficiency and,<br>2. Inferential capacity of don’t cares by substituting don’t care values with values that yields simplest Boolean representation.</p><p>The simplest representation aims to follow Occam’s razor, and thus having the capacity to make inferences due to limited data.</p><p>I am yet to make research how this relates to Bayes’s inference theorem as well as Solomonoff’s theory of inductive inference.</p><p>2. <strong>14th bit results</strong>: for the 14th bit, the range is extended to much higher combinations. You will know the reasons why as I present the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/573/1*9-63D7AbVsOI9Kp1zFTE-A.png" /><figcaption>Figure 4. 14th bit run times. Note that at every increase in input bits, the runtime increases.</figcaption></figure><p>Every time the range of inputs reaches maximum value, the number of bits is increased by 1. For example, for the 14th bit, the plots doesn’t show high levels of predictability even the inputs reached by 1023. So need to increase the number of inputs to 11. What is done during this bit increase is as follows:<br>a. Added list of input names<br>b. For minterms, I inserted a value of 0 on the most significant bit<br>c. For dontcares, I inserted a value of 0 on the most significant bit and for the new digits (&gt; 1023), I continued the list to 2n-1.</p><p>For the runtimes, notice the observable decrease in runtime as samples learned increase. But it is also expected that since the algorithm used is Quine McCluskey, the algorithm is NP-hard, resulting to significant increase in runtime as the number of bits are added to accommodate higher samples. <br>It seems that the range of data for training is not enough to reach a ‘learned’ state, similar to what happened in bit 11, where the data is correct from 501 onwards.</p><p>Due to runtime issues, I will discontinue examining the learning progression of bit 14. Instead, I will look for the ‘learned state’ of this bit, that is the correct values (50% correct up to the last range) must take a large number of correct answers continuously up to the last number in that range.</p><p>Procedure:<br>1. Increase the input range. (e.g, from 11 bits to 12 bits)<br>2. Decrease the number of input training data exponentially by 2s, starting from n. Example is to set training data from 1 to (211–2n). If the verification is correct up to 211, n is increased by 1, and the process is done again.<br>Here are the results of this heuristic in searching for the ‘learned state’ of bit 14:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*mmJbD9ZChuiOiICBEFcYLA.png" /></figure><p>This shows that the results are replicated as well on bit 14, that it is able to predict 50% continuous missing values up to the last range.<br>On the nature of learning how to square integers:</p><p><strong>V. Interpretation</strong>:<br>To square integers, one must learn the following concepts:<br><em>1. Concept of counting</em>. Counting is a compression technique to allow the representation of quantities be compressed exponentially by applying the concept of permutation of symbols. The next significant bit is instantiated if the less significant bit exhausts its symbol usage. <br><em>2. Concept of addition</em>. Addition is repeated counting. It involves iterating the sequence of permuted set of symbols where the addend is the length of iteration.<br><em>3. Concept of multiplication</em>. It is repeated addition with the same iteration to be done as it is in addition.</p><p>Given these complex challenges, the Quine-McCluskey algorithm applied to ISF is able to achieve a<strong> ‘learned state’</strong>, which is able to accurately predict the remaining 50% of the values of a given input range.</p><p>Apart from these, this interpretation is realized:<br>The training data is just selected <em>1 bit</em>, which it has to deduce the nature of squaring integers using only training data from 1 bit of the outputs. Yet it has achieved a learned state after given a sufficient number of training data. In the 14th bit, it only needs at least 4095 out of 8192 samples. Note that the relationships of the digits aren’t yet identified and the relationships of the other output bits has no evidence.</p><p><strong>VI. Summary and conclusion:</strong><br>It therefore shows that the application of Occam’s Razor to Boolean logic simplification through Quine McCluskey algorithm can be an foundational algorithm for modern artificial intelligence when applied through computers. This is because of the Boolean nature of the logic circuits that made up modern computers.</p><p>The runtime effect of this algorithm is still not yet considered. It is still not yet studied if the rate of learning as a function of computer resources vs other methods like neural networks and tree searches as foundation will have higher overall performance in terms of computer usage and predictive power.</p><p><strong>VII. Reference source code</strong></p><p>You can clone or try out the code via <a href="https://github.com/arjay55/boolLLM">https://github.com/arjay55/boolLLM</a></p><p>This is a simple testcase for you to try out. It is a full adder and see its predictive power per bit.</p><p><strong>VIII. Future work</strong></p><p>It may seem obvious that there will be <em>combinatorial explosion</em> when the number of<em> inputs/outputs</em> increase. I am heavily experimenting now on algorithms that simplify the simplification process and it is proving to be exceptionally difficult, but there are promising hypothesis that are yet to be proven. This next step is crucial towards applying Boolean logic, the most fundamental form of symbolic reasoning, into a full blown, scalable machine learning algorithm applicable in real world scenarios. I will publish my progress here at Medium.</p><p>Recently, I found another good literary work on this on <a href="https://opencog.org/">OpenCog’s </a>AI platform. Try checking this and am excited to see their work and contribute to their progress.</p><p>Thanks for reading :)</p><p><strong>References</strong>:</p><p>[1] Google search dictionary, <a href="https://www.google.com/search?q=knowledge">“Knowledge” definition</a></p><p>[2] Marco Muselli, Switching Neural Networks: <a href="http://www.rulex.ai/wp-content/uploads/2017/04/Switching-Neural-Networks-A-New-Connectionist-Model-for-Classification.pdf">A New Connectionist Model for Classification</a>, Rulex Analytics Technical Papers</p><p>[3] Marco Muselli, Enrico Ferrari, <a href="http://www.rulex.ai/wp-content/uploads/2017/04/Coupling-Logical-Analysis-of-Data-and-Shadow-Clustering-for-Partially-Defined-Positive-Boolean-Function-Reconstruction.pdf">Coupling Logical Analysis of Data and Shadow Clustering for Partially Defined Positive Boolean Function Reconstruction</a> (2011), Rulex Analytics Technical Papers</p><p>[4] <a href="https://en.wikipedia.org/wiki/Occam%27s_razor">Occam’s Razor</a>, Wikipedia</p><p>[5] <a href="https://en.wikipedia.org/wiki/Espresso_heuristic_logic_minimizer#Espresso_algorithm">Espresso Algorithm</a>, Wikipedia</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d5a88fec847" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-based-on-boolean-logic/machine-learning-through-logic-simplification-on-predicting-isf-2d5a88fec847">Machine learning and reasoning through Boolean logic</a> was originally published in <a href="https://medium.com/machine-learning-based-on-boolean-logic">Machine learning based on boolean logic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>