<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Somayyeh Gholami on Medium]]></title>
        <description><![CDATA[Stories by Somayyeh Gholami on Medium]]></description>
        <link>https://medium.com/@somayyeh_43305?source=rss-cc9da3ccea38------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*9mRicyTDbSfP3RQ2G0a9Ww.jpeg</url>
            <title>Stories by Somayyeh Gholami on Medium</title>
            <link>https://medium.com/@somayyeh_43305?source=rss-cc9da3ccea38------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 19:25:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@somayyeh_43305/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Bayesian Approach: An Efficient Way For Hyperparameters Tuning (Optimizing)]]></title>
            <link>https://medium.com/@somayyeh_43305/the-bayesian-approach-an-efficient-way-for-hyperparameters-tuning-optimizing-cf00a157a3b2?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf00a157a3b2</guid>
            <category><![CDATA[hyperparameter-tuning]]></category>
            <category><![CDATA[imbalanced-dataset]]></category>
            <category><![CDATA[gaussiannb]]></category>
            <category><![CDATA[bayesian-optimization]]></category>
            <category><![CDATA[lightgbm]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Thu, 24 Aug 2023 12:34:12 GMT</pubDate>
            <atom:updated>2023-08-25T16:23:19.454Z</atom:updated>
            <content:encoded><![CDATA[<h3>:.:.:.:.:.: Bayesian Optimization :.:.:.:.:.:</h3><h3><strong>The Bayesian Approach: An Efficient Way For Hyperparameters Tuning</strong></h3><p>:.:.:.:.:.:.:.:.:.:.:.:.:.:.:.:.: <strong>By: Somayyeh Gholami — May 2023</strong> :.:.:.:.:.:.:.:.:.:.:.:.:</p><p><a href="https://www.kaggle.com/code/somayyehgholami/the-bayesian-approach-for-efficient-optimization">https://www.kaggle.com/code/somayyehgholami/the-bayesian-approach-for-efficient-optimization</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*TRWEauRsELhtzE-W1EWQwA.png" /></figure><h4>Hyperparameters Tuning (Optimizing)</h4><p>Optimizing the hyperparameters is an essential step to achieve high accuracy in machine learning. Hyperparameters are the parameters that control how a model learns from the data, such as the learning rate, the regularization strength, the number of hidden layers, etc. Unlike the model parameters that are learned from the data, hyperparameters are usually fixed before the training starts and need to be tuned manually or automatically.</p><p>There are different solutions for hyperparameter tuning, depending on the size and complexity of the hyperparameter space. On the other hand, the search methods can be <strong>uninformed and informed</strong>. Uninformed search tries sets of hyperparameters repeatedly and independently. <strong>Each search does not inform or suggest the other searches.</strong> Examples of uninformed searches are <strong>GridSearchCV</strong> and <strong>RandomizedSearchCV</strong>.</p><p>In the grid search, evaluates the model performance for every possible combination of hyperparameters in a predefined grid. This method is simple and exhaustive, but it can be very costly and inefficient when the number of hyperparameters is large Another solution is random search, which randomly samples a fixed number of hyperparameter settings from a distribution and evaluates them. This method can be faster and more effective than grid search, especially when some hyperparameters have more impact than others or when the optimal values are not evenly distributed.</p><p>Other solutions for hyperparameter tuning include <strong>Bayesian optimization</strong>, which uses a probabilistic model to guide the search based on previous evaluations. These methods can be more efficient and robust than grid search or random search.</p><h4>How does “Bayesian optimization” work?</h4><ul><li>Reference: <a href="https://github.com/bayesian-optimization/BayesianOptimization/blob/master/README.md">https://github.com/bayesian-optimization/BayesianOptimization/blob/master/README.md</a></li></ul><p><strong>Bayesian optimization</strong> works by constructing a posterior distribution of functions (Gaussian process) that best describes the function you want to optimize. As the number of observations grows, the posterior distribution improves, and the algorithm becomes more certain of which regions in parameter space are worth exploring and which are not, as seen in the picture below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*1_PdcS7tBUi2LTDkzeLRuQ.png" /></figure><p>As you iterate over and over, the algorithm balances its needs of exploration and exploitation, considering what it knows about the target function. At each step, a Gaussian Process is fitted to the known samples (points previously explored), and the posterior distribution, combined with an exploration strategy (such as UCB (Upper Confidence Bound), or EI (Expected Improvement)), are used to determine the next point that should be explored (see the gif below).</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*1HhgVrhk7ABeEaLsTLbWHA.gif" /></figure><h4>How do we implement these solutions in Python?</h4><p>There are different ways to implement hyperparameter tuning in Python, depending on the library and the method you choose. One popular library for machine learning in Python is <strong>Scikit-learn</strong>, which provides several tools and classes for hyperparameter tuning. Some of the most common ones are:</p><ul><li><strong>GridSearchCV:</strong> This class performs an exhaustive search over a grid of hyperparameters, evaluating each combination using cross-validation.</li><li><strong>RandomizedSearchCV:</strong> This class performs a random search over a distribution of hyperparameters, sampling a fixed number of candidates and evaluating them using cross-validation.</li><li><strong>HalvingGridSearchCV and HalvingRandomSearchCV:</strong> These classes are similar to GridSearchCV and RandomizedSearchCV, but they use a successive halving technique to reduce the number of candidates at each iteration, based on their performance. This can speed up the search process and reduce the computational cost.</li></ul><p>Another popular library for machine learning in Python is <strong>Neptune.ai</strong>, which provides a platform for experiment tracking and hyperparameter tuning. You can use it by installing the neptune-client package and creating a project on the Neptune website.</p><p>The <strong>“Optuna”</strong> library, which has many fans among the participants of Kaggle competitions, introduces itself on <a href="https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html">its website</a> as follows: Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. In addition, in this link, <strong>seven sampling algorithms</strong> are introduced.</p><p><strong>bayes_opt:</strong> This library implements the <strong>Bayesian Optimization algorithm</strong> using Gaussian Processes. You can install the bayes_opt package and create a BayesianOptimization object with your objective function and parameter bounds. Then, you can call the maximize method to find the best parameters and score.</p><p><strong>hyperopt:</strong> This library implements various methods for hyperparameter optimization, such as Tree-structured Parzen Estimator (TPE), Adaptive TPE, and Random Search.</p><h4>Sections of this notebook</h4><p>In this <a href="https://www.kaggle.com/code/somayyehgholami/the-bayesian-approach-for-efficient-optimization">notebook</a>, I have used the dataset of one of the <strong>Kaggle competitions</strong>. In this competition, we are asked to predict whether a customer will make a transaction or not, regardless of the amount of money transacted. Hence, our goal is to solve a <strong>binary classification problem</strong>. In the data description, you can see that the features given are numeric and anonymized.</p><h4>1 — Exploratory Data Analysis (EDA)</h4><p>In this section, it was found that the data of this challenge is considered <strong>imbalanced</strong>. That is, ninety percent of the samples belong to class-zero and only ten percent belong to class-one. In addition, two hundred numerical features are available for each sample. Randomly drawn for a number of features, histogram graphs as well as “Correlation matrix”.</p><h4>2 — Gaussian Naive Bayes (GaussianNB)</h4><p>In this section I used Gaussian Naive Bayes (GNB). Gaussian Naive Bayes (GNB) is a classification technique used in Machine Learning (ML) based on <strong>the probabilistic approach and Gaussian distribution</strong>. Gaussian Naive Bayes assumes that each parameter (also called features or predictors) has an independent capacity of predicting the output variable. I combined the quantile transformer and Gaussian naive Bayes classifer, sklearn.naive_bayes.GaussianNB, into a pipeline using sklearn.pipeline.make_pipeline. The results of this section were acceptable and <strong>had a good speed</strong> (oof Score: 0.888).</p><h4>3 — Scatter graph for TrainSet&amp;TestSet (2D)</h4><p>According to the above results for TrainSet and TestSet in two dimensions, I drew a scatter graph. I used <strong>Principal Component Analysis (PCA)</strong> to make the data two-dimensional.</p><h4>4 — LightGBM (Light Gradient-Boosting Machine)</h4><p>LightGBM is a fast, distributed, high-performance gradient boosting framework based on <strong>decision tree algorithms</strong>, used for ranking, classification and many other machine learning tasks. LGBM is the second algorithm I have used for this challenge. Besides, I have optimized the hyperparameters for this framework. Of course, LGBM has many hyperparameters, but I only chose four of them to tune (optimize).</p><h4>5 — Hyperparameters Tuning (Optimizing)</h4><p>In this section, I used two different libraries for optimization. First, I used the <strong>bayes_opt</strong> library. As stated earlier, This is a library that implements the Bayesian Optimization algorithm using Gaussian Processes. The second library is <strong>GridSearchCV</strong>. I also optimized the same four hyperparameters with this library.</p><h4>6 — Cross Validation &amp; Resampling</h4><p>After finalizing the hyperparameter values, I used a seven-fold <strong>cross-validation</strong>. Of course, the data of this challenge is <strong>imbalanced</strong> (ninety percent versus ten percent), that’s why I did <strong>resampling</strong> along with cross validation. so that in each fold, the number of two classes is almost balanced.</p><h4>7 — Ensembling</h4><p>In the last section, I Ensembled the results of <strong>GaussianNB</strong> and <strong>LightGBM</strong> together and got a better result. That is, the <strong>Ensembling</strong> score was better than the score of both methods.</p><h4>Competition Kaggle (Real-world dataset)</h4><p><a href="https://www.kaggle.com/competitions/santander-customer-transaction-prediction"><strong>Santander Customer Transaction Prediction</strong></a><strong>: </strong>Can you identify who will make a transaction?</p><p><a href="https://www.kaggle.com/competitions/santander-customer-transaction-prediction">Santander Customer Transaction Prediction</a></p><h4>Exploratory Data Analysis (EDA)</h4><pre>import warnings # suppress warnings<br>warnings.filterwarnings(&#39;ignore&#39;)<br>#########################################<br>import os<br>import gc<br>import glob<br>import random<br>import numpy as np <br>import pandas as pd<br>import seaborn as sns<br>from tqdm import tqdm<br>from scipy import stats<br>from pathlib import Path<br>#########################################<br>import matplotlib.pyplot as plt<br>#import plotly.figure_factory as ff<br>#import plotly.express as px<br>%matplotlib inline<br>#!ls ../input/*<br>#########################################<br>pd.set_option(&#39;display.max_columns&#39;, 500)</pre><pre>df_train  = pd.read_csv(&#39;./train.csv&#39;, index_col=&#39;ID_code&#39;)<br>df_test   = pd.read_csv(&#39;./test.csv&#39;, index_col=&#39;ID_code&#39;)<br>df_sample = pd.read_csv(&#39;./sample_submission.csv&#39;)<br><br>display(df_train.shape , df_test.shape, df_sample.shape)<br># display(df_train, df_test, df_sample)</pre><pre>(200000, 201)<br><br><br><br>(200000, 200)<br><br><br><br>(200000, 2)</pre><pre>display(df_train.info())<br>df_train.describe().transpose()</pre><pre>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;<br>Index: 200000 entries, train_0 to train_199999<br>Columns: 201 entries, target to var_199<br>dtypes: float64(200), int64(1)<br>memory usage: 308.2+ MB<br><br><br><br>None</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*iyryisrWO2TRUXmGWcyIzQ.png" /></figure><h4>Check Null Values</h4><pre>MV1 = df_train.isnull().sum()<br>MV2 = df_test.isnull().sum()<br><br>print(&#39;=&#39; * 30)<br>print(f&#39;Missing Value df_train:\n{MV1[MV1 &gt; 0]}&#39;)<br>print(&#39;=&#39; * 30)<br>print(f&#39;Missing Value df_test:\n{MV2[MV2 &gt; 0]}&#39;)<br>print(&#39;=&#39; * 30)</pre><pre>==============================<br>Missing Value df_train:<br>Series([], dtype: int64)<br>==============================<br>Missing Value df_test:<br>Series([], dtype: int64)<br>==============================</pre><h4>Let’s see how unbalanced the dataset is:</h4><pre>train = df_train.copy()<br>test = df_test.copy()<br><br>target = train.pop(&#39;target&#39;)<br>target.value_counts().plot(kind=&#39;barh&#39;, figsize=(12,1), title=&#39;Target Count&#39;, color=[&#39;darkcyan&#39;,&#39;red&#39;])<br>plt.gca().set_facecolor(&#39;lightgray&#39;)<br><br>print(&#39;=&#39; * 30)<br>print(target.value_counts())<br>print(&#39;=&#39; * 30)</pre><pre>==============================<br>0    179902<br>1     20098<br>Name: target, dtype: int64<br>==============================</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*nxuPlUmq6ILpPobXNMK0aw.png" /></figure><pre>target.value_counts().plot(kind=&#39;pie&#39;, figsize=(4, 4), colors=[&#39;darkcyan&#39;,&#39;red&#39;], autopct=&#39;%1.0f%%&#39;)<br><br>print(&#39;=&#39; * 30)<br>print(target.value_counts(normalize=True))<br>print(&#39;=&#39; * 30)</pre><pre>==============================<br>0    0.89951<br>1    0.10049<br>Name: target, dtype: float64<br>==============================</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*n6rGMgr89C7aIWRuJ7kd_w.png" /></figure><h4>Features</h4><pre>features = train.columns.tolist() <br>len(features)</pre><pre>200</pre><p>Random selection of 28 features out of 200 features, just to simplify the visual.</p><pre>features_select = np.random.choice(features, size=28, replace=False).tolist()<br><br>len(features_select)</pre><pre>28</pre><h4>Correlation Matrix</h4><pre>cor_matrix = df_train[features + [&#39;target&#39;]].corr().round(2)<br><br>fig = plt.figure(figsize=(12,12));<br>cmap = sns.color_palette(&#39;Blues&#39;)<br>sns.heatmap(cor_matrix, center=0, annot=False, cmap=cmap);<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*OSUgpn9EKdUAKZGf05u2Mw.png" /></figure><h4>Correlation matrix for randomly selected features (28 Columns)</h4><pre>cor_matrix = df_train[features_select + [&#39;target&#39;]].corr().round(2)<br><br>fig = plt.figure(figsize=(12,12));<br>cmap = sns.diverging_palette(140, 10, s=75, l=50, sep=1, n=6, center=&#39;light&#39;, as_cmap=False);<br>sns.heatmap(cor_matrix, center=0, annot=False, cmap=cmap);<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*0wbwQ7Hwpk8X9UDRPOV8LQ.png" /></figure><h4>Histograms of the Randomly selected features (28 Columns)</h4><pre>sns.set()<br>plt.style.use(&#39;seaborn-whitegrid&#39;) <br>_, axs = plt.subplots(7, 4, figsize=(15,30), facecolor=&#39;lightyellow&#39;)<br><br>for f, ax in zip(features_select, axs.ravel()):<br>    ax.set_facecolor(&#39;lightcyan&#39;)<br>    ax.hist(train[f], bins=100, color=&#39;red&#39;)<br>    ax.set_title(f&#39;Feature: {f}&#39;, fontsize=10)<br><br>plt.suptitle(&#39;Histograms of the Randomly selected features (28 Columns)&#39;, y=0.91, fontsize=14)<br>plt.show()</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*0R8JBX_ISkH9Hz9ggTI7_Q.png" /></figure><p><strong>Evaluation Metric (AUC)</strong></p><p>In this Kaggle challenge; submissions are evaluated on area under the ROC curve between the predicted probability and the observed target.</p><pre>from sklearn.metrics import roc_auc_score, roc_curve, auc<br><br>def roc_auc(true_list, pred_list, a, b):<br>    <br>    fpr, tpr, _ = roc_curve(true_list, pred_list)    <br>    roc_auc = auc(fpr, tpr)<br><br>    # print(f&#39;FPR: {fpr}&#39;)<br>    # print(f&#39;TPR: {tpr}&#39;)<br>    # print(f&#39;{list(zip(fpr,tpr))}&#39;) <br>    print(f&#39;\n&gt;&gt;&gt;&gt;&gt; ROC_AUC: %0.6f &lt;&lt;&lt;&lt;&lt;\n&#39; %roc_auc)<br>    <br>    sns.set()<br>    plt.style.use(&#39;seaborn-whitegrid&#39;)<br>    plt.figure(figsize=(a, b), facecolor=&#39;lightyellow&#39;)<br>    plt.gca().set_facecolor(&#39;lightblue&#39;)<br>    plt.plot(fpr, tpr, color=&#39;darkorange&#39;, lw=2, label=&#39;ROC curve&#39;)<br>    plt.plot([0, 1], [0, 1], color=&#39;navy&#39;, lw=2, linestyle=&#39;--&#39;)<br>    plt.xlim([-0.01, 1.0])<br>    plt.ylim([0.0, 1.05])<br>    plt.xlabel(&#39;False Positive Rate&#39;)<br>    plt.ylabel(&#39;True Positive Rate&#39;)<br>    plt.title(&#39;The area under the ROC curve\n&#39;)<br>    plt.legend(loc=&quot;lower right&quot;)<br>    plt.show()</pre><h4>:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::</h4><h4>Gaussian Naive Bayes (GaussianNB)</h4><h4>:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::</h4><pre>from sklearn.naive_bayes import GaussianNB<br><br>from sklearn.pipeline import make_pipeline<br>from sklearn.model_selection import cross_val_score<br>from sklearn.preprocessing import QuantileTransformer</pre><pre>transformed = pd.DataFrame(QuantileTransformer(output_distribution=&#39;normal&#39;).fit_transform(train))<br><br>pipeline = make_pipeline(QuantileTransformer(output_distribution=&#39;normal&#39;), GaussianNB())<br>pipeline.fit(train, target)</pre><pre>Pipeline(steps=[(&#39;quantiletransformer&#39;,<br>                 QuantileTransformer(output_distribution=&#39;normal&#39;)),<br>                (&#39;gaussiannb&#39;, GaussianNB())])</pre><pre>cross_val_score(pipeline, train, target, scoring=&#39;roc_auc&#39;, cv=10).mean()</pre><pre>0.8894505460950208</pre><pre>roc_auc(target, pipeline.predict_proba(train)[:,1], 6, 6)</pre><pre>&gt;&gt;&gt;&gt;&gt; ROC_AUC: 0.891120 &lt;&lt;&lt;&lt;&lt;</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*HvDzSXuwj98AF5xmv9Eslw.png" /></figure><h4>Prediction Histogram</h4><pre>preds_bayes = pipeline.predict_proba(test)[:,1]<br><br>sns.set()<br>plt.hist(preds_bayes, bins=50)<br>plt.gca().set_facecolor(&#39;lightblue&#39;)<br>min(preds_bayes), max(preds_bayes)</pre><pre>(2.7901255073929795e-05, 0.9988100474618177)</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*ZzMEv4GQLsCDsFKfb65Apw.png" /></figure><h4>Kaggle Submission (BAYES)</h4><pre>sub_bayes = df_sample.copy()<br>sub_bayes[&#39;target&#39;] = preds_bayes<br># sub_bayes.to_csv(&#39;submission_bayes.csv&#39;,index=False)<br><br># !ls</pre><p><strong>Principal Component Analysis (PCA)</strong></p><p>Principal component analysis, or PCA, is a statistical procedure that allows you to summarize the information content in large data tables by means of a smaller set of “summary indices” that can be more easily visualized and analyzed.</p><p>Since our dataset has many features, we reduce these features by using PCA to visualize the distribution of samples in a two-dimensional graph.</p><pre>from sklearn.decomposition import PCA<br><br>pca = PCA(n_components=2)<br>train_2d = pca.fit_transform(train)<br>test_2d = pca.fit_transform(test)<br><br>train_2d.shape, test_2d.shape</pre><pre>((200000, 2), (200000, 2))</pre><h4>Plotting Function</h4><pre>def plot_task(X, y, headline):   <br>    colors  = [&#39;darkcyan&#39;,&#39;red&#39;]<br>    classes = [&#39;Class: 0&#39;,&#39;Class: 1&#39;]<br>    <br>    plt.style.use(&#39;seaborn-whitegrid&#39;) <br>    plt.figure(figsize=(7, 7), facecolor=&#39;lightyellow&#39;)<br>    for u, c in zip(np.unique(y), colors):<br>        plt.scatter(X[y==u, 0], X[y==u, 1], c=c, s=0.3)<br>        <br>    plt.gca().set_facecolor(&#39;lightgreen&#39;)<br>    plt.title(headline, fontsize=14)<br>    plt.legend(classes, loc=2)<br>    plt.show()</pre><h4>Scatter graph for TrainSet (2D)</h4><pre>print(&#39;The number of each class:&#39;)<br>print(target.value_counts(), &#39;\n&#39;)<br>plot_task(train_2d, target, &#39;Scatter graph for TrainSet_2D\n&#39;)</pre><pre>The number of each class:<br>0    179902<br>1     20098<br>Name: target, dtype: int64</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*ISVDA2wXhkPI-nCgLdlcqA.png" /></figure><h4>Scatter graph for TestSet (2D)</h4><pre>yy = pipeline.predict(test)<br>df_yy = pd.DataFrame(yy)<br><br>print(&#39;The number of each class:&#39;)<br>print(df_yy.value_counts(), &#39;\n&#39;)<br>plot_task(test_2d, yy, &#39;Scatter graph for TestSet_2D\n&#39;)</pre><pre>The number of each class:<br>0    192413<br>1      7587<br>dtype: int64</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*rsPJzjm7NPlcRN4qwq-yQA.png" /></figure><h4>:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::</h4><h4>LightGBM (Light Gradient-Boosting Machine)</h4><h4>:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::</h4><p>LightGBM is a fast, distributed, high-performance gradient boosting framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks. LightGBM grows tree vertically while other tree based learning algorithms grow trees horizontally. It means that LightGBM grows tree leaf-wise while other algorithms grow level-wise.</p><h4>Parameters</h4><p>classlightgbm.LGBMClassifier(boosting_type=’gbdt’, num_leaves=31, max_depth=-1, learning_rate=0.1, n_estimators=100, subsample_for_bin=200000, objective=None, class_weight=None, min_split_gain=0.0, min_child_weight=0.001, min_child_samples=20, subsample=1.0, subsample_freq=0, colsample_bytree=1.0, reg_alpha=0.0, reg_lambda=0.0, random_state=None, n_jobs=None, importance_type=’split’, **kwargs)</p><pre>! pip install lightgbm</pre><pre>Requirement already satisfied: lightgbm in c:\users\infok\anaconda3\lib\site-packages (3.3.5)<br>Requirement already satisfied: wheel in c:\users\infok\anaconda3\lib\site-packages (from lightgbm) (0.37.0)<br>Requirement already satisfied: scipy in c:\users\infok\anaconda3\lib\site-packages (from lightgbm) (1.7.1)<br>Requirement already satisfied: numpy in c:\users\infok\anaconda3\lib\site-packages (from lightgbm) (1.20.3)<br>Requirement already satisfied: scikit-learn!=0.22.0 in c:\users\infok\anaconda3\lib\site-packages (from lightgbm) (0.24.2)<br>Requirement already satisfied: joblib&gt;=0.11 in c:\users\infok\anaconda3\lib\site-packages (from scikit-learn!=0.22.0-&gt;lightgbm) (1.1.0)<br>Requirement already satisfied: threadpoolctl&gt;=2.0.0 in c:\users\infok\anaconda3\lib\site-packages (from scikit-learn!=0.22.0-&gt;lightgbm) (2.2.0)</pre><pre>! pip install bayesian-optimization</pre><pre>Requirement already satisfied: bayesian-optimization in c:\users\infok\anaconda3\lib\site-packages (1.4.3)<br>Requirement already satisfied: scikit-learn&gt;=0.18.0 in c:\users\infok\anaconda3\lib\site-packages (from bayesian-optimization) (0.24.2)<br>Requirement already satisfied: colorama&gt;=0.4.6 in c:\users\infok\anaconda3\lib\site-packages (from bayesian-optimization) (0.4.6)<br>Requirement already satisfied: numpy&gt;=1.9.0 in c:\users\infok\anaconda3\lib\site-packages (from bayesian-optimization) (1.20.3)<br>Requirement already satisfied: scipy&gt;=1.0.0 in c:\users\infok\anaconda3\lib\site-packages (from bayesian-optimization) (1.7.1)<br>Requirement already satisfied: threadpoolctl&gt;=2.0.0 in c:\users\infok\anaconda3\lib\site-packages (from scikit-learn&gt;=0.18.0-&gt;bayesian-optimization) (2.2.0)<br>Requirement already satisfied: joblib&gt;=0.11 in c:\users\infok\anaconda3\lib\site-packages (from scikit-learn&gt;=0.18.0-&gt;bayesian-optimization) (1.1.0)</pre><pre>from lightgbm import LGBMClassifier<br>from bayes_opt import BayesianOptimization<br><br>from sklearn.model_selection import GridSearchCV<br>from sklearn.model_selection import RandomizedSearchCV<br><br>from sklearn.model_selection import cross_val_score<br>from sklearn.model_selection import StratifiedKFold<br>from sklearn.model_selection import train_test_split</pre><h3>Optimizing the Hyperparameters</h3><h4>Bayesian Optimization: bayes_opt</h4><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*mzKYniO0Flizr-7n3HVkSA.png" /></figure><p>Reference graph: <a href="https://github.com/bayesian-optimization/BayesianOptimization/blob/master/README.md">https://github.com/bayesian-optimization/BayesianOptimization/blob/master/README.md</a></p><pre>def lgbm_cl_bo(min_child_samples, colsample_bytree, num_leaves, learning_rate):<br>    <br>    params_lgbm = {}<br>    params_lgbm[&#39;min_child_samples&#39;] = round(min_child_samples)<br>    params_lgbm[&#39;colsample_bytree&#39;] = colsample_bytree<br>    params_lgbm[&#39;num_leaves&#39;] = round(num_leaves)<br>    params_lgbm[&#39;learning_rate&#39;] = learning_rate<br>    <br>    params_lgbm[&#39;boosting_type&#39;] = &#39;gbdt&#39;   # Manual optimization<br>    params_lgbm[&#39;objective&#39;] = &#39;binary&#39;     # Manual optimization<br>    params_lgbm[&#39;subsample&#39;] = 1.0<br>    params_lgbm[&#39;max_bin&#39;] = 1023<br>    params_lgbm[&#39;n_jobs&#39;] = -1<br>    params_lgbm[&#39;reg_alpha&#39;] = 0.1<br>    params_lgbm[&#39;reg_lambda&#39;] = 0.2<br>    # params_lgbm[&#39;n_estimators&#39;] = 1200<br><br>    scores = cross_val_score(LGBMClassifier(**params_lgbm, random_state=11),<br>                             train, target, scoring=&#39;roc_auc&#39;, cv=5).mean()<br>    score = scores.mean()<br>    return score</pre><p><strong>n_iter: How many steps of bayesian optimization you want to perform. The more steps the more likely to find a good maximum you are.</strong></p><p><strong>init_points: How many steps of random exploration you want to perform. Random exploration can help by diversifying the exploration space.</strong></p><pre>params_lgbm ={&#39;min_child_samples&#39;:(800, 1200),<br>              &#39;colsample_bytree&#39;:(0.3, 1.0),<br>              &#39;num_leaves&#39;:(20, 60),<br>              &#39;learning_rate&#39;:(0.005, 0.06)}<br><br>lgbm_bo = BayesianOptimization(lgbm_cl_bo, params_lgbm, random_state=11)<br>lgbm_bo.maximize(n_iter=10, init_points=20)</pre><pre>|   iter    |  target   | colsam... | learni... | min_ch... | num_le... |<br>-------------------------------------------------------------------------<br>| [0m1        [0m | [0m0.7944   [0m | [0m0.4262   [0m | [0m0.006071 [0m | [0m985.3    [0m | [0m49.0     [0m |<br>| [95m2        [0m | [95m0.836    [0m | [95m0.5941   [0m | [95m0.0317   [0m | [95m805.1    [0m | [95m39.49    [0m |<br>| [95m3        [0m | [95m0.8451   [0m | [95m0.9593   [0m | [95m0.05179  [0m | [95m1.092e+03[0m | [95m24.35    [0m |<br>| [95m4        [0m | [95m0.8543   [0m | [95m0.9257   [0m | [95m0.05214  [0m | [95m866.0    [0m | [95m45.29    [0m |<br>| [0m5        [0m | [0m0.8019   [0m | [0m0.3143   [0m | [0m0.01142  [0m | [0m926.5    [0m | [0m26.32    [0m |<br>| [0m6        [0m | [0m0.8486   [0m | [0m0.8313   [0m | [0m0.05001  [0m | [0m937.8    [0m | [0m32.75    [0m |<br>| [0m7        [0m | [0m0.8043   [0m | [0m0.3782   [0m | [0m0.009617 [0m | [0m1.085e+03[0m | [0m43.98    [0m |<br>| [0m8        [0m | [0m0.8431   [0m | [0m0.339    [0m | [0m0.03139  [0m | [0m960.7    [0m | [0m53.92    [0m |<br>| [0m9        [0m | [0m0.8479   [0m | [0m0.8025   [0m | [0m0.03811  [0m | [0m1.021e+03[0m | [0m57.96    [0m |<br>| [0m10       [0m | [0m0.8265   [0m | [0m0.9907   [0m | [0m0.02359  [0m | [0m895.9    [0m | [0m51.86    [0m |<br>| [0m11       [0m | [0m0.8265   [0m | [0m0.3446   [0m | [0m0.02505  [0m | [0m828.0    [0m | [0m32.77    [0m |<br>| [0m12       [0m | [0m0.831    [0m | [0m0.3493   [0m | [0m0.02096  [0m | [0m1.116e+03[0m | [0m56.22    [0m |<br>| [0m13       [0m | [0m0.8379   [0m | [0m0.8548   [0m | [0m0.0359   [0m | [0m1.046e+03[0m | [0m34.46    [0m |<br>| [0m14       [0m | [0m0.8246   [0m | [0m0.4182   [0m | [0m0.02899  [0m | [0m1.093e+03[0m | [0m22.52    [0m |<br>| [95m15       [0m | [95m0.8547   [0m | [95m0.3145   [0m | [95m0.04738  [0m | [95m920.0    [0m | [95m48.05    [0m |<br>| [0m16       [0m | [0m0.853    [0m | [0m0.8143   [0m | [0m0.05631  [0m | [0m960.1    [0m | [0m34.34    [0m |<br>| [0m17       [0m | [0m0.8529   [0m | [0m0.8646   [0m | [0m0.04705  [0m | [0m1.061e+03[0m | [0m52.44    [0m |<br>| [95m18       [0m | [95m0.8588   [0m | [95m0.7496   [0m | [95m0.05766  [0m | [95m933.5    [0m | [95m49.53    [0m |<br>| [0m19       [0m | [0m0.8203   [0m | [0m0.9649   [0m | [0m0.02339  [0m | [0m1.045e+03[0m | [0m34.63    [0m |<br>| [0m20       [0m | [0m0.7899   [0m | [0m0.6231   [0m | [0m0.009125 [0m | [0m807.7    [0m | [0m50.39    [0m |<br>| [0m21       [0m | [0m0.7942   [0m | [0m0.6277   [0m | [0m0.01126  [0m | [0m900.0    [0m | [0m43.42    [0m |<br>| [0m22       [0m | [0m0.8061   [0m | [0m0.6337   [0m | [0m0.02099  [0m | [0m975.1    [0m | [0m20.05    [0m |<br>| [0m23       [0m | [0m0.8156   [0m | [0m0.3096   [0m | [0m0.01271  [0m | [0m825.0    [0m | [0m41.32    [0m |<br>| [0m24       [0m | [0m0.8408   [0m | [0m0.7621   [0m | [0m0.03168  [0m | [0m1.002e+03[0m | [0m54.17    [0m |<br>| [0m25       [0m | [0m0.8347   [0m | [0m0.4121   [0m | [0m0.03833  [0m | [0m890.2    [0m | [0m21.7     [0m |<br>| [0m26       [0m | [0m0.8554   [0m | [0m0.8346   [0m | [0m0.04872  [0m | [0m1.021e+03[0m | [0m58.17    [0m |<br>| [0m27       [0m | [0m0.8463   [0m | [0m0.508    [0m | [0m0.03489  [0m | [0m1.021e+03[0m | [0m59.29    [0m |<br>| [0m28       [0m | [0m0.8523   [0m | [0m0.6925   [0m | [0m0.04364  [0m | [0m1.022e+03[0m | [0m58.05    [0m |<br>| [0m29       [0m | [0m0.8405   [0m | [0m0.337    [0m | [0m0.02911  [0m | [0m933.1    [0m | [0m50.53    [0m |<br>| [0m30       [0m | [0m0.8489   [0m | [0m0.8472   [0m | [0m0.04278  [0m | [0m933.4    [0m | [0m48.31    [0m |<br>=========================================================================</pre><pre>pmax_bayes = lgbm_bo.max[&#39;params&#39;]<br>pmax_bayes</pre><pre>{&#39;colsample_bytree&#39;: 0.7495504011187386,<br> &#39;learning_rate&#39;: 0.057659422905427526,<br> &#39;min_child_samples&#39;: 933.5497685804369,<br> &#39;num_leaves&#39;: 49.53010491394692}</pre><h4>Hyperparameters tuning Using GridSearchCV</h4><pre>param_grid = {&#39;min_child_samples&#39;: [800, 900, 950, 1000, 1050 , 1100, 1150, 1200],<br>              &#39;colsample_bytree&#39;: [0.4, 0.5, 0.7, 0.9, 1.0],<br>              &#39;num_leaves&#39;: [20, 30, 40, 50, 60],<br>              &#39;learning_rate&#39;: [0.005, 0.01, 0.015, 0.02, 0.03, 0.04, 0.05, 0.06],<br><br>              &#39;boosting_type&#39;: [&#39;gbdt&#39;],   # Manual optimization<br>              &#39;objective&#39;: [&#39;binary&#39;],     # Manual optimization<br>              &#39;subsample&#39;: [1.0],<br>              &#39;max_bin&#39;: [1023],<br>              &#39;n_jobs&#39;: [-1],<br>              &#39;reg_alpha&#39;: [0.1],<br>              &#39;reg_lambda&#39;: [0.2]}</pre><pre>grid = GridSearchCV(estimator=LGBMClassifier(), param_grid=param_grid, scoring=&#39;roc_auc&#39;, cv=5)<br><br>grid.fit(train.iloc[:200,], target.iloc[:200,])</pre><pre>GridSearchCV(cv=5, estimator=LGBMClassifier(),<br>             param_grid={&#39;boosting_type&#39;: [&#39;gbdt&#39;],<br>                         &#39;colsample_bytree&#39;: [0.4, 0.5, 0.7, 0.9, 1.0],<br>                         &#39;learning_rate&#39;: [0.005, 0.01, 0.015, 0.02, 0.03, 0.04,<br>                                           0.05, 0.06],<br>                         &#39;max_bin&#39;: [1023],<br>                         &#39;min_child_samples&#39;: [800, 900, 950, 1000, 1050, 1100,<br>                                               1150, 1200],<br>                         &#39;n_jobs&#39;: [-1], &#39;num_leaves&#39;: [20, 30, 40, 50, 60],<br>                         &#39;objective&#39;: [&#39;binary&#39;], &#39;reg_alpha&#39;: [0.1],<br>                         &#39;reg_lambda&#39;: [0.2], &#39;subsample&#39;: [1.0]},<br>             scoring=&#39;roc_auc&#39;)</pre><pre>grid.best_estimator_</pre><pre>LGBMClassifier(colsample_bytree=0.4, learning_rate=0.005, max_bin=1023,<br>               min_child_samples=800, num_leaves=20, objective=&#39;binary&#39;,<br>               reg_alpha=0.1, reg_lambda=0.2)</pre><pre>pmax_grid = grid.best_params_<br>pmax_grid</pre><pre>{&#39;boosting_type&#39;: &#39;gbdt&#39;,<br> &#39;colsample_bytree&#39;: 0.4,<br> &#39;learning_rate&#39;: 0.005,<br> &#39;max_bin&#39;: 1023,<br> &#39;min_child_samples&#39;: 800,<br> &#39;n_jobs&#39;: -1,<br> &#39;num_leaves&#39;: 20,<br> &#39;objective&#39;: &#39;binary&#39;,<br> &#39;reg_alpha&#39;: 0.1,<br> &#39;reg_lambda&#39;: 0.2,<br> &#39;subsample&#39;: 1.0}</pre><h4>Imbalanced Data — Resampling</h4><h4>Split — TrainData</h4><pre>df_class0 = df_train[df_train[&#39;target&#39;] == 0]<br>df_class1 = df_train[df_train[&#39;target&#39;] == 1]<br><br>df_class0.shape, df_class1.shape</pre><pre>((179902, 201), (20098, 201))</pre><pre>train0 = df_class0.copy()<br>train1 = df_class1.copy()<br><br>target0 = train0.pop(&#39;target&#39;)<br>target1 = train1.pop(&#39;target&#39;)<br><br>train0.shape, target0.shape, train1.shape, target1.shape</pre><pre>((179902, 200), (179902,), (20098, 200), (20098,))</pre><h4>The Final Model — LightGBM</h4><pre>model = LGBMClassifier(n_estimators= 10000, <br>                       learning_rate= 0.04,<br>                       objective= &#39;binary&#39;, <br>                       boosting_type= &#39;gbdt&#39;, <br>                       <br>                       subsample= 1.0,<br>                       num_leaves= 50,  <br>                       max_bin= 1023,<br>                       n_jobs= -1,<br>                           <br>                       reg_alpha= 0.1,<br>                       reg_lambda= 0.2,<br>                       colsample_bytree= 0.7,<br>                       min_child_samples= 930,     <br>                       random_state= 123)</pre><h4>Cross Validation &amp; Resampling</h4><pre>auc_mean = 0<br>preds = np.zeros(len(test))<br>skf = StratifiedKFold(n_splits=7, shuffle=True, random_state=11)<br><br>for fold, (train_idx, valid_idx) in enumerate(skf.split(train0, target0)):    <br>    X_train0, y_train0 = train0.iloc[train_idx], target0.iloc[train_idx]<br>    X_valid0, y_valid0 = train0.iloc[valid_idx], target0.iloc[valid_idx]  <br>    <br>    X_train1, X_valid1, y_train1, y_valid1 = train_test_split(train1, target1, test_size=0.20)<br>    <br>    X_train = pd.concat([X_train0, X_train1], axis=0)<br>    X_valid = pd.concat([X_valid0, X_valid1], axis=0)<br>    y_train = pd.concat([y_train0, y_train1], axis=0)<br>    y_valid = pd.concat([y_valid0, y_valid1], axis=0)<br>    <br>    print(f&#39;\n:::::::::::::: Fold ~ {fold+1} :::::::::::::::\n&#39;)<br>    <br>    model.fit(X_train, y_train,             <br>              eval_set=[(X_valid, y_valid)], <br>              eval_metric=&#39;auc&#39;, <br>              early_stopping_rounds=250,              <br>              verbose=200)<br>    <br>    oof = model.predict_proba(X_valid)[:, -1]<br>    print(&#39;\nAUC Score:&#39;, roc_auc_score(y_valid, oof))<br>    roc_auc(y_valid, oof, 5, 5)<br>    <br>    auc_mean += roc_auc_score(y_valid, oof) / skf.n_splits<br>    preds += model.predict_proba(test)[:, -1] / skf.n_splits<br>    <br>print(&#39;\n&#39;, &#39;=&#39;* 40)<br>print(&#39; .&#39;* 20)<br>print(&#39; AUC Score (mean):&#39;, auc_mean)<br>print(&#39; .&#39;* 20)<br>print(&#39;=&#39;* 40, &#39;\n&#39;)</pre><pre>:::::::::::::: Fold ~ 1 :::::::::::::::<br><br>[200]	valid_0&#39;s auc: 0.86659	valid_0&#39;s binary_logloss: 0.297675<br>[400]	valid_0&#39;s auc: 0.882505	valid_0&#39;s binary_logloss: 0.274516<br>[600]	valid_0&#39;s auc: 0.888117	valid_0&#39;s binary_logloss: 0.265749<br>[800]	valid_0&#39;s auc: 0.890136	valid_0&#39;s binary_logloss: 0.262983<br>[1000]	valid_0&#39;s auc: 0.891016	valid_0&#39;s binary_logloss: 0.263054<br><br>AUC Score: 0.890647459175079<br><br>&gt;&gt;&gt;&gt;&gt; ROC_AUC: 0.890647 &lt;&lt;&lt;&lt;&lt;</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*U444AlLSj5Z7FUv_BTGI7Q.png" /></figure><pre>:::::::::::::: Fold ~ 2 :::::::::::::::<br><br>[200]	valid_0&#39;s auc: 0.875174	valid_0&#39;s binary_logloss: 0.292591<br>[400]	valid_0&#39;s auc: 0.891258	valid_0&#39;s binary_logloss: 0.267903<br>[600]	valid_0&#39;s auc: 0.89655	valid_0&#39;s binary_logloss: 0.257925<br>[800]	valid_0&#39;s auc: 0.898556	valid_0&#39;s binary_logloss: 0.254362<br>[1000]	valid_0&#39;s auc: 0.899224	valid_0&#39;s binary_logloss: 0.25395<br><br>AUC Score: 0.8991630017687137<br><br>&gt;&gt;&gt;&gt;&gt; ROC_AUC: 0.899163 &lt;&lt;&lt;&lt;&lt;</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*Gr9G08fCz23Ch8aqzNqZuQ.png" /></figure><pre>:::::::::::::: Fold ~ 3 :::::::::::::::<br><br>[200]	valid_0&#39;s auc: 0.876742	valid_0&#39;s binary_logloss: 0.293321<br>[400]	valid_0&#39;s auc: 0.890518	valid_0&#39;s binary_logloss: 0.269714<br>[600]	valid_0&#39;s auc: 0.895586	valid_0&#39;s binary_logloss: 0.260212<br>[800]	valid_0&#39;s auc: 0.897327	valid_0&#39;s binary_logloss: 0.256916<br>[1000]	valid_0&#39;s auc: 0.897909	valid_0&#39;s binary_logloss: 0.25666<br><br>AUC Score: 0.8979412373928025<br><br>&gt;&gt;&gt;&gt;&gt; ROC_AUC: 0.897941 &lt;&lt;&lt;&lt;&lt;</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*sdrzzw2khJd4Mk63zTYJ2g.png" /></figure><pre>:::::::::::::: Fold ~ 4 :::::::::::::::<br><br>[200]	valid_0&#39;s auc: 0.867155	valid_0&#39;s binary_logloss: 0.29636<br>[400]	valid_0&#39;s auc: 0.882858	valid_0&#39;s binary_logloss: 0.273218<br>[600]	valid_0&#39;s auc: 0.8884	valid_0&#39;s binary_logloss: 0.264115<br>[800]	valid_0&#39;s auc: 0.890045	valid_0&#39;s binary_logloss: 0.261582<br>[1000]	valid_0&#39;s auc: 0.890398	valid_0&#39;s binary_logloss: 0.261802<br><br>AUC Score: 0.8905488123584413<br><br>&gt;&gt;&gt;&gt;&gt; ROC_AUC: 0.890549 &lt;&lt;&lt;&lt;&lt;</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*bxlsjiSEbXxXQGAagRkcTA.png" /></figure><pre>:::::::::::::: Fold ~ 5 :::::::::::::::<br><br>[200]	valid_0&#39;s auc: 0.868794	valid_0&#39;s binary_logloss: 0.29601<br>[400]	valid_0&#39;s auc: 0.883868	valid_0&#39;s binary_logloss: 0.273191<br>[600]	valid_0&#39;s auc: 0.888986	valid_0&#39;s binary_logloss: 0.264739<br>[800]	valid_0&#39;s auc: 0.890877	valid_0&#39;s binary_logloss: 0.262014<br>[1000]	valid_0&#39;s auc: 0.891137	valid_0&#39;s binary_logloss: 0.262395<br><br>AUC Score: 0.8909670422208026<br><br>&gt;&gt;&gt;&gt;&gt; ROC_AUC: 0.890967 &lt;&lt;&lt;&lt;&lt;</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*tcGkSgnspmmJW4Sugk7rlA.png" /></figure><pre>:::::::::::::: Fold ~ 6 :::::::::::::::<br><br>[200]	valid_0&#39;s auc: 0.867121	valid_0&#39;s binary_logloss: 0.297118<br>[400]	valid_0&#39;s auc: 0.883684	valid_0&#39;s binary_logloss: 0.273896<br>[600]	valid_0&#39;s auc: 0.889	valid_0&#39;s binary_logloss: 0.264878<br>[800]	valid_0&#39;s auc: 0.891303	valid_0&#39;s binary_logloss: 0.261709<br>[1000]	valid_0&#39;s auc: 0.891646	valid_0&#39;s binary_logloss: 0.261943<br><br>AUC Score: 0.8915966277561608<br><br>&gt;&gt;&gt;&gt;&gt; ROC_AUC: 0.891597 &lt;&lt;&lt;&lt;&lt;</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*hH23-UVppqiJzFkL5sA59g.png" /></figure><pre>:::::::::::::: Fold ~ 7 :::::::::::::::<br><br>[200]	valid_0&#39;s auc: 0.86711	valid_0&#39;s binary_logloss: 0.297103<br>[400]	valid_0&#39;s auc: 0.88302	valid_0&#39;s binary_logloss: 0.273565<br>[600]	valid_0&#39;s auc: 0.887891	valid_0&#39;s binary_logloss: 0.265141<br>[800]	valid_0&#39;s auc: 0.889325	valid_0&#39;s binary_logloss: 0.262646<br>[1000]	valid_0&#39;s auc: 0.889721	valid_0&#39;s binary_logloss: 0.262905<br><br>AUC Score: 0.8893856108562247<br><br>&gt;&gt;&gt;&gt;&gt; ROC_AUC: 0.889386 &lt;&lt;&lt;&lt;&lt;</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*Zmm7NRJaCyaZttrqEcI3HQ.png" /></figure><pre>========================================<br> . . . . . . . . . . . . . . . . . . . .<br> AUC Score (mean): 0.8928928273611749<br> . . . . . . . . . . . . . . . . . . . .<br>========================================</pre><h4>Prediction Histogram</h4><pre>sns.set()<br>plt.hist(preds, bins=50)<br>plt.gca().set_facecolor(&#39;lightblue&#39;)<br><br>min(preds), max(preds)</pre><pre>(4.988535168506163e-05, 0.9864792432482054)</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*arsUtXPSsHzqn_bba_U3PA.png" /></figure><h4>LightGBM — Feature importance</h4><p>“Feature importance” determines the relationship between features and the target variable, and also identifies features that are irrelevant to the model.</p><pre>from lightgbm import plot_importance <br><br>plot_importance(model, figsize=(12, 30), color=[&#39;r&#39;,&#39;y&#39;,&#39;g&#39;,&#39;b&#39;], height=0.6, max_num_features=100,<br>                title=&#39;LightGBM - Feature importance&#39;, xlabel=&#39;Value&#39;, ylabel=&#39;Name Feature&#39;);<br><br>plt.gca().set_facecolor(&#39;lightyellow&#39;)</pre><figure><img alt="png" src="https://cdn-images-1.medium.com/proxy/1*hB8RnL0Wx5UvIDV5eDIjeQ.png" /></figure><h4>Kaggle Submission (LGBM)</h4><pre>sub_lgbm = df_sample.copy()<br>sub_lgbm[&#39;target&#39;] = preds<br># sub_lgbm.to_csv(&#39;submission_lgbm.csv&#39;,index=False)<br><br># !ls</pre><h4>Ensembling &amp; Submission</h4><pre>sub = df_sample.copy()<br>sub[&#39;target&#39;] = (0.20 * sub_bayes[&#39;target&#39;]) + (0.80 * sub_lgbm[&#39;target&#39;])<br># sub.to_csv(&#39;submission.csv&#39;,index=False)<br><br># !ls</pre><h4>The results of this notebook in Kaggle</h4><h4>Gaussian Naive Bayes (GaussianNB):</h4><ul><li>Private Score: 0.88670</li><li>Public Score: 0.88787</li><li>oof Score: 0.88805</li></ul><h4>LightGBM:</h4><ul><li>Private Score: 0.89611</li><li>Public Score: 0.89849</li><li>oof Score: 0.89671</li></ul><h4>Ensembling:</h4><ul><li>Private Score: 0.89674</li><li>Public Score: 0.89869</li></ul><p>You can check the code on Kaggel: <a href="https://www.kaggle.com/code/somayyehgholami/the-bayesian-approach-for-efficient-optimization">https://www.kaggle.com/code/somayyehgholami/the-bayesian-approach-for-efficient-optimization</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf00a157a3b2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Digital Pathology: A Journey into the Future of Diagnostic through deep learning]]></title>
            <link>https://medium.com/@somayyeh_43305/exploring-digital-pathology-a-journey-into-the-future-of-diagnostic-through-deep-learning-8394335821c2?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/8394335821c2</guid>
            <category><![CDATA[cnn-image-analysis]]></category>
            <category><![CDATA[ai-in-medical-diagnosis]]></category>
            <category><![CDATA[ai-cellular-analysis]]></category>
            <category><![CDATA[tissue-classification]]></category>
            <category><![CDATA[slide-image-segmentation]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Fri, 03 Feb 2023 10:17:30 GMT</pubDate>
            <atom:updated>2023-10-29T13:28:33.889Z</atom:updated>
            <content:encoded><![CDATA[<h3>Part 1: The Evolution of Pathology</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k7vE7WX47Y65g55WeyHO3Q.png" /></figure><p><em>As a machine learning engineer, I am excited to delve into the world of digital pathology and explore the role of deep learning in this field. With the increasing use of whole slide imaging technology and the growing demand for more accurate and efficient diagnoses, the application of Deep Learning in digital pathology is a fascinating and relevant topic to study. In a collection of articles, I will examine how deep learning algorithms transform the field of pathology, from preprocessing images to automating diagnoses. These articles will be an invaluable resource for those looking to understand the intersection of deep learning and digital pathology.</em></p><p><em>This article was written by </em><a href="https://www.linkedin.com/in/somayyeh-gholami-1ba73a3?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base_contact_details%3BMrzQXRWrQ0eaU0NrXXGQNA%3D%3D"><em>Somayyeh Gholami </em></a><em>in Feb 2023 in English</em>. <em>Meanwhile, I’m interested and intrigued by digital pathology, and AI-driven image analysis in digital pathology was my master’s thesis at VUB university.</em></p><h3>Introduction</h3><h4>Pathology and WSI</h4><p>Pathology is a branch of medicine that focuses on the diagnosis. It involves the examination of body tissue samples or biopsies in the laboratory and producing a pathology report. The diagnostic process integrates all sources of clinical information, both gross and microscopic, into the pathology slides. The emergence of whole slide scanner technology has taken pathology’s workflow to the next level. They digitally record histologic details of tissues from a glass slide in high resolution. The digital pathology slides are called images or whole slide imaging(WSI).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*UVpBrWGzFCPF2cp_XAvFhw.png" /><figcaption>This image was made by the author.</figcaption></figure><h4>The Evolution of Pathology: The Integration of AI and Whole Slide Imaging</h4><p>Incorporating whole slide imaging (WSI) into the pathology diagnosis process has resulted in the accumulation of vast amounts of digital slides, creating a big data scenario that has opened up new possibilities for AI-driven image analysis. The demand for second opinions, the need to accelerate the diagnostic process and telepathology, and the desire to reduce human errors have highlighted the potential benefits of AI-driven image analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*lbu6kzqnnJevFn7Qp9kX4Q.png" /><figcaption>This image was made by the author.</figcaption></figure><h4>Applying the protocol to produce the digital image(WSI)</h4><p>The protocol for getting digital images involves several steps: collection and fixation, dehydration and clearing, paraffin embedding, microtomy, staining, mounting, and digitalization¹.</p><p>Staining is crucial to highlight specific structures or substances within the tissue. A common technique used to examine tissue samples under a microscope is H&amp;E staining, which is frequently mentioned in digital pathology papers. The H&amp;E technique makes cell nuclei appear dark blue (due to Hematoxylin staining) and other cellular structures seem red or pink (due to Eosin staining). This staining allows the identification of different tissue parts and is essential for accurate diagnoses. Nevertheless, sometimes the H&amp;E technique is not sufficient for diagnosis; for example, to identify plasma cells, a biomarker CD138 Immunohistochemistry is used to highlight plasma cells. At the same time, applying CD138 shows a slide mainly in brown and blue colors (Also different shades and combinations of these two colors). The membranes of plasma cells and epithelium cells are brown, and stroma cells are blue.</p><p>Different artifacts can arise from each step and cause lower image quality. The choice of fixative, proper dehydration, uniform microtomy, and accurate staining are essential factors in maintaining the appearance of the tissue. Mounting and digitalization can also lead to artifacts such as dust, air bubbles, color variations, and image blurring. Proper tissue sample storage is also crucial to avoid discoloration and alteration of the staining pattern¹.</p><figure><img alt="Protocol applied to to produce histological slide.This image was sourced from the paper¹." src="https://cdn-images-1.medium.com/max/535/1*dgCQZj2QQ6Jej3HOwByT7Q.jpeg" /><figcaption>This image was sourced from the paper¹.</figcaption></figure><p>Meanwhile, image preprocessing for artifact detection is necessary to accelerate training and enhance model generalizability¹.</p><h4>Extracting the information from WSI: Understanding Cellular Analysis and Segmentation</h4><p>The most used categories of measurements that can be extracted from digital slides(WSI) are area-based, cell-based, and measurements relating to objects in the tissue aside from cells².</p><p>The cells are one of the most fundamental units in a typical histology image (Cellular analysis), and their nuclei and cytoplasm can be clearly seen. Digital image analysis can be used to provide a quantitative representation of cell staining, shape, and organization to support diagnosis and prediction.</p><p>The term “segmentation” represents a process in which an image is spatially broken down into constituent parts with some significance or utility. Segmentation can be applied to a histology image to outline tissue compartments such as stroma or epithelium, to distinguish tumor from benign regions, or to identify subcellular structures such as cell nuclei. Cell segmentation generally refers to the process of identifying groups of pixels that represent a cell. Segmentation typically involves two key steps: identification of candidate pixels and distinction of these pixels into compartments that best approximate the spatial confines of a single cell. This process is often complicated in the presence of adjacent or overlapping cells, which can lead to under-segmentation (the representation of more than one cell as a single object)³.</p><h4>Automation of the Cellular Analysis and Segmentation</h4><p>Digital pathology has automated the cellular analysis and segmentation process through the following three typical tasks: segmentation, detection tasks, and classification⁴.</p><p>Segmentation and Detection Tasks in Digital Pathology: Defining Boundaries and Identifying Object Centers. A segmentation task aims to accurately outline the boundaries of histologic primitives(i.e., nuclei, epithelium, tubules, and IDC) so that precise morphological features can be extracted. In contrast, detection tasks such as lymphocyte and mitosis detection focus on simply locating the center of the target primitive. Segmentation proves to be more challenging than detection, especially when the primitive can have multiple visual representations. This diversity in the presentation may require a more complex model that can capture the full range of visual cues used to identify a specific histologic primitive⁴.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/575/1*2q6kcLz3qKr4dgiLxIdBEA.png" /><figcaption>This image was made by the author.</figcaption></figure><p>Tissue level classification (i.e., lymphoma subtype identification): Instead of explicitly identifying individual tissue-based primitives (e.g., mitoses, nuclei) and trying to identify primitive specific features to predict tissue class, an alternative strategy is to directly apply a deep Learning classifier on the image patches which have been tagged with the class label⁴.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UOzsFxxt6R8WMmEbnlofdg.png" /><figcaption>A typical workflow for DP research</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/411/1*_QmgICLBW9qMxaK3tX8dSw.png" /><figcaption>This image was sourced from the paper⁴.</figcaption></figure><p>Typical architectures used in deep learning frameworks. (a) Classification task: starting from the input image, the network predicts the class label. (b) Detection task: the network locates the objects of interest within the image. (c)Segmentation task: the deep network predicts the objects’ contour¹.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/622/1*4smnh4VnB17Lqds0gZBzYg.jpeg" /><figcaption>This image was sourced from the paper¹.</figcaption></figure><p>This subject will be further explored in a series of articles.</p><h4>This concludes Part One. Stay tuned for the next installment on my Medium.</h4><p><strong><em>References</em></strong></p><ol><li>Salvi, M.; Acharya, U. R.; Molinari, F.; Meiburger, K. M. The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis. Computers in Biology and Medicine 2021, 128, 104129.</li></ol><p>2. Aeffner, F.; Wilson, K.; Bolon, B.; Kanaly, S.; Mahrt, C. R.; Rudmann, D.; Charles, E.; Young, G. D. Commentary: Roles for Pathologists in a High-throughput Image Analysis Team. Toxicologic Pathology 2016, 44, PMID: 27343178, 825–834.</p><p>3. Irshad, H.; Veillard, A.; Roux, L.; Racoceanu, D. Methods for Nuclei Detection, Segmentation and Classification in Digital Histopathology: A Review Current Status and Future Potential. IEEE reviews in biomedical engineering 2014, 7, 97–114.</p><p>4. Janowczyk, A.; Madabhushi, A. Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases. Journal of Pathology Informatics 2016, 7, 29.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8394335821c2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a “Smart Blockchain” with Solidity, and amazed at how it prevents “Double-Spending”]]></title>
            <link>https://medium.com/thecapital/build-a-smart-blockchain-with-solidity-and-amazed-at-how-it-prevents-double-spending-25becb31feb8?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/25becb31feb8</guid>
            <category><![CDATA[smart-blockchain]]></category>
            <category><![CDATA[solidity]]></category>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[double-spending]]></category>
            <category><![CDATA[erc20]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Thu, 18 Feb 2021 08:22:10 GMT</pubDate>
            <atom:updated>2021-02-22T23:24:09.953Z</atom:updated>
            <content:encoded><![CDATA[<p>This is how we define “Smart Blockchain”; Any blockchain that uses smart contracts to verify transactions is a “Smart Blockchain.” In several previous articles, we reviewed the benefits of “Smart Blockchain” and explained the details of this idea for future blockchain use. But the next question is whether we can deploy a smart contract on an active blockchain (like Ethereium); To verify transactions without the need for miners? The answer is yes. In this article, we decide to create a smart Ethereum contract; Which can prevent it from “Double-Spending” its own tokens.</p><p>In this article, we first explain the method of producing tokens with ERC20 standard and write a smart contract for producing these types of tokens. Then we will develop the same smart contract so that it can store the details of all accounts since the birth of the tokens. This means that in addition to transactions, the details of all accounts that have received these types of tokens at least once are stored. In this way, this smart contract over time can calculate the balance of all accounts (i.e., the general ledger for this type of token) using transaction specifications. But the important point is that this contract has been deployed on the decentralized network of Ethereum, and the state of smart contracts in this network will be updated only after the consensus of all miners and the construction of a new Ethereum block.</p><p>In general, delays in updating the state of the smart contracts can allow users to “Double-Spending.” But we all know that miners and block producers verify transactions (with spending very high power consumption) to create each new block on the Ethereum network, and will not allow anyone to “Double-Spending.” But we assume that, for example, a fraudster has been able to take advantage of the decentralization of the network and record an invalid transaction of our token in the Ethereum blockchain and, so to speak, spend his token twice. In this way, the new state of our contract will also be invalid. But we already add codes to our contract that even, in this case, our smart contract will automatically detect and delete invalid transactions in its new state. In other words, we will see that the balance of all general ledger accounts will always be correct, and there is no need to verify the Ethereum network miners.</p><p>Do not be surprised; Verification of all transactions is possible with just a few lines of code with a Solidity language, and we will explain all the necessary details for the first time in this article. We also simulate fraud in transactions, so that together we can test the performance of our smart contract. In fact, we will show once again that the task of verifying tokens and cryptocurrency transactions can be entrusted to smart contracts with confidence, and there is no need for all sorts of consensus mechanisms and the loss of hundreds of millions of dollars in electricity costs. The present article was written in February 2021 by <a href="https://medium.com/@somayyeh_43305">Somayeh Gholami</a> and <a href="https://www.mehrankazeminia.com/">Mehran Kazeminia</a> in two versions in English and Persian.</p><p><a href="https://www.soliset.com/solidity-smart-blockchain"><em>Persian Article</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3-5kFpJFpVUZ7eE-2T8jbg.jpeg" /><figcaption><strong>Understanding “Smart Blockchain” by coding</strong></figcaption></figure><h3>Step 1 — Producing ERC20 tokens</h3><p>First, we want to produce one million ERC20 tokens called “AKA.” To do this, we have written a smart contract according to the ERC20 standard. You can download all the codes of this article <a href="https://github.com/SomayyehGholami/Smart-Blockchain-with-Solidity">here</a>. But note that our file name at this stage is <a href="https://github.com/SomayyehGholami/Smart-Blockchain-with-Solidity/blob/master/SmartCreator101.sol">SmartCreator101.sol</a>. We will review the details of this contract together.</p><p>The most widely used smart Ethereum network contracts are ERC20 token contracts. Fortunately, good articles and libraries have been published in this field. There are even other standards such as ERC223, ERC621, ERC777, ERC827 to complete and develop the ERC20 standard. But to make it easier for readers to understand the details of contract development, we avoided importing standards and libraries. Meanwhile, the essential SafeMath library, mandatory functions, and ERC20 standard events are all included in the SmartCreator101.sol contract.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rtk4tYDuh3LCK9XLzvwe0Q.jpeg" /></figure><h3>The basic assumptions of the SmartCreator101.sol contract are as follows:</h3><p>We store the address of the person who deploys this smart contract on the network as owner. The owner must specify the name, symbol, and number of tokens when deploying the contract. These parameters will always remain constant. Also, from the very beginning, all the generated tokens are equal to the balance of the owner account. By doing this, Only the owner will decide to sell or transfer all the tokens. We set the decimals parameter to zero. That is, we assumed that a fraction of a token could not be transferred and that an integer number of tokens could always be spent.</p><h3>The functions of the SmartCreator101.sol contract are as follows:</h3><p>The <em>totalSupply</em> and <em>balanceOf</em> functions are of the view function. These types of functions only read values from the blockchain and return those values. For this reason, they do not even have the cost of gas. This means that if someone calls <em>totalSupply</em>, they will receive the total number of tokens, and if someone calls the <em>balanceOf</em> function with an address, will receive the balance of the same address. These two functions do not do any calculations, but in any case, they are mandatory functions of the ERC20 standard and must remain the same and without any changes in the contract. For example, if you want to change the <em>balanceOf</em> function and add the token name to the function output before the balance value; ERC20 wallets may not work properly.</p><p>The next function is <em>‘transfer.’</em> The sender always calls this function and at the same time specifies the recipient’s address and the number of tokens sent. This function then performs all the necessary controls, and if it does not find a problem, it updates the transmitter and receiver balance, and so the transfer is completed. Since the sender calls this function, it has to pay for the gas himself. Also, because we have to model the transfer process in the same way in other functions, we decided to add another function called ‘<em>_transfer</em>’ internally. This way, only we write transfer controls and calculations at once. We also added another function called <em>‘add’</em> to control <em>Overflow</em> (when adding operations for uint256 variables). This is an internal and <em>pure</em> function. It means it does not read blockchain information and does not store anything in the blockchain.</p><p>There are three functions that have not yet been explored. The <em>‘approve’</em> function is used when someone wants to delegate someone else to spend a number of their own tokens. To do this, he must call this function and at the same time specify the address of the spender and the number of tokens (<em>allowance</em>). This function then stores the holder’s address, spender’s address, as well as the number of tokens in an embedded mapping. The name of the next function is <em>‘transferFrom’</em>. This function is performed by a spender call, which must simultaneously specify the holder’s address, recipient’s address, and the number of tokens sent (less than or equal to <em>allowance</em>). Of course, this function will also perform all the necessary controls, and if there is no problem, this function will be executed and will update the sender’s balance (holder), recipient’s balance, and allowance value. Please note that the gas cost is paid by the holder to perform the ‘<em>approve</em>’ function. But to perform the <em>‘transferFrom’</em> function, the gas fee must be paid by the executor of a contract, the spender. In other words, although the spender’s balance is the same and does not change when running the <em>‘approve’</em> function, it reduced the cost from gas when performing the <em>‘transferFrom’</em> function. The last function is called ‘allowance’, which is a <em>view</em> function. Anyone can call this function without paying gas and read the remaining<em> allowance</em> from the blockchain. Obviously, when calling this function, you must specify the address of the holder and the address of the spender you want.</p><h3>The events of the SmartCreator101.sol contract are as follows:</h3><p>The name of the first event is <em>‘Transfer’</em>. After each successful transaction, this event sends a message to the outside environment of the contract and the transaction details are stored in the blockchain. The parameters of this event are the sender’s address, the recipient’s address, and the number of tokens sent. The second event is <em>‘Approval’</em>. Each time the <em>‘approve’</em> function is executed successfully, this event is also stored in the blockchain. The second event also has three parameters. The parameters of this event are holder address, spender address, and <em>allowance</em> value, respectively.</p><h3>Testing SmartCreator101.sol contract</h3><p>Now it’s time to test our contract. To do this, we use the Rinkeby and MetaMask test networks. After compiling the contract in Remix, we go to the deployment section. Then in the <em>ENVIRONMENT</em> box, we must select the <em>Injected web3</em> option. Of course, we have several accounts in the Rinkeby test network. For example, with account number one, we deploy the contract and pass the following values to the contract:</p><blockquote>name = “AKA Token” ________The name of the coins<br>symbol = “AKA” ____________Symbol of the coins<br>totalValue = 1000000 ______Total number of coins</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RE_bge0XSLqvEr9cN27DEw.jpeg" /></figure><p>In this way, the address of account number one will be the owner of this contract, and according to the contract codes, the owner’s balance will be registered equal to one million AKA tokens. Before we test each function, it is better to introduce Aka Token to MetaMask wallet so that we can see the balance of accounts in MetaMask at the same time. So we have to add AKA Token to MetaMask wallet. To do this, first from the Remix, we copy the address of the deployed contract, and in MetaMask, we paste this address on a special box. Also, type the token symbol “AKA” in the lower box. When Aka Token is added to the number one account, MetaMask displays the balance of this account as one million AKA Tokens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E5wVKlHYOqj77qfyPANA1g.jpeg" /></figure><p>Now we can easily test the performance of all the functions of the contract. The blue functions are view type and have no gas cost. But if we call the orange functions, we will be charged for the gas by the blockchain. Of course, for now, our contract has been deployed on the Rinkeby test network, and the gas fee will be paid with fake Ether. Below, you can see the pictures of some of the tests performed. You can also do more tests to make sure this contract works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AxYY4Cu4YkiCg7JdlgYUAg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gdj0xTbDD_41P0bigZGKMQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g4wkaYAxvjPUgLiRmwNnZA.jpeg" /></figure><h3>Step 2 — ERC20 contract development</h3><p>As mentioned earlier, besides the transaction information, we want to store the records of all addresses, from the birth moment of tokens, that at least once have received an Aka Token. In this way, over time, we can calculate the balance of all accounts using transaction specifications. You will also find that verifying transactions and solving the problem of double-spending could be done easily by using the same stored information. So at this point, we need new Structs, mappings, and calculations. On the other hand, let’s remind you that the ERC20 standard must always be considered.</p><p>To do all this, one way is to individually deploy a smart block producer contract(BPSC) on the Ethereum network and use that contract in the existing contract. Of course, in this case, the deployment of contracts must be done in two stages. (First, for example, contract X must be deployed, and after specifying its address, this address can be registered in contract Y, and finally, contract Y must be deployed.) Also, in this method, we must observe other cases. But our main goal in this article is to make it easy for the readers to understand the details. So for the second step, we have prepared a contract called <a href="https://github.com/SomayyehGholami/Smart-Blockchain-with-Solidity/blob/master/SmartCreator102.sol">SmartCreator102.sol</a>, to access this file, all the necessary items have been added to the text of the SmartCreator101.sol contract, and in fact, the functions of the first contract have been developed in the second contract.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MFPo8e7BEyqutVQLPfFHUQ.jpeg" /></figure><p>The most important data of the new contract are stored in mapping accounts and struct <em>trxes</em>. Mapping accounts stores all customer information (balance, etc.). Even if an address receives an “Aka Token” for the first time and is not yet a member of this mapping, all the details of this account will be added to the mapping accounts immediately and this account will be a member forever. The <em>‘trxes’</em> struct stores transaction details. The functionality of the embedded mapping of <em>‘allowances’</em> in the new contract is exactly the same as the previous contract. In addition, some other parameters have been added to the new contract to make programming easier.</p><p>In the new contract; The first specification stored in the mapping accounts (client number zero) is the owner specification. And the first specification stored in the <em>trxes</em> struct (transaction number zero) is that the owner balance is equal to the total number of tokens. In addition, the other functions of the old and new contracts are very similar. But the <em>‘_transfer’</em> function is an exception and has changed a lot. Most of the tasks we expect from a new contract take place in this function. In the following, we will examine the functionality of this function in detail and take a closer look at its codes.</p><h3>The Specifications of the ‘_transfer’ function and checking its functionality in the new contract</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SBlhLICqD6KYsSMcvBmrJw.jpeg" /></figure><p>As you can see in the picture above; Almost the basic code of the <em>‘_transfer’</em> function has not changed. It means the function specifications and the basic requirements, are like the first contract. Only we added a new modifier called <em>‘isLocked’</em> to the specifications of this function. This modifier does not allow this function to call two different addresses at the same time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1t8mdJP1gBhFqtL0gSIqrg.jpeg" /></figure><p>In the image above, you can see the continuation of the <em>_transfer</em> function codes. These codes are the most important part of the new contract. It may seem a bit confusing, but do not worry because these codes do only a few simple things in order. We have divided the image above into several sections, and in the following, we will give a brief explanation for each of these sections.</p><p><strong>In Section A;</strong> The balance of all accounts stored in the contract is added together and the result is stored in the <em>‘totalChecking’</em> variable. Bear in mind that the basis of calculations is always the state of the moment of the contract, which has been updated with the latest Ethereum network block.</p><p>Sections C, B, and D; are only activated when the value obtained for the <em>‘totalChecking’</em> variable does not equal <em>‘totalValue’</em> the actual number of contract tokens. In this case, we find that individuals have been able to take advantage of network decentralization and record one or more invalid transactions from our token in the Ethereum blockchain. In other words, fraudsters have managed to spend their tokens twice. Therefore, in order to correct the balance of all accounts in this contract, we must identify and delete invalid transactions.</p><p><strong>In Section B;</strong> We zero the balance of each contract account. Of course, this may sound scary, but fortunately, there is no problem, and we still have information on all transactions.</p><p><strong>In section C;</strong> We start with transaction number zero (related to the owner) and examine each transaction in order. Obviously, the balance between the sender and receiver accounts is updated only when a transaction is valid. The main condition for a transaction to be valid is that the sender’s balance at that moment is equal to or greater than the transaction amount. In this way, the codes in this section cause all invalid transactions that have been registered in our contract through the Ethereum network by deception or even mistakes of the miners to be ignored. This means that double-spending transactions can never be a problem in our calculations, and account balances are always corrected automatically. So there is practically no need to verify the miners. Of course, you can delete invalid transactions immediately, and even before deleting them, notify it outside the contract by Event, so that it can be registered on the Ethereum blockchain. But even if you do not delete, each time these transactions are detected and ignored. Note that these invalid transactions are already stored in the Ethereum blockchain, however.</p><p>The code in this section might seem a bit complicated, but actually, it’s simple functionality. If you look at the details of the code for a few minutes, you will notice that these complexities are created solely to access the data stored in the struct and mappings. Also, by paying more attention to codes, you will make sure that the mathematical logic of these few lines of code is much stronger than the logic of various consensus mechanisms, and the codes of this contract can be a much better alternative to various consensus mechanisms (for cryptocurrency transactions and token transactions). The main problem of decentralized networks has always been the problem of double-spending. But in this contract, this is mathematically impossible to happen. Even for double-checking the contract codes, we will have another <em>‘require’</em> in section E.</p><p><strong>In Section D;</strong> The balance of all accounts (modified in Section C) are added together, and the result is stored in the same <em>‘totalChecking’</em> variable. In other words, the value calculated in section A for the <em>‘totalChecking’</em> variable is modified.</p><p><strong>In Section E;</strong> There is only one <em>‘require’</em>. This ‘require’ once again controls the equality of the value obtained for the <em>‘totalChecking’</em> variable with the actual number of contract tokens (<em>totalValue</em>). Please note that this condition will definitely be met. Because the variable <em>‘totalChecking’</em> was calculated once in section A, and if it was not equal to <em>‘totalValue’</em>, it was modified in sections B, C, and D, and now this condition certainly has met.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zg1FWy4VceuLk-w8v33Kxg.jpeg" /></figure><p>In the following, we will give a brief explanation for the last three parts of the ’<em>_transfer’</em> function (above picture). These three sections are only executed when all the necessary controls have been run. Of course, as you saw in the previous sections, even in the presence of invalid transactions, the balance of all accounts has been carefully adjusted from the beginning.</p><p><strong>In section F;</strong> If the recipient receives the tokens “AKA token” for the first time, all the recipient details will be stored in the mapping accounts. The <em>‘NewAccountWasOpened’</em> event is also executed immediately.</p><p><strong>In section G;</strong> First, the number of tokens in this transfer means <em>‘amount’</em> is deducted from the sender’s balance, and then exactly the <em>amount</em> is added to the recipient’s balance. As you know, it is always recommended to update the sender’s balance first and then update the receiver’s balance.</p><p><strong>Section H;</strong> The final part of the function is <em>‘_transfer’</em>. In this section, all new transaction details are stored in the <em>trxes</em> struct. Also, the <em>‘Transfer’</em> event will be executed immediately.</p><h3>Testing SmartCreator102.sol contract</h3><p>You can simply test a new contract just like the initial one. But please note that normally transactions are executed only if they are valid. In other words, you simply can not observe the performance and test this contract in dealing with an invalid transaction. So what to do?</p><p>Our suggestion is to make small changes to the code before deploying the contract so that we can artificially simulate the occurrence of double-spending. To do this, we have created the <a href="https://github.com/SomayyehGholami/Smart-Blockchain-with-Solidity/blob/master/SmartCreator102x.sol">SmartCreator102x.sol</a> file, and in the following, we will explain the changes made to this file. Bear in mind that this file is for testing only, and the main file at this stage is the <a href="https://github.com/SomayyehGholami/Smart-Blockchain-with-Solidity/blob/master/SmartCreator102.sol">SmartCreator102.sol</a> file. Also, in order for the tokens of this stage to be different from the tokens of the first stage, we have named the symbol of the tokens of the second stage AKAx.</p><p><strong>For example,</strong> suppose that the total number of second-stage tokens is one million “AKAx” tokens and the ‘owner’ of the contract is account number one. In this way, after deploying the contract, the balance of account number one will be equal to one million. In the first transaction, we send five hundred thousand tokens from the first account to the second account. In the second transaction, another five hundred thousand tokens will be sent from the first account to the third account. Now we assume that in the third transaction there is fraud or error and the third account succeeds in spending all five hundred thousand tokens twice. This means, for example, that the third account can send one million tokens to the fourth account. Therefore, after the third transaction, the account balances will be as follows:</p><blockquote>Account balance number one = 0<br>Account balance number two = 500000<br>Account balance number three = 0<br>Account balance number four = 1000000</blockquote><p>But you will see that this mistake does not continue and our contract automatically detects the invalid transaction during the fourth transaction and ignores it. It will also correct the balance of all accounts at the same time. That is, if, for example, in the fourth transaction, one hundred thousand tokens are sent from account number two to account number one, in this case, you will be surprised and the balance of accounts will be corrected as follows:</p><blockquote>Account balance number one = 100000<br>Account balance number two = 400000<br>Account balance number three = 0<br>Account balance number four = 500000</blockquote><p>Of course, there are several ways to create this type of fraud (or mistake) for the contract. For example, in the SmartCreator102x.sol file, in order for transaction number three to be able to spend twice, we have just added the following code to the SmartCreator102.sol file:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9kshxJQOx2xtRWYjT2G9yQ.jpeg" /></figure><p>For testing the contract in the second stage; We deployed the SmartCreator102x.sol file on the Rinkeby test network according to the specifications mentioned earlier. Then, according to the above description, we did four transactions. If you look at the pictures below, you will see that the test results are completely in line with our predictions. Of course, you can also do more tests to make sure this contract works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1FILxxA0TILy05nhUbddFw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bVJMfdu9LETjt-k-0xtxzw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cE377RZ35uTzYahz6g06-g.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z-mpJWnNTBHEhQAQOzsu-Q.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v1eo_8x5PFkg0LZ2HfRaHA.jpeg" /></figure><p>In this way, over time, we will always have an independent and reliable general ledger for our tokens, and there is no need to verify by miners. Of course, as you can see, in this contract, the verification of the transaction was done before executing each transaction. But this scheduling, in other contracts, can be adjusted on another basis (for example, every few seconds). Meanwhile, In the third step, we provide several functions to access account and transaction information. Note, however, that this agreement is deployed on the Ethereum blockchain, and transaction information is stored and accessed as an ‘event’ in the Ethereum blockchain. You can see this information by using the tips and pictures below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vR_K8ZsyekDn7EyBqqWHOg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oCNm8lNKlULalMpFkslwOA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D2T3Z4MuXDKfcJykNQifPQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yih96uPFIvxs_kpOidOxnQ.jpeg" /></figure><h3>Step 3 — Adding new features and complete the contract</h3><p>To complete the contract, we add five new functions to the second phase contract. The name of our contract in the third phase is <a href="https://github.com/SomayyehGholami/Smart-Blockchain-with-Solidity/blob/master/SmartCreator103.sol">SmartCreator103.sol</a>. Note that the information of all transactions, the balance of all accounts, and other important information are stored in this contract. Therefore, three of the new functions are of the<em> ‘view’</em> type, and their function is simply to return the information stored in the contract. In addition, two other functions provide the possibility of selling tokens based on Ether. Read a brief description of the performance of these five new functions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nst9zPWAtguIsm9EylYH4w.jpeg" /></figure><p>As you have already seen; Addresses, that were sent AKA tokens at least once, are recognized as members, and their full account information (balance, etc.) is stored and updated forever. Now, if one of the members calls <em>myAccountInformation</em> function, he can receive his account balance, the time of his membership, the time of the last sending of the token by himself, etc., without paying for gas.</p><p>But the <em>systemInformation</em> function can be called any Atrium network address. This function returns general information about the status of tokens. That is, in order; Returns the name and symbol of the tokens, the total number of tokens, the selling price of the token based on ether (Wei), the inventory of the contract address based on the ether, the total number of members and even the total number of transactions performed.</p><p>Any Ethereum network address can also call the<em> lastBlock</em> function. This function returns the information of the last transaction performed.</p><p>The <em>buyTokens</em> function is the <em>payable</em> type. Any address can perform this function and buy some aka tokens by paying ether. Of course, as you know from the beginning all Aka tokens are in the owner account and the contract address does not have a token. This function, after the necessary controls for this transaction, first receives the ether from the customer and deposits it to the contract address, and then transfers the tokens from the owner’s address to the customer’s address.</p><p>But <em>payToOwner</em> function can only be executed by the owner. If with the sale of aka tokens, some ether accumulates in the contract address, the owner can perform this function. By performing this function, all ethers collected in the contract address are transferred to the owner’s address.</p><h3>Testing SmartCreator103.sol contract</h3><p>Just like the previous steps, you can deploy the SmartCreator103.sol file on the Rinkeby test network and test these five new functions. Only when testing the <em>buyTokens</em> function, note that in the codes of this contract, by default, the price of each aka token is equal to 1e12 wei. On the other hand, in this contract, the value of decimals is zero, and no fraction is defined for aka tokens. So the amount paid based on wei must be a multiple of 1e12. Otherwise, the <em>buyTokens</em> function will not run.</p><h3>Summary and conclusion:</h3><blockquote>You can download all the article files <a href="https://github.com/SomayyehGholami/Smart-Blockchain-with-Solidity">here</a> and do more tests.</blockquote><blockquote>The <strong>SmartCreator101.sol</strong> file is a smart contract that generates tokens according to the ERC20 standard.</blockquote><blockquote>The <strong>SmartCreator102.sol</strong> file is another smart contract that, in addition to generating ERC20 tokens, stores all transaction details and member account details. In addition, this contract can verify the transaction of its own tokens and does not allow anyone to spend twice. In other words, this contract does not require the services of Ethereium network miners to verify transactions. The mathematical logic of the codes of this agreement is much stronger than the logic of various consensus mechanisms, and the general ledger of this intelligent agreement is quite reliable.</blockquote><blockquote>The <strong>SmartCreator102x.sol</strong> file is the same as the previous contract, except that in the third transaction, it sends the transaction amount twice to the recipient. This was done simply to simulate double-spending. Of course, we saw that immediately in the fourth transaction, the contract was able to identify the error or fraud and change the balance of all accounts to the correct state.</blockquote><blockquote>The <strong>SmartCreator103.sol</strong> file is the final version of this smart contract. This contract has five more functions than the second stage contract, and with these functions, new features have been added to the contract.</blockquote><h3>The last word</h3><p>Ethereum Smart Contracts have many limitations and are designed initially for specific applications. However, we were able to deploy a smart contract on this network that could detect invalid transactions. Of course, we hope that future blockchains will be planned from the beginning to use BPSC (Block Producer Smart Contracts) and that these types of smart contracts can be a very cheap alternative to current methods. We have already considered the <a href="https://medium.com/hackernoon/what-is-smart-blockchain-4b134275e90f"><strong>“Smart Blockchain”</strong></a> proposal for future blockchains in another article.</p><p>Unfortunately, currently, hundreds of millions of dollars per month in electricity cost for miners and blockchain builders, and a large portion of that staggering cost is related to controlling the double spending of cryptocurrencies and tokens using consensus mechanisms. On the other hand, the environmental problems of the current methods cannot be ignored. For all these reasons, we think that “smart blockchain” can be an important step for the sustainable development of blockchain technology. If you are interested in the topic, you can also read <a href="https://www.soliset.com/"><strong>our previous articles</strong></a>.</p><p><a href="https://medium.com/@somayyeh_43305"><em>Somayyeh Gholami</em></a><br><a href="https://www.mehrankazeminia.com/"><em>Mehran Kazeminia</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HLM8PHfHQs6ZUYAdfMc4qA.jpeg" /><figcaption><a href="https://cbeci.org/">Cambridge Bitcoin Electricity Consumption Index</a></figcaption></figure><p><strong>Check out our new platform </strong>👉<strong> </strong><a href="https://thecapital.io/"><strong>https://thecapital.io/</strong></a></p><p><a href="https://twitter.com/thecapital_io">https://twitter.com/thecapital_io</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Ff%2F3c1144%3Fas_embed%3Dtrue&amp;dntp=1&amp;display_name=Upscribe&amp;url=https%3A%2F%2Fupscri.be%2F3c1144&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3b6b127891c5c8711ad105e61d6cc81f/href">https://medium.com/media/3b6b127891c5c8711ad105e61d6cc81f/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25becb31feb8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/thecapital/build-a-smart-blockchain-with-solidity-and-amazed-at-how-it-prevents-double-spending-25becb31feb8">Build a “Smart Blockchain” with Solidity, and amazed at how it prevents “Double-Spending”</a> was originally published in <a href="https://medium.com/thecapital">The Capital</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The backup of the flight recorder’s data on blockchain networks during the flight]]></title>
            <link>https://medium.com/thecapital/the-backup-of-the-flight-recorders-data-on-blockchain-networks-during-the-flight-b3c3494c7bea?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3c3494c7bea</guid>
            <category><![CDATA[inflight-internet]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[flight-recorder]]></category>
            <category><![CDATA[black-box]]></category>
            <category><![CDATA[inflightwifi]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Mon, 18 May 2020 16:23:07 GMT</pubDate>
            <atom:updated>2020-05-22T01:08:15.059Z</atom:updated>
            <content:encoded><![CDATA[<h4>By <a href="https://medium.com/u/cc9da3ccea38">Somayyeh Gholami</a> on The Capital</h4><p>Horrible aviation accidents and incidents occur all over the world annually. After a plane crashes or explodes, usually, one of the most important issues is finding the plane’s flight recorder. Everyone is hoping it to be still intact and its stored data be retrievable. Unfortunately, this data is the only clue to discover the secret of the plane crash in many cases.</p><p>In this article, we will show that such important and vital data can be sent to one or more blockchain networks during flight, in addition to storing on the plane’s black box. Of course, the data encryption is done inside the aircraft and the blockchain technology will ensure the safety and accuracy of the stored data. This article was written in May 2020 by <a href="https://www.soliset.com/">Somayyeh Gholami</a> and <a href="https://www.mehrankazeminia.com/">Mehran Kazeminia</a> in two editions in English and Persian.</p><p><a href="https://www.soliset.com/black-box"><em>Persian Article</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tbWK4HCoxIwx2rcLV8jFYw.jpeg" /></figure><h3>Backup of the flight recorder’s data</h3><p>A black box (flight recorder) is a device used for airplanes, ships, and helicopters, and despite its name, it has a bright orange color to make it easier to find after a crash.</p><p>The flight recorder of aircraft is usually located at the tail of the aircraft, as this area is a low risk of collision. The aircraft’s flight recorder has two parts; the first is the flight data recorder (FDR) which records and stores data about instructions sent to the aircraft’s electronic system.</p><p>The second part is the cockpit voice recorder(CVR) in the cockpit, which stores the pilot’s conversations, wireless communications, and other types of sounds in the cockpit.</p><p>The subject matter of this article is merely the feasibility study to create a backup on blockchain networks and there is no difference between the types of flight recorder data to do so in practice.</p><p>In other words, if this proposal put into action, all information generated during the flight will be encrypted and immediately sent to blockchain networks in separate blocks.</p><p>In this case, the chain of blocks is formed only based on the arrival time on the blockchain network, and later on, it is impossible to change or tamper with the contents of these blocks.</p><p>Basically, in a blockchain network, the hashed contents of each block will be placed inside the following block, and with the slightest change in the contents of each block, the same block and all subsequent blocks will be invalid, and therefore no flaw or violation is possible. For more information on this topic, you can refer to the blockchain articles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*twWSo1Yk2TG0AS78q7VBWQ.jpeg" /></figure><h3>Feasibility to create a backup</h3><p>It’s been several years that airline passengers can use wifi to connect the internet easily during the flight. While the plane in the sky, the airline passengers check their email or stock market prices, post on social media, or even trade Bitcoin.</p><p>But still, the file of the audio environment of the cockpit stored only and only in the flight recorder. That makes no sense at all because for whatever reason that plane crashed, the plane manufacturer, its owner, the country where the plane crashed, and plane occupants’ families should be waiting to retrieve the flight recorder’s information to find out the main cause or the culprit of the plane crash.</p><p>Unfortunately, in many cases, the black box data is damaged when the plane crashed or exploded. In some cases, falls occur in impassable areas or deep oceans so it’s arduous to find a black box. There are also often legal, political, and even military problems with finding and retrieving the black box information.</p><p>So really, why this crucial information not sent to a safe place other than inside the plane? In the following, we will examine the answers to the likely arguments against this proposal.</p><blockquote><em>A- Only the moment of the crash is important, and when it crashes, the Internet is likely to shut down. That’s why the backup will be incomplete and doesn’t matter much.</em></blockquote><p>This argument is not compelling. A review of several recent crashes shows that from the beginning of the flight, all the pilot’s conversations, the control tower, all the instructions sent by the pilot for the system, etc. can be of paramount importance.</p><p>For this reason, in addition to complying with the protocol, pilots of passenger planes usually turn on the flight recorder manually even before the automatic recording of it started, to record the conversations with the control tower before turning on the aircraft engines.</p><p>Meanwhile, by utilizing blockchain technology, such information is continuously encrypted in the airplane computers and prepared to be sent as the block.</p><p>If the airline’s internet is disconnected and reconnected intermittently for any reason, it’s okay, after reconnecting a few blocks will be sent with a time lag. But if the Internet would no longer be connected, for instance, when a crash or explosion occurs, one or more of the last blocks might never be sent.</p><p>This is the main challenge of this approach, and you should keep an internet connection during or even after the crash. Of course, the status quo is maintained by this proposal, and the original version is still on the flight recorder(black box).</p><p>But with a simple and inexpensive method, we were able to have a pretty complete backup.</p><blockquote><em>B- Sending the data out of the aircraft could jeopardize the data security, also the risk of manipulation far outweigh the benefit.</em></blockquote><p>This argument cannot be correct either because using blockchain technology, all information encrypted and has a public and private key.</p><p>It means, that only the holder of a private key can access confidential information which predetermined who has a private key, for example, the manufacturer or the owner of the aircraft or any other legal entity.</p><p>Also, as mentioned earlier, no one can change or tamper with the data after storing it on the blockchain network.</p><blockquote><em>C- Technically, connecting the plane to the Internet and sending data out continuously is impossible or even costly.</em></blockquote><p>The above argument also is not credible at all. Internet service provider companies have made significant progress so far. Rising demand led the world’s major airlines in cooperation with technology and telecommunication companies, to provide Wi-Fi Internet on board for a large number of planes. Even, Gogo(inflight internet provider) has recently promised its customers 5G technology, Albeit, some airlines have not equipped yet.</p><p>There are two ways to connect to WiFi on the Internet. The first method is much like the way cell phones use it. It means that the plane has an antenna under its body and is connected to telecommunication towers located on the ground.</p><p>The sending and receiving equipment is installed on the same telecommunication towers which serve for mobile phones. The only difference is that the equipment transmits waves to the sky and the planes.</p><p>Along the way, the plane shifts connection among the nearest telecommunication towers to keep the Internet connection as much as possible and provide it to passengers.</p><p>Of course, this method has a big problem, for example, if the plane is passing over the oceans and seas, the connection of passengers to the Internet will be cut off.</p><p>But, the second method is through satellites. The planes are connected to the nearest satellite using an antenna above their bodies. This method has much fewer disconnection and interruption problems. Even, it is well thought out about aerodynamic issues of the antenna above the fuselage and these antennas work well.</p><p>However, the quality or tariffs of airline internet are never to compete with the regular urban internet, and the regular urban internet will always be faster and cheaper. <strong>Nevertheless, today, it’s possible to transmit the compressed and encrypted data of the plane to the reliable servers at a low cost. What are we waiting for?</strong></p><p><a href="https://medium.com/@somayyeh_43305"><em>Somayyeh Gholami</em></a><br><a href="https://www.mehrankazeminia.com/"><em>Mehran Kazeminia</em></a></p><p><a href="https://twitter.com/thecapital_io">JavaScript is not available.</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Ff%2F3c1144%3Fas_embed%3Dtrue&amp;dntp=1&amp;display_name=Upscribe&amp;url=https%3A%2F%2Fupscri.be%2F3c1144&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3b6b127891c5c8711ad105e61d6cc81f/href">https://medium.com/media/3b6b127891c5c8711ad105e61d6cc81f/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3c3494c7bea" width="1" height="1" alt=""><hr><p><a href="https://medium.com/thecapital/the-backup-of-the-flight-recorders-data-on-blockchain-networks-during-the-flight-b3c3494c7bea">The backup of the flight recorder’s data on blockchain networks during the flight</a> was originally published in <a href="https://medium.com/thecapital">The Capital</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a “Smart Blockchain” in Python]]></title>
            <link>https://python.plainenglish.io/implementing-a-smart-blockchain-with-python-3183e0c1052e?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/3183e0c1052e</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[smart-contracts]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Thu, 16 Apr 2020 05:30:56 GMT</pubDate>
            <atom:updated>2021-05-28T13:14:58.594Z</atom:updated>
            <content:encoded><![CDATA[<h3>Implementing a “Smart Blockchain” with Python</h3><p><strong>Introduction:</strong> In this article, we are going to implement a simple and plain “smart blockchain” with Python language and compare it with a blockchain. We hope that by doing this, the main concepts and advantages of this technology will be more clearly specified and welcomed by the developers and the blockchain community. A Block Producer Smart Contract (BPSC) exists in a plain “smart blockchain” network. To make a successful transaction in the network, the sender must first transfer the cryptocurrency to the (BPSC) wallet, and immediately the (BPSC) will automatically send the same cryptocurrency to the final recipient’s wallet. This seemingly simple task saves accurate and impervious data from all successful transactions in the (BPSC) Smart Contract. The present article was written in April 2020 by <a href="https://www.soliset.com/"><em>Somayeh Gholami</em></a> and <a href="https://www.mehrankazeminia.com/"><em>Mehran Kazeminia</em></a> in two versions in English and Persian.</p><p><a href="https://www.soliset.com/python-smart-blockchain"><em>Persian Article</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XkpRtHBa5TiTWqdW.jpg" /></figure><h3>Before you start, consider the following points:</h3><p>If you are unfamiliar with “smart blockchain” and have not read our previous articles on this technology, you can see the outline of the subject in the following article. However, the necessary explanations are given during the implementation.</p><p><a href="https://hackernoon.com/what-is-smart-blockchain-4b134275e90f">.css-k31kfd-skeletonStyles-Skeleton{background-color:#eee;background-image:linear-gradient( 90deg,#eee,#f5f5f5,#eee );background-size:200px 100%;background-repeat:no-repeat;border-radius:4px;display:inline-block;line-height:1;width:100%;-webkit-animation:animation-16jpnkj 1.2s ease-in-out infinite;animation:animation-16jpnkj 1.2s ease-in-out infinite;}@-webkit-keyframes animation-16jpnkj{0%{background-position:-200px 0;}100%{background-position:calc(200px + 100%) 0;}}@keyframes animation-16jpnkj{0%{background-position:-200px 0;}100%{background-position:calc(200px + 100%) 0;}}‌.css-k31kfd-skeletonStyles-Skeleton{background-color:#eee;background-image:linear-gradient( 90deg,#eee,#f5f5f5,#eee );background-size:200px 100%;background-repeat:no-repeat;border-radius:4px;display:inline-block;line-height:1;width:100%;-webkit-animation:animation-16jpnkj 1.2s ease-in-out infinite;animation:animation-16jpnkj 1.2s ease-in-out infinite;}@-webkit-keyframes animation-16jpnkj{0%{background-position:-200px 0;}100%{background-position:calc(200px + 100%) 0;}}@keyframes animation-16jpnkj{0%{background-position:-200px 0;}100%{background-position:calc(200px + 100%) 0;}}‌</a></p><p>We assume that the reader is familiar with the Python language and comprehend blockchain concepts, hash, and so on. For this reason and to avoid getting the story long and tedious, we’re focusing on “smart blockchain” features. Of course, if you forgot some of the concepts of blockchain, you can refer to the following article. In this article, the author has fully implemented a blockchain with Python. We also use his ideas and codes to implement “smart blockchain” so that we can easily compare “smart blockchain” with blockchain.</p><p><a href="https://hackernoon.com/learn-blockchains-by-building-one-117428612f46">Learn Blockchains by Building One | HackerNoon</a></p><p>Our Python version is Python 3.7 and we use Spyder-Anaconda3 as the IDE. You can use your favorite IDE like PyCharm and so on. Of course, for example, Jupyter Notebook is not suitable for this purpose. We will also use Postman, but you can also use cURL</p><h3>Understand the functionality of “smart blockchain” through coding.</h3><p>Let’s get started and create a class called Smart_Blockchain. The constructor method of this class currently creates two empty lists, one for storing transactions and the other for storing chain of blocks. Meanwhile, in the constructor method of this class, we also write the code for creating the initial genesis block. Now we need to complete the following four methods:<br><strong><em>new_block, new_transaction, last_block, hash</em></strong></p><p>But there are no consensus issues in “Smart Blockchain” and neither the proof-of-work (POW) mechanism nor the proof-of-stock (POS) mechanism nor … needed. In other words, there is no need to produce a couple of alternatives of a single block like the Bitcoin and Ethereum network to let miners can choose a valid and correct block through spending billions of dollars and staggering power consumption. The following code is set exactly based on these features.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fc088f65d007a7ceb2f1aab1a114e0d7/href">https://medium.com/media/fc088f65d007a7ceb2f1aab1a114e0d7/href</a></iframe><p>In the above codes, there is no proof variable or nonce and there is no need to block reward. Because, as mentioned earlier, there is no consensus on “smart blockchain.” Even transaction fees could be zero. But to avoid spamming, for example, we consider 0.00005 of the total cost of each transaction. This small amount is stored in the (BPSC) smart contract, and the total cost of transactions can be used for development, etc., or even burned to create negative inflation. Negative inflation is one of the most effective ways to maintain and strengthen the value of money as well as for cryptocurrencies.</p><p>We are now using the Flask framework so that we can communicate with the “smart blockchain” through requests based on the HTTP protocol platform. To do this, we create the following three methods:<br><strong><em>/transactions/new</em></strong><br>Request to build a new transaction and add it to a block<br><strong><em>/mine</em></strong><br>Ask our server to mine a new block<br><strong><em>/chain</em></strong><br>Request to return a complete list of blocks</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f80bc2ac11293cff4016a9037a003084/href">https://medium.com/media/f80bc2ac11293cff4016a9037a003084/href</a></iframe><p>According to the above code, the server will be launched on port 5000. Of course, another way is to import sys and make the following changes to the end codes. With this, you can always specify your desired port when running.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c43abc911f1ec622e84b665ae3e25e63/href">https://medium.com/media/c43abc911f1ec622e84b665ae3e25e63/href</a></iframe><p>Now we concatenate the two main parts of the code. Of course, you can <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain"><strong><em>download all the code for this article here</em></strong></a>, and at this point, our file name is <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/bpsc101.py"><strong><em>bpsc101.py</em></strong></a>. In the next step, we will run the server. We also use Postman to connect to the new API on the network.</p><p>We will start communicating with the new API by sending a POST transaction request to our server address:<br><strong><em>http://localhost:5000/transactions/new</em></strong><br>The information for this transaction must be entered in the Body section, and obviously, we select raw and then JSON.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UVCSepc4BkSTQdU6aIaOkA.jpeg" /></figure><p>In the example above, we wanted to send one million tokens (for example, Aka tokens) from our wallet to another person (with a specific address). To do this, we sent the tokens and the request for this transaction to the server (which acts as a Block Producer Smart Contract). As you see, the server responded that this transaction would be added to the second block. Because the Genesis Block has already been defined in the code.</p><p>We will now send a GET mining request to our server address: <strong><em>http://localhost:5000/mine</em></strong><br>This will create the next block. The content of each block includes all performed transactions and pending ones, for instance, here, the second block is created that contains only one transaction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_SXqoW76Yl59Fy0jh45LWA.jpeg" /></figure><p>According to the image above, the amount of fifty tokens for the transaction fee remains in the (BPSC) wallet, and the rest of the amount is sent to the final recipient by (BPSC). Of course, as mentioned before, the transaction fees will either be burned or spent on network development and so on. However, the type of spending these amounts has not been modeled yet. Now, We know that there must be two blocks in our chain’s server. To see the current chain, we send a GET chain request to our server address: <br><strong><em>http://localhost:5000/chain</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nuOeSbf89h2ZIVxTyAUDhQ.jpeg" /></figure><p>For further testing, you can send new requests for the transaction, and each time after a few transaction requests, you can send a mining request. Even, without having to do a transaction, you can mine an empty block. Meanwhile, the chain request determines the status of the chain for you at any time.</p><h3>The decentralized network for “smart blockchain”</h3><p>The “smart blockchain” network like Blockchain is decentralized. That’s why we need to implement a way for each node to be informed of its neighbor nodes on the network and keep a list of them. To do this, we use the set() so, each node recorded only once. We also create the following method to accept a list of new nodes in the form of URLs:<br><strong><em> /nodes/register</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9a3dc56bc8012b5d10aaab0794efabfc/href">https://medium.com/media/9a3dc56bc8012b5d10aaab0794efabfc/href</a></iframe><p>Remember that in “smart blockchain” only the block producer smart contract of (BPSC) can create or mine the new block. But, the rest nodes of the network can only send a request to (BPSC) and through this receive the last chain. So we need to implement the nodes request from (BPSC) to return the last chain. To do this, we create the following method:<br><strong><em>/smart/chain</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7bc4fcf34f0ec1b567b328273775e4b3/href">https://medium.com/media/7bc4fcf34f0ec1b567b328273775e4b3/href</a></iframe><p>As mentioned earlier, in this example, we want to launch our server (which acts as a block producer smart contract) on the port 5000. According to the above codes, any other nodes in the network can send a request to (BPSC) and receive the last chain. The command “requests” on line 26 do the same. In other words, with one request, the last chain created by (BPSC) will replace the node’s chain. To continue, we need to concatenate the pieces of code (up to this point). You can <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain"><strong><em>download all the code for this article here</em></strong></a>, and so far <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/bpsc102.py"><strong><em>bpsc102.py</em></strong></a> is the name of our file.</p><p>But, if we want to test the <strong><em>/nodes/register</em></strong> method as well as the <strong><em>/smart/chain</em></strong> method, we need to have other nodes in the network. This means that we either have to use several machines or define different ports on the same machine. We do the second one and create the other two nodes with the port numbers 5001 and 5002 on the same machine. But, the main problem is that except the (BPSC) which is responsible for making the transaction, mining the block, and creating a chain for specific tokens (such as Aka tokens), the rest of the nodes shouldn’t be able to do such activities. Of course, it is simple and we just need to remove some of the code from other nodes such as the ability of mining and…</p><p>To do this, we created the files: <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/nodes_v1_5001.py"><strong><em>nodes_v1_5001.py</em></strong></a> and the <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/nodes_v1_5002.py"><strong><em>nodes_v1_5002.py</em></strong></a>. Please see the details of these two files by clicking on them, as these modifications are not included in the article for avoiding context lengthier. In these files, compared to the original <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/bpsc102.py"><strong><em>bpsc102.py</em></strong></a>, only some of the code has been deleted and nothing has been added yet. Of course, the port number has also changed.</p><p>If you are using a VPN for any reason, you must turn it off during subsequent tests. Anyway, run all three files on separate consoles and create three nodes. This means that we will have the following nodes at the same time:<br><strong><em>http://localhost:5000</em></strong> <br>Which acts as a smart contract (BPSC).<br><strong><em>http://localhost:5001 </em></strong><br>One of the networks’ nodes that acts as a token sender.<br><strong><em>http://localhost:5002 </em></strong><br>One of the networks’ nodes that acts as a token receiver.</p><p>To test, we first send two consecutive POST requests so that the node with port number 5001 is notified of its neighbor’s nodes on the network and stores the list of these nodes:<br><strong><em>http://localhost:5001/nodes/register</em></strong></p><p>It means, first <strong><em>“nodes”: [“http://127.0.0.1:5000&quot;]</em></strong> as (BPSC) And then <strong><em>“nodes”: [“http://127.0.0.1:5002&quot;]</em></strong> as another node is stored in the mentioned list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IHS-TdJqXn5pDaWP1UhIPQ.jpeg" /></figure><p>Now we want to test the <strong><em>/smart/chain</em></strong> method. But before this test, we’d better send at least one transaction request and then a mining request to the (BPSC). This will save a chain with two blocks in (BPSC).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2mM5eXJfER98O30hUmQeQw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rll8FNO9k3AgsiOLnARy4g.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wq2tOT4toMN_M-RWEugfIw.jpeg" /></figure><p>Now we send the following request, which is a GET type, to the address of the node that has port number 5001:<br><strong><em>http://localhost:5001/smart/chain</em></strong><br>By doing this, the node with port 5001 will receive the last chain from (BPSC). Thus, the node’s chain replaced with the latest chain created by (BPSC).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vOykJNpBlhyGdZW64F1Ggw.jpeg" /></figure><p>So in “smart blockchain”, although none of the nodes can produce blockchain, all of them can make their transactions through (BPSC) and receive the last chain from (BPSC). Also, as a decentralized network, nodes can communicate with each other, and even network nodes might profit by the services of dozens of (BPSCs) because it is assumed that each (BPSC) is dedicated to one type of token.</p><h3>Dozens of “smart blockchains” on a decentralized network</h3><p>To make this seemingly complex subject, clear and transparent, we’d better implement a second chain for network nodes. Because by developing this pattern, you will be able to store dozens of parallel chains on each node. It’s simple, but it’s always good to drink more coffee. :)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1931bff879d6082d78ebe71e714a4c91/href">https://medium.com/media/1931bff879d6082d78ebe71e714a4c91/href</a></iframe><p>If you take a closer look at the above code, two parallel and independent blockchains are stored on one node. Of course, one of them is the previous “smart blockchain” and the other is a simple blockchain that has been added to the previous code to store personal data. Of course, the second blockchain could be a “smart blockchain,” and a node could use dozens of (BPSC) services in the same way. (For example, the transaction of different tokens stored in independent blockchains.)</p><p>To create the second chain, two lists have been added to the previous lists. Line 12 is for storing data and Line 14 is for storing blockchain. You can check the rest of the modification yourself. The <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/nodes_v2_5001.py"><strong><em>nodes_v2_5001.py</em></strong></a> and the <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/nodes_v2_5002.py"><strong><em>nodes_v2_5002.py</em></strong></a> files are created exactly according to the above code. Please run these two files as well as the <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/bpsc102.py"><strong><em>bpsc102.py</em></strong></a> file at the same time so that you can do the following tests.</p><p>Now, we want to test the <strong><em>/chain2</em></strong> method. The <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/nodes_v2_5001.py"><strong><em>nodes_v2_5001.py</em></strong></a> file represents a node that has port number 5001. We send a request for storing personal data to this node. Of course, we know that the initial block or Genesis Block is built at first, so the server responds that this data will be added to the second block. Then we send the mining request for the same node. This will create the next block (the second block on the second chain).<br><strong><em>http://localhost:5001/information/new</em></strong><br><strong><em>http://localhost:5001/mine</em></strong></p><p>We send the following request, which is of the GET type, to the same node: <br><strong><em>http://localhost:5001/chain2</em></strong><br>By doing this test, we make sure that the two blocks are in the second chain of our server, and the second chain has no relation with the first chain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oJQRfTXIQ2z5O-9kygL8tA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AkaLrKg3CU5CneBYvqEYZQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fVonR7scqy2JVY1XBQotjQ.jpeg" /></figure><p>To make sure that the second chain does not interfere with the functionality of the “smart blockchain”, we repeat exactly the previous tests for the “smart blockchain”. That is, we first test the <strong><em>/nodes/register</em></strong> method for all three nodes. Then, from the node that plays the role of (BPSC), we request several transactions and multiple mining:<br><strong><em>http://localhost:5000/transactions/new</em></strong><br><strong><em>http://localhost:5000/mine</em></strong></p><p>Finally, we send the following request, which is of GET type, to the address of the node that has port number 5001:<br><strong><em>http://localhost:5001/smart/chain</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Osx7k0VgTWcq61_KwSXGiA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*36sWON0VRSTlN-d2I0ISLg.jpeg" /></figure><p>As you can see in the pictures, the last chain created by (BPSC) substituted for the main chain (first chain) of this node. And finally, we conclude that both the “smart blockchain” and the second chain stored in this node completely independent.</p><h3>Planning and scheduling for mining and producing operations a block in a “smart blockchain”</h3><p>When a smart contract is registered onto the blockchain network, it becomes immortal. That is, as long as that blockchain network is working, no human (even the coder and owner of the smart contract) can make the slightest change in the initial content of the contract. Block producer smart contract (BPSC) has the same situation. So from the beginning, the time of mining operation (block production) should be clear and implemented in the codes. Otherwise, (BPSC) will always have to wait for the mining request to be able to mine the new block.</p><p>There are three ways to schedule a (BPSC) smart contract for mining operations(block producing):<br>A — based on time, build a new block, for example, every second or every minute.<br>B — based on the number of transactions; for example, after every ten transactions or one hundred transactions.<br>C — a combination of methods A and B; for example, ten seconds or ten transactions, each occurring sooner.</p><p>Of course, we only implement one example of the first method. For example, suppose that after each transaction, a new block is created by (BPSC). So each block contains only one transaction.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c7d74c84c1455787b0c0ee695f0fb1fe/href">https://medium.com/media/c7d74c84c1455787b0c0ee695f0fb1fe/href</a></iframe><p>As you see, the modifications have only been made in the <strong><em>/transactions/new</em></strong> method, and exactly the codes of <strong><em>/mine</em></strong> have been added to the end of that method. We created the <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/bpsc103.py"><strong><em>bpsc103.py</em></strong></a> file by applying the same changes. You can run this file and perform your desired tests using Postman.</p><p>In this case, if you send the following request to the server: <br><strong><em>http://localhost:5000/transactions/new</em></strong><br>(BPSC) immediately makes both the transaction and mining a block and adds it to the chain.</p><p>Now, if, for example, you decide to send the mining request to the server: <strong><em>http://localhost:5000/mine​ </em></strong><br>only an empty block will be minded because no transaction is expected.</p><h3>Summary and conclusion:</h3><p>To simulate the (BPSC) block producer smart contract, we considered a node with a port number 5000 and implemented the following three files:</p><p><a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/bpsc101.py"><strong><em>bpsc101.py</em></strong></a> This version is not final and cannot be interacted with other nodes.</p><p><a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/bpsc102.py"><strong><em>bpsc102.py</em></strong></a> This is the final version and all cases have been tested for this version. We tested the transaction request, the mining request, the request to send the last chain for the other nodes, and so on.</p><p><a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/bpsc103.py"><strong><em>bpsc103.py</em></strong></a> This version is also final, but in this version, we have added a new feature to the previous version. That is, (BPSC) required to create a new block after each transaction and add that block to its chain.</p><p>The ports 5001 and 5002 have the representation of other network nodes. To simulate these nodes, we implemented the following two versions:</p><p><a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/nodes_v1_5001.py"><strong><em>nodes_v1_5001.py</em></strong></a> and <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/nodes_v1_5002.py"><strong><em>nodes_v1_5002.py</em></strong></a><br>This version is according to “Smart Blockchain” technology and final. Although these nodes cannot produce blocks, they can do their transactions through (BPSC) and also receive the last chain from (BPSC).</p><p><a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/nodes_v2_5001.py"><strong><em>nodes_v2_5001.py</em></strong></a> and <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain/blob/master/nodes_v2_5002.py"><strong><em>nodes_v2_5002.py</em></strong></a><br>This is also final, but in this version, we have added a new feature to the previous version. It means a simple blockchain (To store personal data) has been added to the previous code.</p><p>The coding is over and <a href="https://github.com/SomayyehGholami/Implementing-Smart-Blockchain"><strong><em>you can download all the article files here</em></strong></a>. You can also do more tests through separate machines or with friends. But bearing in mind that Python simulation, whether for blockchain or “smart blockchain,” usually doesn’t include all the necessary controls and operations. In this article, some controls were not performed; for example, the necessary controls for wallets were not performed. We also want to return all tokens to the sender if, for any reason, (BPSC) fails to send the tokens to the final recipient. But it’s not implemented into codes either. This kind of control should not be neglected in real work. Most smart contracts are written in specific languages, so-called high-level. For example, smart contracts in Ethereum blockchain are usually written in Solidity language. Fortunately, implementing this type of control with Solidity is even easier than Python.</p><p>If you are interested, you can also read the following article. We have implemented exactly one “Smart Blockchain” with Solidity language:</p><p><a href="https://hackernoon.com/how-to-build-a-smart-blockchain-that-prevents-double-spending-a-step-by-step-guide-vw9m33aq">https://hackernoon.com/how-to-build-a-smart-blockchain-that-prevents-double-spending-a-step-by-step-guide-vw9m33aq</a></p><h3>The last word</h3><p>In this paper and several previous ones, we have shown that miners, block producers, and their activities are not an integral part of blockchain technology. Of course, it is a new and controversial idea to replace miners and block producers with smart contracts, and like any big work, there are many objections and obstacles in the way, but maintaining the status quo is not possible in the long run. For example, by increasing the volume of calculations in machine learning algorithms, we are witnessing an increase in the processing capacity of the machine at the same time. But in practice, the proof of work mechanism (POW) for Bitcoin is a real impasse. All miners are increasing their processing capacity, but to compete with each other, they still have to consume the amount of electrical power that equals the annual consumption of a country with ten million population. This is not fair.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PsAvTyoWKE2mYyqrhtymvg.jpeg" /><figcaption><a href="https://cbeci.org/">Cambridge Bitcoin Electricity Consumption Index</a></figcaption></figure><p>Anyway, if you are interested in the subject, you can read our previous articles on “smart blockchain” as well as bitcoin problems.<br>Also, if you have any questions or you see any ambiguities or errors in this article, please let us know.</p><p><a href="https://medium.com/@somayyeh_43305"><em>Somayyeh Gholami</em></a><br><a href="https://www.mehrankazeminia.com/"><em>Mehran Kazeminia</em></a></p><h4>A note from Python In Plain English</h4><p>We are always interested in helping to promote quality content. If you have an article that you would like to submit to any of our publications, send us an email at <a href="/service/mailto:submissions@plainenglish.io"><strong>submissions@plainenglish.io</strong></a><strong> </strong>with your Medium username and we will get you added as a writer.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3183e0c1052e" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/implementing-a-smart-blockchain-with-python-3183e0c1052e">Build a “Smart Blockchain” in Python</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Storing Personal digital records; A blessing or a curse]]></title>
            <link>https://medium.com/@somayyeh_43305/storing-personal-digital-records-a-blessing-or-a-curse-f4885df70c29?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4885df70c29</guid>
            <category><![CDATA[soliset]]></category>
            <category><![CDATA[cambridge-analytica]]></category>
            <category><![CDATA[digital-record]]></category>
            <category><![CDATA[smart-blockchain]]></category>
            <category><![CDATA[decentralized-network]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Mon, 15 Jul 2019 09:31:32 GMT</pubDate>
            <atom:updated>2019-07-15T09:31:32.948Z</atom:updated>
            <content:encoded><![CDATA[<h4>Storing personal digital records in decentralized networks by the help of smart contracts</h4><p>Recently we see the launch of different decentralized networks. The decentralized networks do not have the main server and work peer to peer. Meanwhile, we know that in a centralized social network such as Facebook, the big companies hold the title to the main servers which store the users’ digital records and exploit or potentially exploit this information. But this is different for a decentralized social network because the digital records of personal behavior don’t save anywhere, and abuse is also excluded. For this reason, this feature commonly referred to as an advantage of the decentralized social network, but in this article, we will show that this approach only serves to gloss over the issue, and should not continue to turn a blind eye to such valuable information forever. In other words, we hope that with the help of smart contracts and artificial intelligence, users will soon be able to use and manage their own digital records in decentralized networks without the involvement of big companies.</p><p>The authors of this article are <a href="https://www.soliset.com/">Somayyeh Gholami</a> and <a href="https://www.mehrankazeminia.com/">Mehran Kazeminia</a>. It’s published in English and Farsi In July 2019.</p><p><a href="https://www.soliset.com/digital-records2"><em>Persian Article</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nieVZC3mml33K8Ds2n1xDg.jpeg" /></figure><h3>The importance of digital records</h3><p>After World War II, Oil had an important role to play in the industrial section progress. Every day, the various industries and vehicles demands for oil have been increasing directly and indirectly, and oil was called black gold. Many countries that owned oil reserves were long unaware of the importance and value of oil and had conceded the right to explore and exploit their reserves at the lowest prices. The people and governments of these countries usually referred to oil as black, smelly, and worthless substance, which easily burns. But when they gradually realized the importance and real value of oil, they increased the price of oil sales, and even in many cases they canceled the previous agreements and considered them cruel.</p><p>From different angles, the digital records of individual behavior are like oil and even given the title of digital gold. Until twenty years ago, it was just conceivable for a few people to earn billions of dollars through collecting and storing insignificant information (in the form of digital records of individual behavior). Some have moved forward step by step in this direction, and their achievements were billion-dollar companies for themselves, which also have increased prosperity in the lives of the people of the world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qAIGtl-wwpqiSMzK_cqr3A.jpeg" /></figure><p>Companies like Facebook and Google have earned $ billions from online advertising. The initiative, creativity, and innovation of their managers and teams were the most important factor in such successes. Of course, the feed and fuel of this process are the online trackings of web users or digital records of individual behavior which are gradually and constantly collected and stored by these companies. In other words, in many cases, there were no today’s successes without accessing and using digital records of individual behavior. Let’s remind that the use of artificial intelligence is based only on information and details.</p><h3>How are digital records of human behavior used?</h3><p>Many efforts have been made to predict the private traits and attributes of individuals through access to their digital records. There are also numerous research papers and articles on this subject. For example, in 2012, in the <a href="https://www.pnas.org/content/110/15/5802">PNAS article</a>, by using a statistical sample of fifty eight thousand, demonstrated that digital records accessible of Facebook can predict automatically and accurately a wide range of personal characteristics, such as sexual orientation, ethnicity, religious and political perspectives, personality traits, intelligence, happiness, use of addictive substances, parental separation, age, and gender. Meanwhile, Cambridge Analytica scandal in 2018 was in this context, and you probably remember, in 2018, the Facebook CEO Mark Zuckerberg who gave testimony to Senate committees about Facebook’s data privacy practices in the wake of the company’s Cambridge Analytica scandal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AxJX8JPlabF6FGCVoEDT1A.jpeg" /><figcaption><a href="http://www.laloyolan.com/news/facebook-s-cambridge-analytica-data-scandal/article_702b3c7a-f972-539c-a7f8-f202fd478634.html">Facebook’s Cambridge Analytica data scandal</a></figcaption></figure><p>Other efforts have been made for transparency, such as <a href="https://dataselfie.it/#/">Data Selfie</a>, which is an extension of the Chrome browser, to help people figure out how much Facebook knows about them. By downloading Data Selfie, this software tracks people’s activity just like Facebook but the goal of Data Selfie is to show how much users’ information is traceable without being informed them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*FiEL6k5srxvMWjTyPI96Sw.jpeg" /></figure><p>It’s quite clear that companies like Facebook, Google, and … by collecting more and more users’ information, they will earn more. On the contrary, if we are optimistic and do not conceive of any further misuse in the future, Maybe these circumstances are not bad for users. For example, users can more easily find their favorite books or other similar items.</p><h3>What is the necessity for storing personal digital records?</h3><p>Now, the digital records of individual behavior of users in the decentralized network aren’t saved anywhere, and in centralized networks, users have no access to manage it and practically, there is no way to personal utilization of this available information. For example, people can never modify their information or develop and integrate it, or they cannot provide this information to other advertising companies, or their counselor or psychiatrist. Meanwhile, storing, processing and analyzing such information with the help of artificial intelligence can have other achievements in personal form, but for the time being, everything is in possession and monopoly of the companies who collect and process this information. But it is not fair and rational because people of the society with no access to their own data have only the too much-limited share of these achievements.</p><p>Meanwhile, if a specific mechanism enables each person to collect her/his digital records throughout her/his life and manage its data, it would be a great treasure for everyone. At the same time, by using artificial intelligence and personal bots, more achievements can be gained for society. In this case, bots and crawlers of Google, Facebook or other companies would be replaced by personal bots or at least forced to collaborate and work with personal bots.</p><p>It’s said that only by considering 10 Likes on Facebook, It can predict personality and behavior of individual better than their colleagues, 70 Likes better than neighbors, 150 Likes better than parents, and 300 Likes even better than a spouse. Let’s imagine by using the highest achievements of artificial intelligence how the personal robot could be useful and close to humans which reach all digital records of an individual throughout his/her life. These robots identify personality traits deeply and with a wealth of accurate information, can serve people.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*INJm2zWz4A00F5by7htBGQ.jpeg" /></figure><p>Perhaps the ambiguity is that we know Facebook, Google and others do not recognize their users and store users’ digital records to help to know them more. But why should individual keep their own digital records when they can express their own opinions, tastes, and needs about any subject?</p><p>First of all, people are not precise in describing themselves and their taste, and they can not convey everything in detail to the machine. Secondly, human memory is very limited. For example, digital records can save all the Internet addresses that a person visited during life with details such as the visit duration, but the memory of people does not have such the power to do so. Thirdly, artificial intelligence needs widespread databases and details for making the right decision. As much as information and details grow, the performance of artificial intelligence increased. The remarkable growth and development of artificial intelligence in recent years has directly impacted by the rise of databases and details.</p><p>That’s why Facebook does not need a special questionnaire to choose an advertisement or similar subject to show for a specific user, already knows a user what might be looking for. Of course, the digital records of personal behavior is the fuel for Artificial intelligence of Facebook. Most of the time, the questionnaire for each subject is very massive and certainly not feasible.</p><h3>Utilization of smart contracts to store digital records</h3><p>The decentralized networks will soon have a significant share of the Internet and, of course, in the absence of the central main server, the digital records of individual’s behavior are not stored in these networks. But as you might have noticed, this information is the essential fuel of Artificial intelligence and the storage of them can’t simply be ignored. If there is any concern about misuse, ignoring the issue is not the solution, but rather the mechanism for storing and processing digital records of individuals in decentralized networks must be completely personal. In this way, it is practically prevented from potential misuse of companies while users themselves must be able to decide for their own digital records. It is fair and rational that users possess and manage this information in decentralized networks (and even centralized networks).</p><p>Fortunately, the concept of smart contracts can be the solution to the problem. In this way, all the necessary information is automatically sent to one or more personal smart contracts which automatically store encrypted data for the use of personal robots. We’ve already done a similar thing in <a href="https://www.newchains.info/">Smart Blockchain</a>, and finally, in this technology, smart contracts have replaced miners and block producers.</p><p>Anyway, certainly, the possible abuse of individual digital record must be prevented with the help of smart contracts or any other method in decentralized networks. Meanwhile, the mechanism of storing individual digital records in centralized networks is different and possibly easier. Because, there’s some kind of experience about it, and big companies like Google and Facebook take advantage of doing it for years.</p><p><a href="https://www.soliset.com/">Somayyeh Gholami</a><br><a href="https://www.mehrankazeminia.com/">Mehran Kazeminia</a></p><p><a href="https://www.soliset.com/">Website</a> | <a href="https://twitter.com/SetSoli">Twitter</a> | <a href="https://t.me/soli_set">Telegram</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4885df70c29" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Billions dollars cost of electricity consumption of bitcoin mining, really is paying from what…]]></title>
            <link>https://medium.com/@somayyeh_43305/billions-dollars-cost-of-electricity-consumption-of-bitcoin-mining-really-is-paying-from-what-36811c7b7148?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/36811c7b7148</guid>
            <category><![CDATA[soliset]]></category>
            <category><![CDATA[smart-blockchain]]></category>
            <category><![CDATA[eco-friendly]]></category>
            <category><![CDATA[climate-change]]></category>
            <category><![CDATA[bitcoin-mining]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Thu, 11 Jul 2019 16:04:57 GMT</pubDate>
            <atom:updated>2019-07-12T03:57:09.061Z</atom:updated>
            <content:encoded><![CDATA[<h3>Billions dollars cost of electricity consumption of bitcoin mining, really is paying from what resource?</h3><p>The electricity consumption of Bitcoin mining is really staggering. <a href="https://cbeci.org/">Cambridge University has recently launched an online system provides a real-time estimate of the total electricity consumption of the Bitcoin network</a> and updated every 30 seconds. Usually, the yearly average of power consumption for all mining equipment is estimated to be between 50 to 70 terawatt-hours(TWh) per year. This figure exceeds the annual consumption of a country with ten million populations and costs billions of dollars. In this short article will show such significant cost is covering from what resource.</p><p>The authors of the article are <a href="https://www.soliset.com/">Somayyeh Gholami</a> and <a href="https://www.mehrankazeminia.com/">Mehran Kazeminia</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eecE7z5vLkuoZy0jWCXk8w.jpeg" /><figcaption><a href="https://cbeci.org/">Cambridge Bitcoin Electricity Consumption Index (CBECI)</a></figcaption></figure><p>Suppose a new currency was introduced and sold to the people by a financial institution who announced that users can buy and sell it safely and limitlessly. This financial institution will not receive any payments such as a commission nor fee from users who can trade this money anonymously, easily and free of charge. Meanwhile, using special technology, the reliability of the accuracy of each transaction is 100%. There is neither mistake nor misconduct nor corruption in this financial institution.</p><blockquote>Their only condition is to print new currency at least an annual equivalent of $ 3 billion, in addition to the available amount and to sell to the public to cover its costs.</blockquote><p>The condition of the financial institution ensures that it receives $ 3 billion annually, and will no longer need to receive any other payment, like the transaction fees. That’s exactly why it can always provide its services for free. Meanwhile, let’s not forget that the injection of new money definitely have inflation effects and indirectly all holder of this currency will be paying this cost. In other words, $ 3 billion annually will be reduced from the value of this currency in hand of holders to cover the costs of this financial institution.</p><p>Bitcoin’s decentralized network works just like the financial institution of the above example, makes the new Bitcoins in the form of Bitcoin mining, and immediately gives new Bitcoins to the winner miners. Of course, the transaction fees cover only five percent of the cost of the miners and whether or not it has little impact. So, it should not be surprised at the low transaction fee of Bitcoin transactions or considered it as an advantage of Bitcoin. We have already discussed these issues in a different article and you can learn more here:</p><p><a href="https://medium.com/swlh/bitcoin-mining-big-little-lies-c6911435aa77">Bitcoin mining; Big little lies</a></p><p>One new block is added to the Blockchain about every ten minutes, and immediately the reward plus all transaction fees of the same block is paid to the winner miner. It means that around 52560 blocks are registered annually, considering that the miner’s reward is currently 12.5 Bitcoins this year, about 657,000 new Bitcoins have been generated. If we assume that this year the average price of Bitcoin is $ 5,500 so just this year $ 3.6 billion have been paid to miners. Of course, as mentioned earlier, miners also receive an additional payment as the transaction fees which usually is less than five percent of the miner’s reward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*uEaoz0TWlpvlAaHFIj_Bkw.jpeg" /></figure><p>If we skimmed through Bitcoin daily transactions, we quickly find that in one day, we rarely see Bitcoin transactions worth about $ 1 billion or more. For this reason, we can presume currently the sum of Bitcoin transactions worth about $ 600 million per day, on average. So, in one year $ 3.6 billion is spent just to transfer $ 216 billion. In other words, the actual transaction fee of Bitcoin should be about 0.0167 of the transaction amount.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LhlZwVNJkUY44ls10Ou6yA.jpeg" /></figure><p>The staggering electricity bills of miners is not only about money but <a href="https://www.nature.com/articles/s41558-018-0321-8.epdf?referrer_access_token=h1fcePQqnrntGZtW4POj5dRgN0jAjWel9jnR3ZoTv0N_BlBOQJDHfvssdGkz36RF4-C7V0a3_EVtpCvGlNw-EnbTe8aHj0G6zUia-Ph07D9stGSwVr33awLgvAeF6wJMemMg8v0fJW4Fv4fkvd4UVqUc8AUCraiFlL0wQQpzHXejfnfIUa7cgU5GVf1-m6vebGayFvjf-pOpjCNuCq4tFepjgz_m7haDpjlNlw-l34JwomIaiMXK1GzxOQL8y37Vhl3dzb0oAHggs8BswICMcLAdnfLZbGezQTIfriWLRys%3D">the study, which was carried out in October 2018 on global climate change</a>, showed that in less than three decades, Bitcoin mining could increase global temperatures more than 2 degrees Celsius.</p><p>Unfortunately, in Blockchain society, the main focus is only on possible solutions for consensus, and the types of proof-of-work mechanisms, proof-of-stake, the delegated proof of stake, etc. are proposed with such mentality. But if we concentrated on the main problem, we will have much more affordable and cheaper solutions. There is no need to create multiple alternatives from a block to choose the correct one by spending a lot of energy and using the types of consensus mechanism. Rather, we need to get the flawless data that can be transmitted without interruptions when transactions are made. There may be several ways to do this, but the most tangible way is to use a smart contract. We have named this method “<a href="https://www.newchains.info/">Smart Blockchain</a>“ for this reason.</p><p><a href="https://hackernoon.com/what-is-smart-blockchain-4b134275e90f">What is Smart Blockchain?</a></p><p>Smart Blockchain needs neither a financial institution nor intermediary, meanwhile, there will be no task to do for miners and block producers in this technology. In Smart Blockchain, one or more smart contracts collect accurate and flawless data from the successful transactions of a decentralized network at the moment of transaction occurrence and simultaneously store and register these data in new blocks like an accurate and trust financial institution but free of charge and at all time.</p><p><a href="https://www.soliset.com/">Somayyeh Gholami</a><br><a href="https://www.mehrankazeminia.com/">Mehran Kazeminia</a></p><p><a href="https://www.soliset.com/">Website</a> | <a href="https://twitter.com/SetSoli">Twitter</a> | <a href="https://t.me/soli_set">Telegram</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36811c7b7148" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Write and read “Smart Contract” but it`s an “Automated Contract”]]></title>
            <link>https://medium.com/hackernoon/write-and-read-smart-contract-but-it-s-an-automated-contract-c2325d634a77?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/c2325d634a77</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[smart-contracts]]></category>
            <category><![CDATA[automated-contract]]></category>
            <category><![CDATA[solidity]]></category>
            <category><![CDATA[soliset]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Tue, 25 Jun 2019 07:18:15 GMT</pubDate>
            <atom:updated>2019-06-26T01:05:03.229Z</atom:updated>
            <content:encoded><![CDATA[<h3>Write and read “Smart Contract” but it’s an “Automated Contract”</h3><p>The biggest achievement of smart contracts is that with the help of blockchain technology, they can only run independently their initial code. Since there is no possibility of human interference and tampering with functionalities or transaction results. But anyway, these contracts do not necessarily have the concept of intelligence or smart.</p><p>The authors of the article are Somayyeh Gholami and Mehran Kazeminia</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*34eT3sjD63dcMogH7XJ_lw.jpeg" /><figcaption>Smart Contract Vs. Automated Contract</figcaption></figure><p>The reputation as a SMART is not proportional to the reality of these contracts and is not a proper name. The best name for them may be is the automated contract because the contracts are not necessarily intelligent or smart but certainly, they are automated!</p><p>These contracts, in accordance with the initial codes, work automatically after registering and deploying into a blockchain, and no human, even the owner(s) or developer(s) of contract, can make the slightest change in its initial code or functionality. This is perhaps the main feature of these contracts, and to convey this concept, using the title “automated contract” is appropriate because smartness or intelligence is not the main criterion for these contracts.</p><h3>Smart inventors, automated inventions</h3><p>Mr. Nick Szabo, in the late nineties, referred to the concept of “smart contract”. You can read his conclusions on <a href="https://firstmonday.org/ojs/index.php/fm/article/view/548/469"><em>this subject in his 1997 article</em></a>. Meanwhile, smart contract developers could not begin their work until 2014 (till the Ethereum network began working in 2014). Because these contracts work with blockchain technology. Of course, after Ethereum, other blockchains were also launched to develop smart contracts.</p><p>Smart contracts are computer programs that simulate conventional or classical contract. They can exchange their own cryptocurrency or a special token. In addition, for each contract, an account or wallet is defined which called “contract wallet”. Parts of its coding are the conditions and how to exchange between the contract wallet and users (or between the contract wallet and the other contracts wallet).</p><p>Smart contracts can be registered on some blockchain networks. These contracts will be immortal after launch. It means, as long as that blockchain works, the original smart contract, as well as all transactions and exchanges in full details, continuously and permanently are stored and maintained on the blockchain network. These contracts work for all time, just according to the original contract code, and no human, even owner or developer can make the slightest change in the code of the program.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tYwKLyd-abrhuU45Wwcs1g.jpeg" /><figcaption>Nick Szabo</figcaption></figure><p>People like Nick Szabo or Vitalik Buterin are surely smart people! But their research and study achievement, which is an automated contract on the blockchain platform, is not always smart. For this reason, the title of “smart contract” is not proportional to the reality of these contracts. However, if we can not or do not want to change the name of these contracts, at least we gain the right insight into these contracts.</p><p>In many cases, decentralized networks are more restricted than centralized networks. For example, the random function does not exist in Solidity language, because the EVM Ethereum virtual machine requires total determinism within the smart contracts But all these technical reasons do not stop the efforts to make intelligence these contracts in necessary cases. Even developers are working on blockchain and Artificial Intelligence technology to serve each other and have some achievements, or …… However, it should be noted that today and ever, many of the contracts that have been registered on blockchain networks and deployed, they are not smart or intelligent and may not need to be either! That’s why the title of “smart contract” should not make developers and users mistaken.</p><h3>Smart Contracts that Are Not Smart!</h3><p>After the launching of the Ethereum blockchain, the developers were really excited to create their tokens only by writing a smart contract and without a dedicated blockchain network. The Ethereum network supports the transactions of these tokens, and everything was ready for a great adventure which was the Initial Coin Offering (ICO). For this reason, the first and perhaps the largest number of smart contracts were written for producing tokens and the launch of the ICO.</p><p>An example of smart contracts that is not smart! Maybe, the <a href="https://theethereum.wiki/w/index.php/ERC20_Token_Standard"><em>ERC-20 </em></a><em>standard which</em> has been used by many companies since late 2015 to generate tokens on the Ethereum network. This standard includes six main functions:</p><p><em>totalSupply() public constant returns</em></p><p><em>balanceOf(address tokenOwner) public constant returns</em></p><p><em>allowance(address tokenOwner, address spender) public constant returns</em></p><p><em>transfer(address to, uint tokens) public returns</em></p><p><em>approve(address spender, uint tokens) public returns</em></p><p><em>transferFrom(address from, address to, uint tokens) public returns</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HJ6Agm_JNtXGCT70inUojg.jpeg" /><figcaption>ERC-20 &gt;&gt; Token standard</figcaption></figure><p>If you are familiar with <a href="https://solidity.readthedocs.io/en/v0.4.25/"><em>Solidity programming language</em></a> and its functions, you also confirm that the above standard cannot be a structure for the smartness or intelligence of the token generator contract. Rather, functions are deterministic and nonsmart. Of course, if we want to assume a smart contract as smartness, we have to call “smart” all or most of the computer programs!</p><p>Many of the smart contracts only provide method bodies for all above functions and do not have a specific smart feature, but they are all automated, and after registering and deploying them, they continue their activities forever according to the initial code and without involving any human factors.</p><p>We end this discussion here and we’re eager to receive your comments.</p><p><a href="https://www.soliset.com/">Website</a> | <a href="https://twitter.com/SetSoli">Twitter</a> | <a href="https://t.me/soli_set">Telegram</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fdde502%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fhackernoon%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c2325d634a77" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/write-and-read-smart-contract-but-it-s-an-automated-contract-c2325d634a77">Write and read “Smart Contract” but it`s an “Automated Contract”</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The transfer of over one billion dollars by using two Bitcoin’s transactions; How much really did…]]></title>
            <link>https://medium.com/hackernoon/the-transfer-of-over-one-billion-dollars-by-using-two-bitcoins-transactions-how-much-really-did-2a3905deb7ee?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a3905deb7ee</guid>
            <category><![CDATA[whales]]></category>
            <category><![CDATA[soliset]]></category>
            <category><![CDATA[environmentalism]]></category>
            <category><![CDATA[bitcoin-freak]]></category>
            <category><![CDATA[smart-blockchain]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Tue, 25 Jun 2019 07:09:06 GMT</pubDate>
            <atom:updated>2019-06-26T01:01:34.326Z</atom:updated>
            <content:encoded><![CDATA[<h3>The transfer of over one billion dollars by using two Bitcoin’s transactions; How much really did it cost? <br>$ 120 or $ 20 million?</h3><p>In this short article, we will show that despite it appears Bitcoin’s transaction fees are negligible, more than ninety-five percent of the cost of power consumed by miners, as well as miners’ remuneration, is not provided by the sender or transaction fee but winner miner’s reward, or the so-called Bitcoin mining. For this reason, to pay or not to pay the transaction fee has almost no effect on the subject. Of course, the consequence of paying the winner miner’s reward is specified exclusively by inflation, and unfortunately, this is not so tangible for most people.</p><p>From the very beginning, if either sender or receiver paid directly the total cost of the miner as a transaction fee and there was neither winner miner’s reward nor Bitcoin mining. At least, the inefficiency of the proof mechanism, as well as the staggering consumption of electricity in the decentralized network of Bitcoin, would become obvious for everyone, and more quickly a solution would be found.</p><p>The authors of this article are Somayyeh Gholami and Mehran Kazeminia. It’s published in English and Farsi In June 2019.</p><p><a href="https://www.soliset.com/bitcoin2"><em>Persian Article</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*OvI9f0OMGQflrJmxFgPa9Q.jpeg" /></figure><h3>Transferring over 1 billion dollars; <br>Block #578327 and Block #578328</h3><p>In the morning of 29th May 2019, two Bitcoin transactions, which totaled $ 1.2 billion, were recorded on blocks 578327 and 578328 of the bitcoin network. At the same time, the tweeter account of Whale_alert, which announces large crypto transactions from and to exchanges, tweeted that Binance (Cryptocurrency exchange) transferred a total of 145980 Bitcoin into an unknown wallet. The first transaction was approximately 77410 Bitcoin with a transaction fee of $ 19, and the second transaction was about 68570 Bitcoin with a transaction fee of $ 105. It means a total of $ 1.2 billion transferred with a transaction fee of $ 124 on the same day.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gHpP8h6YnNkxV00O5AxxYA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lg-1sWdNf6P72XrG8foowQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pHOpn94nXlYt8LrpfDrirQ.jpeg" /></figure><p>Bitcoin freak and supporters high praise these large transactions with low cost while conveying this as an example of high-profile and correct Bitcoin application. But the hidden costs of the Bitcoin network are many, and it should not forget the fundamental weaknesses of bitcoin transactions. In the next section, let’s make this clearer.</p><h3>$ 120 or $ 20 million; that is the question.</h3><p>Suppose a new currency was introduced and sold to the people by a financial institution who announced that users can buy and sell it safely and limitlessly. This financial institution will not receive any payments such as a commission nor fee from users who can trade this money anonymously, easily and free of charge. Meanwhile, using special technology, the reliability of the accuracy of each transaction is 100%. There is neither mistake nor misconduct nor corruption in this financial institution.</p><blockquote>Their only condition is to print new currency at least an annual equivalent of $ 3 billion, in addition to the available amount and to sell to the public to cover its costs.</blockquote><p>The condition of the financial institution ensures that it receives $ 3 billion annually, and will no longer need to receive any other payment, like the transaction fees. That’s exactly why it can always provide its services for free. Meanwhile, let’s not forget that the injection of new money definitely have inflation effects and indirectly all holder of this currency will be paying this cost. In other words, $ 3 billion annually will be reduced from the value of this currency in hand of holders to cover the costs of this financial institution.</p><p>Bitcoin’s decentralized network works just like the financial institution of the above example, makes the new Bitcoins in the form of Bitcoin mining, and immediately gives new Bitcoins to the winner miners. Of course, the transaction fees cover only five percent of the cost of the miners and whether or not it has little impact. So, it should not be surprised at the low transaction fee of Bitcoin transactions or considered it as an advantage of Bitcoin.</p><blockquote>Never forget that Bitcoin transactions are one of the most expensive and with the highest energy consumption in the world because the electricity consumption of each Bitcoin transaction is thousands of times more than a regular visa card transaction. Unfortunately, the annual power consumption of miners exceeds 50 terawatts per hour.</blockquote><p>We have already discussed these issues in a different article and you can learn more here:</p><p><a href="https://medium.com/swlh/bitcoin-mining-big-little-lies-c6911435aa77">Bitcoin mining; Big little lies</a></p><p>As we explained in the above article; Round-the-clock, the miners are trying to get a reward as a winner miner in competition with other miners and by using enormous electricity consumption. Every ten minutes, a miner is selected and the new competition begins immediately.</p><p>The mathematical calculations of the miners don’t solve any real problem; they haven’t any creativity and do not help mathematics. In these calculations, only massive random numbers are added (combined) one by one to a particular type of math function (hash of block contents) and the result of this combination is calculated for each random number. The winner miner finds out the correct random number faster than the others. A random number is the correct one that in combination with this mathematical function produces a certain result.</p><blockquote>Finding this correct random number is very similar to play a slot machine. A person inserts cash or ticket into a machine and keeps on playing to win. Either inserting more cash or ticket and playing more help winning chance increases and the same way, in mining operation, adding new mining machine and consuming more electrical power let the chance of winning increases.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*uEaoz0TWlpvlAaHFIj_Bkw.jpeg" /></figure><p>One new block is added to the Blockchain about every ten minutes, and immediately the reward plus all transaction fees of the same block is paid to the winner miner. It means that around 52560 blocks are registered annually, considering that the miner’s reward is currently 12.5 Bitcoins this year, about 657,000 new Bitcoins have been generated. If we assume that this year the average price of Bitcoin is $ 5,500 so just this year $ 3.6 billion have been paid to miners. Of course, as mentioned earlier, miners also receive an additional payment as the transaction fees which usually is less than five percent of the miner’s reward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LhlZwVNJkUY44ls10Ou6yA.jpeg" /></figure><p>Now, we want to answer the main question: it means that we want to find out the actual cost of the transactions of $ 1.2 billion on May 29th. On the face of it, only $ 124 was received from the Bitcoin sender as a fee but as we explained earlier this is just the appearance, and the cost of the miners is far more than it.</p><p>Generally, the fair deal for financial transactions is that the sender or receiver pays the costs which are in proportion to the amount of money of each transaction, to the financial institution. Considering that this year, the total cost of all transactions in Bitcoin decentralized network was $ 3.6 billion, it’s reasonable to pay the total cost of a transaction according to its sum of Bitcoin.</p><p>If we skimmed through Bitcoin daily transactions, we quickly find that in one day, we rarely see Bitcoin transactions worth about $ 1 billion or more. For this reason, we can presume currently the sum of Bitcoin transactions worth about $ 600 million per day, on average. So, in one year $ 3.6 billion is spent just to transfer $ 216 billion. In other words, the actual transaction fee of Bitcoin should be about 0.0167 of the transaction amount.</p><blockquote>So, finally, we can conclude our example that the real transaction fee of $ 1.2 billion on May 29th should be at least $ 20 million. But this amount is currently not paid by Bitcoin sender and receiver, but due to inflationary effects, it is indirectly paid by all Bitcoin holders.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jXyo0KDNh5wEpW-U-uV40Q.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wrt7TPUTFlmXbLJEvqFxsQ.jpeg" /></figure><p>The staggering electricity bills of miners is not only about money but <a href="https://www.nature.com/articles/s41558-018-0321-8.epdf?referrer_access_token=h1fcePQqnrntGZtW4POj5dRgN0jAjWel9jnR3ZoTv0N_BlBOQJDHfvssdGkz36RF4-C7V0a3_EVtpCvGlNw-EnbTe8aHj0G6zUia-Ph07D9stGSwVr33awLgvAeF6wJMemMg8v0fJW4Fv4fkvd4UVqUc8AUCraiFlL0wQQpzHXejfnfIUa7cgU5GVf1-m6vebGayFvjf-pOpjCNuCq4tFepjgz_m7haDpjlNlw-l34JwomIaiMXK1GzxOQL8y37Vhl3dzb0oAHggs8BswICMcLAdnfLZbGezQTIfriWLRys%3D">the study, which was carried out in October 2018 on global climate change</a>, showed that in less than three decades, Bitcoin mining could increase global temperatures more than 2 degrees Celsius.</p><blockquote>Let’s not forget that Bitcoin is the world’s first Blockchain network and until today remains the most popular cryptocurrency in the world and should be preserved. Bitcoin’s creator ten years ago proposed the POW(Proof-of-work)mechanism which was a great idea to launch the first decentralized network (as a pilot project). But this method for long life needs to be corrected and its current state cannot always continue.</blockquote><p>We have addressed these issues in the different perspectives and articles and in order to solve them, we have proposed a new generation of Blockchain network. In our approach, smart contracts replace the miners and block producers so we’ve named it from the very beginning Smart Blockchain. You can learn more about the fundamentals of this technology by studying the following articles:</p><ul><li><a href="https://hackernoon.com/what-is-smart-blockchain-4b134275e90f">.css-k31kfd-skeletonStyles-Skeleton{background-color:#eee;background-image:linear-gradient( 90deg,#eee,#f5f5f5,#eee );background-size:200px 100%;background-repeat:no-repeat;border-radius:4px;display:inline-block;line-height:1;width:100%;-webkit-animation:animation-16jpnkj 1.2s ease-in-out infinite;animation:animation-16jpnkj 1.2s ease-in-out infinite;}@-webkit-keyframes animation-16jpnkj{0%{background-position:-200px 0;}100%{background-position:calc(200px + 100%) 0;}}@keyframes animation-16jpnkj{0%{background-position:-200px 0;}100%{background-position:calc(200px + 100%) 0;}}‌.css-k31kfd-skeletonStyles-Skeleton{background-color:#eee;background-image:linear-gradient( 90deg,#eee,#f5f5f5,#eee );background-size:200px 100%;background-repeat:no-repeat;border-radius:4px;display:inline-block;line-height:1;width:100%;-webkit-animation:animation-16jpnkj 1.2s ease-in-out infinite;animation:animation-16jpnkj 1.2s ease-in-out infinite;}@-webkit-keyframes animation-16jpnkj{0%{background-position:-200px 0;}100%{background-position:calc(200px + 100%) 0;}}@keyframes animation-16jpnkj{0%{background-position:-200px 0;}100%{background-position:calc(200px + 100%) 0;}}‌</a></li><li><a href="https://medium.com/swlh/smart-blockchain-the-rescue-of-bitcoin-and-saving-electricity-more-than-50twh-per-year-c17606976fce">Smart Blockchain; The rescue of Bitcoin and saving electricity more than 50TWh per year</a></li><li><a href="https://medium.com/swlh/the-elimination-of-miners-and-block-producers-by-using-smart-contracts-58b20e317e9b">The elimination of miners and block producers by using smart contracts</a></li></ul><p>Smart Blockchain is a new technology for the next generation of Blockchain networks while solving the problems and limitations of Bitcoin. Of course, in this technology, Bitcoin will still not be dependent on any financial and banking institution.</p><p><a href="https://www.soliset.com/">Website</a> | <a href="https://twitter.com/SetSoli">Twitter</a> | <a href="https://t.me/soli_set">Telegram</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fdde502%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fhackernoon%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a3905deb7ee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/the-transfer-of-over-one-billion-dollars-by-using-two-bitcoins-transactions-how-much-really-did-2a3905deb7ee">The transfer of over one billion dollars by using two Bitcoin’s transactions; How much really did…</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Telegram, Filtering, and Iran floods]]></title>
            <link>https://medium.com/swlh/telegram-filtering-and-iran-floods-dafc8da01abe?source=rss-cc9da3ccea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/dafc8da01abe</guid>
            <category><![CDATA[soliset]]></category>
            <category><![CDATA[digital-record]]></category>
            <category><![CDATA[pavel-durov]]></category>
            <category><![CDATA[filtering]]></category>
            <category><![CDATA[telegram]]></category>
            <dc:creator><![CDATA[Somayyeh Gholami]]></dc:creator>
            <pubDate>Tue, 23 Apr 2019 12:19:53 GMT</pubDate>
            <atom:updated>2019-04-24T19:30:30.224Z</atom:updated>
            <content:encoded><![CDATA[<h4>The concealment of digital records in Telegram</h4><p>The large parts of Iran have been flooding but reaching the contents of Telegram messenger’s channels (via bypassing Telegram filtering) have not diminished among Iranian users although the telegram has been filtered in Iran. Of course, bypassing the filtering for flood victims and the rest of Iranians is not easy and even it slows down Iran’s internet connection, which is already quite slow. So why do not people use other messengers like WhatsApp which is not filtered? In this article, we will discuss it and demonstrate that the Telegram is not just as a messenger for Iranians but it is a combination of the messenger and a special type of social network. Another popularity of the telegram is that the concealment in digital records of human behavior is as simple as possible.</p><p>The authors of the article are Ms. Somayyeh Gholami and Mr. Mehran Kazeminia and it is written in Persian and English versions in April 2019.</p><p><a href="https://www.soliset.com/telegram2"><em>Persian Article</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*oMnwXehW0Asd0cvh1rxezQ.jpeg" /></figure><p>About a year ago, the telegram was filtered in Iran (the reason for this decision is not the subject of this article). At the same time, people were asked to leave the Telegram and to use non-filtered messengers and in particular domestic messengers. Previously, this happened for some other apps like Facebook and gradually most of the Iranian users were migrating from such filtered apps. But according to a recent report by <a href="http://www.irna.ir/fa/PhotoNews/3679505"><strong><em>The Islamic Republic News Agency, or IRNA</em></strong></a>, Iranian users haven’t left Telegram despite filtering or they reactivated their telegram accounts a few months later after leaving.</p><p>Obviously, by bypassing the filtering, Iran’s slow Internet will reduce the speed further and usually interrupt the connection. However, even in the flood-hit zone, people tolerate the difficulties of bypassing the filtering and most of the photos, news, and videos of the flood will be posted by themselves in the Telegram. More surprisingly, a number of top-level government officials, as well as some government agencies, such as television and official news agencies, have resumed their activities in the filtered Telegram and are reporting their flood news and stories in their telegram channels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*zI3e4hRhS_qeA7IeCMBdCA.jpeg" /><figcaption><a href="http://www.irna.ir/fa/PhotoNews/3679505"><strong><em>The Islamic Republic News Agency, or IRNA</em></strong></a></figcaption></figure><p>A non-governmental polling Agency in Iran, known as <a href="http://www.ispa.ir/Default/Details/fa/2030/%D9%81%DB%8C%D9%84%D8%AA%D8%B1%DB%8C%D9%86%DA%AF-%DA%86%D9%87-%D8%A7%D8%AB%D8%B1%DB%8C-%D8%A8%D8%B1-%D8%A7%D8%B3%D8%AA%D9%81%D8%A7%D8%AF%D9%87-%D8%A7%D8%B2-%D8%AA%D9%84%DA%AF%D8%B1%D8%A7%D9%85-%D8%AF%D8%A7%D8%B4%D8%AA%D8%9F"><strong><em>ISPA (Iranian Students Polling Agency)</em></strong></a> by presenting its latest results announced that before the filtering, about 60% of the Iran population used the Telegram, which reached 47% six months after filtering, but now again, after a year, 56% of Iran population use the telegram.</p><p>According to these statistics, about 30% of the Iranian now use Instagram, only 25% use WhatsApp and only 2% use Soroush Messenger (which is a Local Messenger). Note Instagram, WhatsApp, and Soroush are not filtered and do not have a problem of bypassing filtering, but they still have a much fewer user than the Telegram.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8W0_miiIck4tCQoe3C9RlQ.jpeg" /><figcaption><a href="http://www.ispa.ir/Default/Details/fa/2030/%D9%81%DB%8C%D9%84%D8%AA%D8%B1%DB%8C%D9%86%DA%AF-%DA%86%D9%87-%D8%A7%D8%AB%D8%B1%DB%8C-%D8%A8%D8%B1-%D8%A7%D8%B3%D8%AA%D9%81%D8%A7%D8%AF%D9%87-%D8%A7%D8%B2-%D8%AA%D9%84%DA%AF%D8%B1%D8%A7%D9%85-%D8%AF%D8%A7%D8%B4%D8%AA%D8%9F"><strong><em>ISPA (Iranian Students Polling Agency)</em></strong></a></figcaption></figure><p>A year ago, the Telegram was also filtered in Russia, and Russian users were forced to bypass the filtering. The methods of bypassing filtering in Russia and Iran seem to be similar. Recently, <a href="https://vk.com/tg_analytics?w=wall-149096576_111"><strong><em>Telegram Analytics</em></strong></a> announced that in 2019, 67000 Russian users were asked about the method of bypassing the Telegram filtering (Use of the circumvention software, VPN, etc.) and the results of the study are presented.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yzz6zw1dMbtyY0QeXCLcHA.jpeg" /><figcaption><a href="https://vk.com/tg_analytics?w=wall-149096576_111"><strong><em>Telegram Analytics</em></strong></a></figcaption></figure><p>In the following, we examine the reasons for the popularity of the Telegram, and we look at the type of communication in the telegram. However, let’s not forget that a dedicated social network with messenger has been formed on the Telegram platform, and the most basic issue for a social network is the type of network connection that can be the root of network problems. Talking about the relationship of people on social networks is just as important as the connection of people in real-world affairs, and the Telegram experience can be beneficial to everyone.</p><h3>Why do Iranians still use Telegram after it was filtered?</h3><p>After filtering, the telegram cannot compete with its unfiltered rivals like WhatsApp in terms of speed and ease of use. Obviously bypassing the filtering has a negative impact on the internet speed and ease of use of an app. Also, circumvention tools sometimes reduce application security. Even recently, unofficial versions of the Telegram which are advertised as anti-filtering, unfortunately, at the moment of downloading and installing, gain a lot of user’s phone accessibilities and controls. To the extent that after downloading, without users ‘awareness, they send advertisements through the users’ smartphones to different groups of Telegrams! Meanwhile, by controlling the cell phone of users, other things might be done that users are unaware of it. So the reasons for the popularity of telegrams after filtering cannot be speed, convenience or security.</p><p>In our opinion, the reasons for the popularity of the telegram can be defined in three parts; <strong>the first part</strong> is that in addition to the possibility to create groups, it is also possible to create public channels. The possibilities such as using bots with artificial intelligence, using the webTelegram, desktop app, and mobile app version simultaneously, are other features of the Telegram. All of these features have brought about a strong social network formed by Iranians, along with the use of Telegram as a simple messenger. So the substitute for the telegram can’t be just a simple messenger but the first condition is that the replacement should be a combination of Messenger and a desired social network for the people. Meanwhile, we have already examined the nature and concept of the Telegram Network in another article, and even in that article, we wrote about the necessity of Telegram public channels’ contents accessibility to the search engines. Here is the link of the article:</p><p><a href="https://medium.com/swlh/the-necessity-of-reconciliation-between-telegram-messenger-and-search-engines-8faf495b0387">The necessity of reconciliation between Telegram messenger and search engines</a></p><p><strong>The second part</strong> is about users’ inertia. Anyway, the Telegram has been advertised and its usage has been explained for several years directly and indirectly on the official Iranian national TV, and so on. Almost all people are familiar with it and have downloaded Telegram apps on their phone. Now, choosing another messenger application, it will no longer be a personal decision, because the rest of people may not have the same opinion and her/his connection will be discontinued. In other words, in the eyes of the majority of people in society, a new application should be better than filtered Telegram in order to make this change. Of course, this has not happened until today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5iE9eo5nV9vswHE7equVkA.jpeg" /></figure><p><strong>The third part</strong> relates to the digital records of Telegram users’ behavior. Initially, the users’ profiles of the Telegram are designed and configured based on the pattern of a simple messenger. In the user profile, usually other than one or more images, there is no other user information. The age, the status of education and marriage, the interests, the channels and the groups that have been subscribed, are unknown. Also, unlike other messengers (for example WhatsApp) user’ phone number is hidden from other members of the group or channel in Telegram. Users can join and be active in different channels and express their opinion and feelings, while even their phone numbers are invisible. They can even publish contradictory messages in different groups and still remain somehow anonymous. For example, they set their own messages based on common beliefs in each group and don’t concern their contradictory opinions and feelings aren’t matched. Because each message is recorded only in the same group that is written, it is not even clear that the group members are following which groups or channels and no one can follow others.</p><blockquote>In other words, people in the telegram can easily conceal their real political, social and ethical beliefs from some people and by managing their digital records somehow protect themselves. <strong>Also, let’s not forget that the telegram is designed as a messenger in the dark part of the Internet, and now, in addition, to being a messenger, is a widespread social network, yet search engines do not have access to any part of the Telegram platform</strong>, and thus users even for their secrecy, don’t concern about the information stored in the search engines.</blockquote><p>The unethical or ethical nature of this kind of secrecy is not our subject. However, a specific and possibly unique feature for the Telegram, which may have not been deliberately planned, is quite effective on the popularity of the telegram. It should be noted that specialists may be able to hide or manage their digital records on other networks. But for most people, it’s difficult to do it which Telegram makes it easy.</p><p>The proper design of future social networks is as important as the setting of constitution laws of real societies and should be based on all today’s experiences. Certainly, the Telegram experience, as a kind of social network, contains many positive and negative points that need to be addressed. Here is the end of the article, we’re eager to receive your comments.</p><h4>April 2019<br>Somayyeh Gholami<br>Mehran Kazeminia</h4><p><a href="https://www.soliset.com/">Website</a><br><a href="https://twitter.com/SetSoli">Twitter</a><br><a href="https://t.me/soli_set">Telegram</a></p><figure><a href="https://medium.com/swlh"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YqDjlKFwScoQYQ62DWEdig.png" /></a></figure><h4>This story is published in <a href="https://medium.com/swlh">The Startup</a>, Medium’s largest entrepreneurship publication followed by +445,678 people.</h4><h4>Subscribe to receive <a href="https://growthsupply.com/the-startup-newsletter/">our top stories here</a>.</h4><figure><a href="https://medium.com/swlh"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ouK9XR4xuNWtCes-TIUNAw.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dafc8da01abe" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/telegram-filtering-and-iran-floods-dafc8da01abe">Telegram, Filtering, and Iran floods</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>