<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Hubare Ra on Medium]]></title>
        <description><![CDATA[Stories by Hubare Ra on Medium]]></description>
        <link>https://medium.com/@hubare.ra?source=rss-f844fe9a5c9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*jzHsm7rRc1VYGXiMTWGlIQ.jpeg</url>
            <title>Stories by Hubare Ra on Medium</title>
            <link>https://medium.com/@hubare.ra?source=rss-f844fe9a5c9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 18:50:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hubare.ra/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Deep Well and the Neural Web]]></title>
            <link>https://medium.com/@hubare.ra/the-deep-well-and-the-neural-web-c35973a98169?source=rss-f844fe9a5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/c35973a98169</guid>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[fiction]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Hubare Ra]]></dc:creator>
            <pubDate>Thu, 20 Jul 2023 18:02:30 GMT</pubDate>
            <atom:updated>2023-07-20T21:41:25.778Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Deep Well VS The Deep Web</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uNyJI6uS8tXZquKuG0jnUw.jpeg" /><figcaption><a href="https://www.pexels.com/photo/famous-deep-well-of-quinta-da-regaleira-4514109/">https://www.pexels.com/photo/famous-deep-well-of-quinta-da-regaleira-4514109/</a></figcaption></figure><p>In the world of the future, two remarkable networks stood at the forefront of technological wonders — the Deep Well Network and the Deep Neural Web. Alice, the adventurous girl, found herself entwined in a grand scientific expedition that aimed to explore the mysteries of these extraordinary systems.</p><p>The Deep Well Network was discovered hidden deep within the heart of a lush forest. Alice stumbled upon the well while searching for ancient artifacts. As she gazed into its depths, she noticed an ethereal glow that beckoned her inside. Following her adventurous spirit, Alice descended into the well and found herself in an underground chamber of pulsating roots and luminous crystals.</p><p>With every step she took, Alice could feel the energy coursing through the roots, resonating with the Earth’s natural forces. The Deep Well Network seemed alive, and its architecture was unlike anything she had ever seen. As she touched the roots, information flowed through her, filling her mind with a profound understanding of the world. It was as if the network held the collective knowledge of the Earth’s existence.</p><p>Intriguingly, Alice noticed that the Deep Well Network processed data through a complex interconnected web of roots. The roots seemed to possess an inherent understanding of the environment, enabling the network to intuitively interpret and make sense of the data. It was like witnessing nature’s intelligence at its finest.</p><p>As Alice ventured further, she discovered a chamber with holographic representations of Earth’s history. The Deep Well Network had recorded the experiences and wisdom of generations past, preserving them for future generations. She was witnessing the network’s ability to retain and harmonize knowledge throughout time.</p><p>On the other side of the technological spectrum, the Neural Web was an artificial creation, built by brilliant minds in the bustling city of Technopolis. Researchers there had designed a neural network modeled after the human brain. Unlike the Deep Well Network, the Neural Web operated through layers of artificial neurons, each with its set of weighted connections.</p><p>Curiosity led Alice to Technopolis, where she witnessed the Neural Web in action. The researchers had trained it to recognize patterns in vast datasets, making it a powerful tool for image recognition, natural language processing, and various other tasks.</p><p>However, Alice noticed a striking difference. While the Neural Web excelled in specific tasks, it lacked the Deep Well Network’s intuition and consciousness. The Neural Web functioned as a computational tool, devoid of the profound connection to the Earth that the Deep Well Network displayed.</p><p>As she compared the two networks, Alice realized that the Deep Well Network’s architecture harnessed the Earth’s natural energies to develop an intricate understanding of the world. In contrast, the Neural Web’s artificial neurons learned from data without possessing any inherent connection to the environment.</p><p>Drawn to unraveling their differences, Alice became a bridge between the two worlds. She hoped to integrate the Deep Well Network’s intuitive comprehension with the Neural Web’s computational power, envisioning a future where both networks could coexist in harmony, enriching humanity’s understanding of the world.</p><p>But what lies ahead for Alice and the convergence of these two extraordinary networks remains an exciting cliffhanger, filled with boundless possibilities and uncharted territories.</p><p>(To be continued…)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c35973a98169" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Attention Isn’t All You Need- Part2]]></title>
            <link>https://medium.com/@hubare.ra/attention-isnt-all-you-need-part2-baaa3a69631b?source=rss-f844fe9a5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/baaa3a69631b</guid>
            <category><![CDATA[transfomers]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[attention]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Hubare Ra]]></dc:creator>
            <pubDate>Fri, 03 Feb 2023 17:46:57 GMT</pubDate>
            <atom:updated>2023-07-31T16:08:40.232Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XOmcD4AvdGv9lBrHDOeipg.png" /></figure><p>If you read the <a href="https://medium.com/p/2ab8edefd375">previous article</a>, you probably thought that Alice is a scientific fraudster. But the truth was not like that.</p><p>In her second paper “<strong>Attention, the vortex you’re caught in</strong>,” Bob (Former Alice) reflects on the idea that “attention is all you need” and how it became a popular misconception in the field of artificial intelligence. Bob argues that while attention mechanisms are important, they can also lead to a narrow focus on certain aspects of information, missing the broader perspective.</p><p>Bob explains that attention can be deceiving, leading researchers down a path of oversimplification and ignoring other crucial elements. Just like a vortex can trap and hold things in a tight space, the exclusive focus on attention can lead AI researchers to miss important information.</p><blockquote>Bob had a golden saying. Working on the attention mechanism is like being stuck in a swamp, the more you wade, the sooner you sink.</blockquote><p>Bob mentioned five attention problems:</p><blockquote><strong>Narrow focus:</strong> The exclusive focus on attention can lead to a narrow and limited perspective, missing the bigger picture.</blockquote><blockquote><strong>Oversimplification</strong>: Attention mechanisms can lead to oversimplification, causing researchers to ignore other important elements.</blockquote><blockquote><strong>Limited view</strong>: The singular focus on attention can be like being trapped in a swamp, limiting the view and ability to understand the full picture.</blockquote><blockquote><strong>Inaccurate predictions</strong>: By missing important information, AI systems that rely solely on attention mechanisms may make inaccurate predictions.</blockquote><blockquote><strong>Lack of comprehensiveness</strong>: A comprehensive approach to AI development that incorporates additional factors beyond attention is necessary to create truly intelligent machines.</blockquote><p>Bob’s article was published in a not so important journal. His article did not make much noise.</p><p>Indeed, why this critical article was not noticed by researchers? Bob, or former Alice, knew that all formers attacks were under the supervision of his ex-husband, George Petrovitch, he was widely regarded as the godfather of the <strong>Attention Mechanism Mafia</strong>.</p><p>Petrovitch had long been a staunch advocate of the<strong> “attention is all you need”</strong> philosophy and was vehemently opposed to any challenges to this view. He saw Alice’s paper as a direct attack on his life’s work and was determined to discredit it.</p><p>Adding fuel to the fire was the fact that Petrovitch was Alice’s ex-fiancé. It was revealed that the two had a tumultuous breakup, and Petrovitch was still harboring a grudge against Alice. He saw this as the perfect opportunity to exact his revenge.</p><p>Petrovitch launched a smear campaign against Alice, claiming that the paper was based on fraudulent research and that Alice was behind it all. He went so far as to start a poisoning plot against Alice, trying to discredit her and undermine the credibility of the paper.</p><p>Despite the opposition, Alice stood by her findings, and her paper continued to generate discussion and debate in the AI community. The conflict between the two sides was intense, with Petrovitch and his followers on one side and Alice on the other. The “attention vs. context and relationships” debate divided the community, with no clear winner in sight.</p><p><strong>Although this matter was told in the form of a story and was not a scientific fact. But my experience says that attention is not all you need. Do not pay too much attention to this story 😊</strong></p><p>Indeed, this video of mine is about attention, don’t miss it. <a href="https://www.youtube.com/watch?v=-gk0oHPCvAw">https://www.youtube.com/watch?v=-gk0oHPCvAw</a></p><p>I think this <a href="https://medium.com/@hubare.ra/6-reasons-to-migrate-to-reinforcement-learning-8deb0fa1a856">post </a>can be interesting for you. :)</p><blockquote><em>Despite the monetization of my account, Medium does not pay me due to my geographical location. Make me hope to continue writing by buying coffee. Thanks.</em></blockquote><blockquote><em>Tip link is below :)</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=baaa3a69631b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[6 Reasons to Migrate to Reinforcement Learning]]></title>
            <link>https://medium.com/@hubare.ra/6-reasons-to-migrate-to-reinforcement-learning-8deb0fa1a856?source=rss-f844fe9a5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/8deb0fa1a856</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Hubare Ra]]></dc:creator>
            <pubDate>Tue, 17 Jan 2023 16:07:49 GMT</pubDate>
            <atom:updated>2023-07-31T18:21:41.535Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*7h7H3VZ9yKS6eKwYBe1-HQ.png" /><figcaption>From <a href="https://en.wikipedia.org/wiki/Reinforcement_learning">https://en.wikipedia.org/wiki/Reinforcement_learning</a></figcaption></figure><p>Reinforcement Learning (RL) and Supervised Learning (SL) are two popular machine learning techniques. Both have their own advantages and disadvantages. Here are some pros and cons of each method:</p><p><strong>Reinforcement Learning (RL) Pros:</strong></p><ul><li>RL is well-suited for handling complex and dynamic environments, such as robotics, self-driving cars, and game playing.</li><li>RL can handle continuous action spaces, making it well-suited for tasks such as robotic control and continuous control in simulations.</li><li>RL can be used to make real-time decisions, which is important for tasks such as robotics and self-driving cars.</li><li>RL can handle uncertainty and make decisions based on incomplete or uncertain information.</li><li>It can learn from interactions with the environment and improve over time.</li></ul><p><strong>Reinforcement Learning (RL) Cons:</strong></p><ul><li>RL requires a lot of data, and it may be difficult to collect enough data to train RL models.</li><li>RL can be computationally intensive and require a lot of resources to train and run.</li><li>RL can be difficult to debug and interpret, as it is often hard to understand why a model is making certain decisions.</li><li>RL is sensitive to the choice of reward function, which can be hard to define in certain situations.</li></ul><p><strong>Supervised Learning (SL) Pros:</strong></p><ul><li>SL is relatively simple to implement and understand, making it accessible to a wide range of users.</li><li>SL can handle large amounts of data, making it well-suited for tasks such as image and speech recognition.</li><li>SL can be used for both classification and regression tasks.</li><li>SL models can be easily interpreted, as the relationship between input and output is well-defined.</li><li>SL models can be fine-tuned or improved by adding more data and adjusting parameters.</li></ul><p><strong>Supervised Learning (SL) Cons:</strong></p><ul><li>SL requires labeled data, which can be expensive and time-consuming to collect.</li><li>SL assumes that the relationship between input and output is fixed, which may not be the case in dynamic or changing environments.</li><li>SL can perform poorly when the test data is different from the training data, a phenomenon known as overfitting.</li><li>SL may not be able to handle certain types of data, such as sequential or unstructured data.</li></ul><p>In summary, both RL and SL have their own advantages and disadvantages. RL is well-suited for handling complex and dynamic environments, while SL is simpler to implement and understand, and can handle large amounts of data. The choice of method will depend on the specific task and the resources available.</p><h3>Why Should You Migrate to Reinforcement Learning</h3><p>Reinforcement learning (RL) is a type of machine learning that focuses on training agents to make decisions in an environment by maximizing a reward signal. Here are six reasons why you may want to consider migrating to RL:</p><ol><li><strong>Handling Complexity</strong>: RL can handle highly complex and dynamic environments, making it well-suited for tasks such as robotics, self-driving cars, and game playing.</li><li><strong>Flexibility</strong>: RL can be applied to a wide range of problems, from simple to very complex. It can be used for both supervised and unsupervised learning, and can be used in both online and offline settings.</li><li><strong>Handling Uncertainty</strong>: RL is particularly well-suited for tasks that involve uncertainty, such as decision making in dynamic and unpredictable environments.</li><li><strong>Continuous Action Spaces</strong>: RL can handle continuous action spaces, making it well-suited for tasks such as robotic control and continuous control in simulations.</li><li><strong>Scalability</strong>: RL can be scaled up to solve very large and complex problems, such as controlling a fleet of drones or playing complex video games.</li><li><strong>Real-time decision making</strong>: RL can be used to make real-time decisions, which is important for tasks such as robotics and self-driving cars, where decisions need to be made quickly and accurately.</li></ol><p>Overall, RL is a powerful tool for solving a wide range of problems and can be used in many different applications. If you are looking for a flexible, powerful, and versatile machine learning technique, then RL may be the right choice for you.</p><p><strong>If you ask me, reinforcement learning is the most similar mechanism for modeling the human brain. Don’t be afraid to try. It’s the future.</strong></p><blockquote>Indeed, this video of mine is about <strong>Attention is All you Need</strong>, don’t miss it. <a href="https://www.youtube.com/watch?v=-gk0oHPCvAw">https://www.youtube.com/watch?v=-gk0oHPCvAw</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8deb0fa1a856" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Attention Isn’t All You Need]]></title>
            <link>https://medium.com/@hubare.ra/attention-isnt-all-you-need-2ab8edefd375?source=rss-f844fe9a5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ab8edefd375</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[attention]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Hubare Ra]]></dc:creator>
            <pubDate>Mon, 16 Jan 2023 11:04:13 GMT</pubDate>
            <atom:updated>2023-07-31T16:07:37.761Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hlJf_1yNzjyYZ7xk01e1iw.png" /></figure><p>Once upon a time, in a world of artificial intelligence, the researchers believed that “attention is all you need” was the key to creating intelligent machines. They believed that by incorporating attention mechanisms into their models, they could create systems that could understand and process information just like a human brain.</p><p>The researchers worked tirelessly, developing new models and algorithms that incorporated attention mechanisms. They were convinced that they were on the brink of creating true artificial intelligence. But despite their best efforts, the systems they created were still far from perfect.</p><p>One day, a young researcher named Alice decided to take a closer look at the systems they had created. She noticed that while the attention mechanisms allowed the systems to focus on certain aspects of the information they were processing, they were still missing important pieces of information.</p><p>Alice realized that attention alone wasn’t enough. The systems needed to be able to take into account the context and the relationships between different pieces of information. She began to develop new models that incorporated not only attention, but also a way to understand the context and relationships between different pieces of information.</p><p>The new models were a breakthrough. The systems were now able to understand and process information in a way that was more similar to the human brain. They were able to make more accurate predictions and take into account the nuances of the information they were processing.</p><p>The researchers were amazed by Alice’s findings, and they quickly realized that “attention isn’t all you need”. They understood that to create truly intelligent machines, they needed to take into account not only attention, but also context and relationships. From that day on, they worked on developing models that could truly mimic the human brain.</p><p><strong>Alice said:</strong></p><blockquote>The incorporation of context and relationships means that the AI models take into account not only the individual pieces of information but also how they relate to each other and the broader context in which they are presented. This allows the AI systems to understand the nuances and subtleties of the information they are processing, making their predictions and decisions more accurate and human-like. For example, in natural language processing, knowing the context of a sentence and the relationships between the words and phrases in it, the AI model can understand the meaning of the sentence better. In image recognition, an AI model that incorporates context and relationships can understand the objects in an image and how they relate to each other, which helps it to recognize the scene.</blockquote><p>Alice published this idea in the Nature paper on 2025/05/05, but after some time it was proved that all her content was a lie and just a figment of her sick mind. She drew the ire of AI researchers, and finally, after a period of silence, published her second paper under Bob’s name, “Attention, the vortex you’re caught in.”</p><p><strong>Continues … [Part2 is </strong><a href="https://medium.com/@hubare.ra/attention-isnt-all-you-need-part2-baaa3a69631b"><strong>here</strong></a>]</p><p><strong>Although this matter was told in the form of a story and was not a scientific fact. But my experience says that attention is not all you need. Do not pay too much attention to this story 😊</strong></p><p>Indeed, this video of mine is about attention, don’t miss it. <a href="https://www.youtube.com/watch?v=-gk0oHPCvAw">https://www.youtube.com/watch?v=-gk0oHPCvAw</a></p><p>I think this <a href="https://medium.com/@hubare.ra/6-reasons-to-migrate-to-reinforcement-learning-8deb0fa1a856">post </a>can be interesting for you. :)</p><blockquote>Despite the monetization of my account, Medium does not pay me due to my geographical location. Make me hope to continue writing by buying coffee. Thanks.</blockquote><blockquote>Tip link is below :)</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ab8edefd375" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Path towards Artificial Brains Similar to Ants]]></title>
            <link>https://medium.com/@hubare.ra/exploring-the-possibilities-of-chemical-neural-networks-a-path-towards-artificial-brains-similar-32451be0c303?source=rss-f844fe9a5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/32451be0c303</guid>
            <category><![CDATA[chemical-neural-networks]]></category>
            <category><![CDATA[brain-mapping]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[environment]]></category>
            <dc:creator><![CDATA[Hubare Ra]]></dc:creator>
            <pubDate>Sat, 14 Jan 2023 17:35:50 GMT</pubDate>
            <atom:updated>2023-01-15T09:05:03.874Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3b7Q7Gkpx7QXsi_xFWgfpg.jpeg" /><figcaption>From <a href="https://www.pexels.com/photo/extreme-close-up-of-an-ant-on-wood-8155521/">https://www.pexels.com/photo/extreme-close-up-of-an-ant-on-wood-8155521/</a></figcaption></figure><p>Cloning an ant brain would be a difficult task for several reasons. One reason is that the ant brain is incredibly small, measuring only about 0.1 grams. Additionally, the brain of an ant is composed of many small, specialized regions that work together to control the ant’s behavior, which would be difficult to replicate in a single artificial system. Another reason is that, the complexity and intricacy of an ant’s neural network is not fully understood by scientists yet. The field of neuroscience still has a lot to learn about how the brain works, and recreating a brain as complex as an ant’s would require a lot of research and development.</p><p>Another challenge in cloning an ant brain is that the neural networks of an ant are evolved through millions of years of natural selection, resulting in highly efficient and specialized neural circuits. Replicating this level of efficiency in an artificial system would require a deep understanding of the underlying principles of neural computation, as well as the ability to mimic the processes of evolution in the lab. Additionally, the ant brain is also highly decentralized, with many different regions of the brain working together to control different behaviors. This decentralized structure would be difficult to replicate in an artificial system, as it would require a high degree of coordination between different sub-systems.</p><p>Another important thing to consider is that, the ant brain is not just a simple mechanism, but it is a product of the environment and the colony, the ant brain work together with other ants and the environment, this behavior is hard to replicate in a single artificial system.</p><p>In summary, cloning an ant brain is a highly challenging task, requiring a deep understanding of the principles of neural computation, the ability to mimic the processes of evolution, and the ability to replicate the decentralized and highly specialized structure of the ant brain.</p><h3><strong>What should we do to achieve an artificial brain similar to an ant brain?</strong></h3><p>There are several steps that could be taken to achieve an artificial brain similar to an ant brain:</p><ol><li>Increase our understanding of the ant brain: The first step in creating an artificial brain similar to an ant brain would be to increase our understanding of the neural networks and computation principles of the ant brain. This would require extensive research in the fields of neuroscience and entomology.</li><li>Develop new technologies for neural network modeling: To replicate the efficiency and specialization of the ant brain, new technologies would be needed for modeling and simulating neural networks. This could include new methods for creating and training artificial neural networks, as well as new hardware for running large-scale simulations.</li><li>Mimic the process of evolution: The ant brain has been shaped by millions of years of natural selection, resulting in highly efficient and specialized neural circuits. To replicate this level of efficiency in an artificial system, we would need to develop methods for mimicking the processes of evolution in the lab. This could include using genetic algorithms or other evolutionary methods to optimize neural networks.</li><li>Study the behavior of ant colonies: The behavior of ants is not only controlled by the brain, but also by their environment, the colony and their interaction with other ants. To replicate the behavior of ants, we need to study the behavior of ant colonies, and how they work as a group.</li><li>Develop decentralized AI systems: The ant brain is highly decentralized, with many different regions working together to control different behaviors. To replicate this decentralized structure in an artificial system, we would need to develop new methods for creating and controlling decentralized AI systems.</li></ol><p>All of these steps would require significant investment in research and development, as well as interdisciplinary collaborations between scientists and engineers from different fields.</p><h3><strong>Chemical neural networks</strong></h3><p>Chemical neural networks are a promising area of research that could potentially offer some advantages over software-based neural networks. Chemical neural networks are based on chemical reactions rather than electronic signals, which could potentially allow for faster and more energy-efficient computation. Additionally, chemical neural networks could also be more robust to environmental factors such as radiation and temperature changes.</p><p>However, there are also some challenges that would need to be overcome in order to develop practical chemical neural networks. One major challenge is that it would be difficult to create chemical networks that are able to store and process information in the same way that electronic networks do. Additionally, chemical neural networks would also be more difficult to program and control than electronic networks.</p><p>It’s also important to mention that, the chemical neural networks are still in the early stages of research and development and it may be some time before they are ready for practical use.</p><p>In conclusion, chemical neural networks are a promising area of research that could potentially offer some advantages over software-based neural networks, but there are also challenges that would need to be overcome in order to develop practical chemical neural networks. It’s still an open area for research and development, and more research is needed to understand the potential of chemical neural networks.</p><p>Previous studies in chemical neural networks are divided into the following methods.</p><ol><li><strong>Artificial Neural Networks with Memristors</strong>: Memristors are a type of electronic component that can change its resistance in response to an electrical signal. This property can be used to simulate the behavior of synapses in a neural network, and researchers have proposed using memristors to create chemical neural networks that can be controlled by electronic signals.</li><li><strong>Artificial Neural Networks with Electrochemical Neurons</strong>: Electrochemical neurons are electronic devices that mimic the behavior of neurons in the brain by using chemical reactions to transmit signals between neurons. Researchers have proposed using these devices to create chemical neural networks that can be controlled by electronic signals.</li><li><strong>Artificial Neural Networks with Microfluidics</strong>: Microfluidic devices are used to control and manipulate small volumes of liquids. Researchers have proposed using microfluidic devices to create chemical neural networks by controlling the flow of chemicals to simulate the behavior of neurons and synapses in the brain.</li><li><strong>Artificial Neural Networks with Synthetic Biology</strong>: Synthetic Biology is a field of science that uses engineering principles to design and build new biological parts, devices, and systems that do not exist in the natural world. Researchers in this field are developing new ways to create chemical neural networks by using synthetic biology to create new biological parts that can perform computation.</li></ol><p>It’s important to note that, most of these examples are still in the early stages of research and development and it may be some time before they are ready for practical use.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=32451be0c303" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hyperparameter. This is the basic problem of Transformer.]]></title>
            <link>https://medium.com/@hubare.ra/hyperparameter-this-is-the-basic-problem-of-transformer-353dda483d90?source=rss-f844fe9a5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/353dda483d90</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Hubare Ra]]></dc:creator>
            <pubDate>Mon, 09 Jan 2023 09:15:55 GMT</pubDate>
            <atom:updated>2023-07-31T18:22:33.481Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hx381rC1bFQZgxQkdLxsXw.jpeg" /><figcaption><a href="https://www.pexels.com/photo/close-up-photography-of-yellow-green-red-and-brown-plastic-cones-on-white-lined-surface-163064/">https://www.pexels.com/photo/close-up-photography-of-yellow-green-red-and-brown-plastic-cones-on-white-lined-surface-163064/</a></figcaption></figure><p>There are a few major problems that have been identified in transformer models:</p><ol><li><strong>Computational complexity</strong>: Transformer models can be computationally intensive to train, especially for large datasets and long sequences. This can make it challenging to use transformers for real-time applications or on resource-constrained devices.</li><li><strong>Difficulty in parallelization</strong>: The sequential nature of transformer models can make it difficult to parallelize the training process, which can slow down the training time.</li><li><strong>Lack of interpretability</strong>: Transformer models can be difficult to interpret, as they do not have a clear input-output mapping like some other machine learning models.</li><li><strong>Sensitivity to hyperparameters</strong>: Transformer models can be sensitive to the values of their hyperparameters, which can make it challenging to tune them for optimal performance.</li><li><strong>Limited input length</strong>: Transformer models are typically limited in the length of the input sequence they can process, which can be a problem for tasks that require longer context.</li></ol><p>Hyperparameters are values that are set prior to training a machine learning model and control the learning process. Examples of hyperparameters in transformer models include the learning rate, the number of layers in the model, and the size of the hidden states.</p><p>Transformer models can be sensitive to the values of their hyperparameters, which means that small changes in the values of these hyperparameters can result in significant changes in the model’s performance. This can make it challenging to tune the hyperparameters of a transformer model in order to achieve optimal performance on a given task.</p><p>One way to tune the hyperparameters of a transformer model is through a process called hyperparameter optimization, which involves systematically searching for the combination of hyperparameter values that results in the best performance on the validation set. There are several methods for performing hyperparameter optimization, such as grid search, random search and Bayesian optimization which can be time-consuming and computationally intensive.</p><blockquote><strong>Grid search</strong> is a method for hyperparameter optimization that involves specifying a grid of hyperparameter values to search over, and then training and evaluating a model for each combination of hyperparameter values.</blockquote><blockquote>For example, suppose we wanted to tune the learning rate and the batch size of a transformer model. We could specify a grid of possible values for these hyperparameters, such as a learning rate of 0.01, 0.1, or 1.0, and a batch size of 16, 32, or 64. Grid search would then train and evaluate a model for each combination of these hyperparameter values, resulting in a total of 9 different models (3 learning rates x 3 batch sizes).</blockquote><blockquote>The model with the best performance on the validation set is then chosen as the best model, and the corresponding hyperparameter values are used to train the final model on the full training set.</blockquote><blockquote>Grid search can be an effective method for hyperparameter optimization, but it can be computationally intensive, as it involves training and evaluating a large number of models. Additionally, it may be difficult to specify an appropriate grid of hyperparameter values, as the optimal values can depend on the specific task and dataset.</blockquote><blockquote><strong>Random search</strong> is another method for hyperparameter optimization that involves sampling random combinations of hyperparameter values and evaluating the corresponding model on the validation set.</blockquote><blockquote>Unlike grid search, which evaluates a fixed set of hyperparameter combinations, random search allows the search to cover a wider range of hyperparameter values, as it does not rely on a predefined grid. This can be particularly useful when the optimal hyperparameter values are not known in advance and are likely to be outside the range of values specified in the grid.</blockquote><blockquote>To perform random search, we first define a distribution for each hyperparameter, such as a uniform or normal distribution. Then, we sample random combinations of hyperparameter values from these distributions and train and evaluate a model for each combination. The process is repeated a fixed number of times, and the model with the best performance on the validation set is chosen as the best model.</blockquote><blockquote>Random search can be a more efficient method for hyperparameter optimization than grid search, as it does not require training and evaluating as many models. However, it may be less likely to find the optimal hyperparameter values compared to grid search or more sophisticated methods such as Bayesian optimization.</blockquote><blockquote><strong>Bayesian optimization</strong> is a method for hyperparameter optimization that is based on the principles of Bayesian statistics. It is an iterative process that involves building a probabilistic model of the objective function (e.g., the validation loss of a machine learning model) based on the hyperparameter values that have been evaluated so far. This model is then used to select the next set of hyperparameter values to evaluate, with the goal of finding the combination of values that minimizes the objective function.</blockquote><blockquote>One key advantage of Bayesian optimization is that it can incorporate prior knowledge about the objective function through the use of a probabilistic model, which can make it more efficient at finding the optimal solution compared to other methods such as random search or grid search. It can also handle constraints on the hyperparameter values and can be used to optimize expensive-to-evaluate objective functions, such as those that require training a machine learning model.</blockquote><blockquote>However, Bayesian optimization can be more computationally intensive compared to other methods, as it involves building and updating a probabilistic model at each iteration. It may also be more difficult to implement compared to other methods, as it requires specifying the probabilistic model and choosing hyperparameters for the optimization process itself.</blockquote><p>Also we use Reinforcement learning to tune hyperparameter.</p><blockquote><strong>Reinforcement learning (RL) </strong>is a machine learning approach that involves an agent learning to take actions in an environment in order to maximize a reward signal. It has been used to optimize various aspects of machine learning systems, including hyperparameters.</blockquote><blockquote>In the context of hyperparameter optimization, RL can be used to learn a policy that maps a set of hyperparameters to an action (e.g., to train a machine learning model with these hyperparameters). The agent can then learn to adjust the hyperparameters based on the performance of the model, in order to maximize a reward signal that is related to the performance of the model.</blockquote><blockquote>RL has been applied to the optimization of hyperparameters for various types of machine learning models, including neural networks. In principle, it could also be applied to the optimization of hyperparameters for transformer models.</blockquote><blockquote>However, RL-based hyperparameter optimization can be challenging to implement and may require a large amount of data and computation to be effective. It may also be less reliable than other methods, as it can be sensitive to the choice of reward function and can be prone to overfitting. As a result, RL-based hyperparameter optimization is not as widely used as other methods such as grid search, random search, or Bayesian optimization.</blockquote><p>Here are some pros and cons of grid search, random search, Bayesian optimization, and reinforcement learning (RL) for hyperparameter optimization:</p><p><strong>Grid search:</strong></p><ul><li>Pros: Simple to implement, can find the optimal solution if the grid is sufficiently fine.</li><li>Cons: Computationally intensive, requires specifying the grid of hyperparameter values in advance, may not be efficient at exploring a wide range of hyperparameter values.</li></ul><p><strong>Random search:</strong></p><ul><li>Pros: Simple to implement, can cover a wide range of hyperparameter values.</li><li>Cons: May not be as efficient at finding the optimal solution as grid search or Bayesian optimization, can be sensitive to the choice of distribution for sampling hyperparameter values.</li></ul><p><strong>Bayesian optimization:</strong></p><ul><li>Pros: Can incorporate prior knowledge about the objective function through the use of a probabilistic model, can handle constraints on the hyperparameter values, can be used to optimize expensive-to-evaluate objective functions.</li><li>Cons: Can be computationally intensive, requires specifying a probabilistic model and choosing hyperparameters for the optimization process, can be sensitive to the choice of priors.</li></ul><p><strong>RL:</strong></p><ul><li>Pros: Can learn a policy that maps hyperparameters to actions, can optimize a wide range of objectives.</li><li>Cons: Can be challenging to implement, may require a large amount of data and computation, can be sensitive to the choice of reward function, can be prone to overfitting.</li></ul><blockquote><strong>Despite all the explanations above, it seems Setting hyperparameters is an experimental matter and requires high-end hardware. This problem can definitely be solved with teamwork and parallel work. What other ways do you think there are to adjust transformers?</strong></blockquote><p>I suggest you read this interesting article.<br><a href="https://medium.com/@hubare.ra/attention-isnt-all-you-need-2ab8edefd375">Attention is not all you need</a>.</p><blockquote>Indeed, this video of mine is about <strong>Attention is All you Need</strong>, don’t miss it. <a href="https://www.youtube.com/watch?v=-gk0oHPCvAw">https://www.youtube.com/watch?v=-gk0oHPCvAw</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=353dda483d90" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to reduce the destructive effects of deep model training on the environment?]]></title>
            <link>https://medium.com/@hubare.ra/how-to-reduce-the-destructive-effects-of-deep-model-training-on-the-environment-ea0393e10ec4?source=rss-f844fe9a5c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea0393e10ec4</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[the-environment]]></category>
            <category><![CDATA[global-warming]]></category>
            <category><![CDATA[save-environment]]></category>
            <category><![CDATA[optimization]]></category>
            <dc:creator><![CDATA[Hubare Ra]]></dc:creator>
            <pubDate>Wed, 04 Jan 2023 08:54:15 GMT</pubDate>
            <atom:updated>2023-01-19T08:46:26.048Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*azl80grG_UL0smMjAuBGWg.jpeg" /></figure><h3><strong>Introduction</strong></h3><p>There are a number of ways that the negative environmental impacts of neural network training can be reduced. These include: using renewable energy, using energy-efficient hardware, optimizing neural network models, using distributed training, properly disposing of or recycling hardware, and collaborating with researchers and industry partners to develop new techniques and technologies. By implementing these strategies, it is possible to reduce the energy consumption and carbon footprint of neural network training and minimize its environmental impacts. It is important to consider the environmental impacts of neural network training and to take steps to address these impacts whenever possible.</p><p>There are a number of ways that the negative environmental impacts of neural network training can be reduced:</p><ol><li><strong>Use of renewable energy:</strong> Ensuring that the electricity used to power the hardware used for training neural networks is generated from renewable sources can significantly reduce greenhouse gas emissions.</li></ol><p><em>The use of renewable energy sources, such as solar, wind, and hydroelectric power, to generate electricity can significantly reduce greenhouse gas emissions compared to the use of fossil fuels. By ensuring that the electricity used to power the hardware used for training neural networks is generated from renewable sources, the carbon footprint of neural network training can be significantly reduced. This can be accomplished through a variety of means, such as purchasing renewable energy credits, investing in on-site renewable energy generation, or working with utilities that offer renewable energy options. In addition to reducing greenhouse gas emissions, the use of renewable energy can also have other environmental benefits, such as reduced air pollution and water use.</em></p><p><strong>2. Energy-efficient hardware</strong>: Using hardware that is energy-efficient can reduce the overall energy consumption of neural network training.</p><p><em>Using hardware that is energy-efficient can significantly reduce the overall energy consumption of neural network training. Energy-efficient hardware is designed to use less electricity to perform the same tasks as traditional hardware, which can result in significant energy savings. For example, using energy-efficient graphics processing units (GPUs) for training neural networks can reduce the energy consumption of the training process by using less power per unit of computation. Additionally, using energy-efficient hardware can also reduce the cooling requirements of a training setup, which can further reduce energy consumption. There are a number of hardware options available that are designed to be energy-efficient, and choosing these options can help to reduce the environmental impact of neural network training.</em></p><p><strong>3. Model optimization:</strong> Optimizing neural network models to be as efficient as possible can reduce the amount of computational resources and energy required for training.</p><p><em>Optimizing neural network models to be as efficient as possible can significantly reduce the amount of computational resources and energy required for training. There are a number of ways to optimize neural network models, including:</em></p><p><em>1. Reducing the number of parameters: Neural network models with a large number of parameters can be computationally expensive to train and may require more energy. Reducing the number of parameters in a model can make it more efficient and reduce the energy required for training.</em></p><p><em>2. Using efficient architectures: Choosing neural network architectures that are efficient and well-suited to the task at hand can also reduce the energy required for training.</em></p><p><em>3. Compressing models: Using techniques such as pruning and quantization to compress neural network models can also reduce the computational resources and energy required for training.</em></p><p><em>4. Optimizing hyperparameters: Fine-tuning the hyperparameters of a neural network model, such as the learning rate and batch size, can also improve model efficiency and reduce the energy required for training.</em></p><p><em>By optimizing neural network models in these ways, it is possible to significantly reduce the energy consumption of the training process.</em></p><p><strong>4. Distributed training</strong>: Training neural networks in a distributed fashion, using multiple machines or devices, can speed up training and reduce the energy consumption per model.</p><p><em>Distributed training refers to the process of training a neural network model using multiple machines or devices. This can be achieved by dividing the model and the training data across the available devices, and using algorithms to coordinate the training process. Distributed training can significantly speed up the training process compared to training on a single machine, and it can also reduce the energy consumption per model.</em></p><p><em>There are a number of benefits to distributed training:</em></p><p><em>1. Parallelization: By training a model on multiple devices in parallel, it is possible to significantly reduce the training time.</em></p><p><em>2. Scaling: Distributed training allows for the training of larger models that may not fit on a single device.</em></p><p><em>3. Energy efficiency: Training a model on multiple devices can also reduce the energy consumption per model, as the energy required for training is distributed across the devices.</em></p><p><em>4. Fault tolerance: Distributed training can also make the training process more resilient to failures, as the loss of a single device will not halt the training process.</em></p><p><em>By using distributed training, it is possible to speed up the training process and reduce the energy consumption per model.</em></p><p><strong>5. Reuse and recycling of hardware</strong>: Properly disposing of or recycling hardware used for training neural networks can reduce the environmental impacts of production and disposal.</p><p><em>The production and disposal of hardware, such as servers and graphics processing units (GPUs), used for training neural networks can have negative environmental impacts, including greenhouse gas emissions and pollution. To reduce these impacts, it is important to properly dispose of or recycle hardware when it is no longer in use.</em></p><p><em>There are a number of ways to reuse or recycle hardware:</em></p><p><em>1. Donating or selling: Instead of disposing of hardware, it can be donated or sold to others who can use it. This can extend the lifespan of the hardware and reduce the environmental impacts of production.</em></p><p><em>2. Recycling: Many hardware components, such as metals and plastics, can be recycled. Recycling these materials can reduce the environmental impacts of production and disposal.</em></p><p><em>3. Responsible disposal: When hardware cannot be reused or recycled, it is important to dispose of it in a responsible manner. This can include working with e-waste recycling programs or following proper disposal guidelines to ensure that hazardous materials are properly managed.</em></p><p><em>By properly disposing of or recycling hardware used for training neural networks, it is possible to reduce the environmental impacts of production and disposal.</em></p><p><strong>6. Collaboration and research</strong>: Collaborating with researchers and industry partners to develop new techniques and technologies that can reduce the energy consumption of neural network training can also be effective.</p><p><em>Collaborating with researchers and industry partners to develop new techniques and technologies that can reduce the energy consumption of neural network training can be an effective way to address the environmental impacts of this process. There are a number of ways in which collaboration and research can help to reduce the energy consumption of neural network training:</em></p><p><em>1. Developing new algorithms and approaches: Working with researchers to develop new algorithms and approaches for training neural networks can lead to more efficient models that require less energy to train.</em></p><p><em>2. Exploring alternative hardware: Collaborating with industry partners to explore alternative hardware options, such as specialized neuromorphic chips or quantum computers, that may be more energy-efficient for neural network training can also be effective.</em></p><p><em>3. Sharing best practices: Sharing information and best practices with other organizations that are also working on reducing the energy consumption of neural network training can help to identify and disseminate effective approaches.</em></p><p><em>4. Funding research: Funding research on techniques and technologies that can reduce the energy consumption of neural network training can help to drive innovation in this area.</em></p><p><em>By collaborating with researchers and industry partners and investing in research and development, it is possible to make progress in reducing the energy consumption of neural network training.</em></p><h3><strong>Conclusion</strong></h3><p>There are a number of ways that the negative environmental impacts of neural network training can be reduced. These include: using renewable energy, using energy-efficient hardware, optimizing neural network models, using distributed training, properly disposing of or recycling hardware, and collaborating with researchers and industry partners to develop new techniques and technologies. By implementing these strategies, it is possible to reduce the energy consumption and carbon footprint of neural network training and minimize its environmental impacts. It is important to consider the environmental impacts of neural network training and to take steps to address these impacts whenever possible.</p><blockquote>I love to drink coffee. I would be grateful if you could bring me one :)</blockquote><blockquote><em>Tip link is below :)</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea0393e10ec4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>