<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Gökhan Gerdan on Medium]]></title>
        <description><![CDATA[Stories by Gökhan Gerdan on Medium]]></description>
        <link>https://medium.com/@gokhangerdan?source=rss-4789f7c1f7cf------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*GCi9wrsA8ejl_Q8yWQwUrw.jpeg</url>
            <title>Stories by Gökhan Gerdan on Medium</title>
            <link>https://medium.com/@gokhangerdan?source=rss-4789f7c1f7cf------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 13:42:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@gokhangerdan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[OpenAI o1 and How It Works]]></title>
            <link>https://medium.com/bimser-tech/openai-o1-and-how-it-works-0eb270efc341?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/0eb270efc341</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[openai-o1]]></category>
            <category><![CDATA[openai-o1-preview]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Fri, 27 Sep 2024 10:46:02 GMT</pubDate>
            <atom:updated>2024-09-27T10:46:02.638Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-58who636q4mTlL3EA7LyQ.jpeg" /></figure><p>OpenAI recently announced new series of reasoning models, o1 and o1-mini. These models both have 128K context limit just like the previous GPT4-o and GPT4o-mini models but can reason through complex tasks and solve harder problems than previous models in science, coding, and math. o1-mini is the smaller, faster, and cost-effective option and it’s price range is in between GPT-4o and GPT-4o-mini.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/565/1*ygQmY4LYP_5YeURqyq1cTQ.png" /><figcaption><a href="https://openai.com/api/pricing/">https://openai.com/api/pricing/</a></figcaption></figure><p>While the larger model o1 has strong reasoning capabilities and broad world knowledge, the smaller model o1-mini is 3–5x faster and competitive with o1 at coding tasks and usefull for application that require reasoning without broad world knowledge. You can see a comparison between o1-mini, o1-preview, and GPT-4o on varios coding benchmarks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*wP2zRQ7fmNP4IoQ9buAH2g.png" /></figure><p>The new reasoning models use a similar way of thinking to human by using a chain of thought, when attempting to solve a complex problem. These models learns to refine its strategies when creating a chain of thought, recognize and correct its mistakes and break down complex steps into simpler steps by using reinforcement learning.</p><p>Even though these models offer significant advancements in reasoning. They are not intended to replace GPT-4o and GPT-4o-mini in some use-cases. For applications that need image understanding, function calling, streaming, or consistently fast responses, GPT-4o and GPT-4o-mini models are still the right choice. But if you need complex reasoning capabilities, and longer response times are not a problem, you can consider o1 models.</p><p>Before o1 models we only had input tokens and completion tokens but for o1 models to works they need a third type of tokens called <strong><em>reasoning tokens</em></strong>. They use these reasoning tokens to break down their understanding of the prompt and considering multiple ways to generate response. These reasoning tokens are not visible and generated from input tokens. After generating reasoning tokens, o1 produces an answer as visible completion tokens, and discards the reasoning tokens from its context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*JEMRp0h1jtwm9GFTuqoJ5g.png" /><figcaption>An example of multi-step conversation between a user and an assistant.</figcaption></figure><p>Let’s breakdown this example:</p><p>Turn 1: User enters a prompt, o1 generates reasoning tokens by using input tokens, and o1 generates a completion by using input prompt and reasoning tokens.</p><p>Turn 2: Input tokens and output tokens from the previous step of the multi-step conversation passed to this step but reasoning tokens from the previous step are not passed to this step. o1 models generated new reasoning tokens from the input tokens, and generated new completion tokens by using the input tokens and the new reasoning tokens.</p><p>Turn 3: In this example this turn shows how o1 models truncates the completion tokens when the input tokens and reasoning tokens got larger in the third step of this multi-turn conversation. The o1 and o1-mini models are both have 128K context window but maximum output token limit for o1-preview is 32K~ and for o1-mini is 65K~. OpenAI recommends to allocate at leask 25K tokens for reasoning and outputs.</p><p>The new reasoning models have their benefits as well as disadvantages. But let’s dive into how they actually work by analyzing one of OpenAI’s showcase examples for a coding task.</p><p><strong>This is how the input prompt looks like:</strong></p><blockquote>Write a bash script that takes a matrix represented as a string with format ‘[1,2],[3,4],[5,6]’ and prints the transpose in the same format.</blockquote><p><strong>This is how the chain of thought looks like:</strong></p><blockquote>So the user is requesting a bash script that can take a string representing a matrix, such as ‘[1,2],[3,4],[5,6]’ and output its transpose, in the same format.</blockquote><blockquote>Let’s first understand the input and output formats.</blockquote><blockquote>Input: ‘[1,2],[3,4],[5,6]’<br>which corresponds to the matrix:<br>1 2<br>3 4<br>5 6</blockquote><blockquote>So the input is a series of rows, each in brackets separated by commas.</blockquote><blockquote>The transpose of this matrix would be:<br>1 3 5<br>2 4 6</blockquote><blockquote>So the requested output is ‘[1,3,5],[2,4,6]’</blockquote><blockquote>…</blockquote><p>This chain of thought example above is just the first part of the actual example. You can see the full example from <a href="https://openai.com/index/learning-to-reason-with-llms/">this link</a>. In this part we are seeing that, the model is first trying to understand the input and the desired output. If you check out the full example, you can see other steps: Overall Task, Constraints, Approach, Implementation Steps, and Coding. After model understands the input and the desired output, it determines the overall task, constraints and the approach. After that the model generates the implementation steps from capturing the input to output the transposed matrix in the required format. Once the implementation steps are generated the model starts coding and also simulates the code it generated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/858/1*_ZWBohI1dVkg9QsdufrakA.png" /></figure><p>This was the thinking part of the new o1 models. To generate the actual completion, model is using both the input and the reasoning tokens and generate the desired output.</p><blockquote>Here’s a bash script that takes a matrix represented as a string in the format “[1,2],[3,4],[5,6]” and prints its transpose in the same format:</blockquote><blockquote>…</blockquote><h4>References</h4><p><a href="https://openai.com/index/openai-o1-mini-advancing-cost-efficient-reasoning/">OpenAI o1-mini</a></p><p><a href="https://openai.com/index/learning-to-reason-with-llms/">Learning to Reason with LLMs</a></p><p><a href="https://openai.com/index/introducing-openai-o1-preview/">Introducing OpenAI o1-preview</a></p><p><a href="https://platform.openai.com/docs/guides/reasoning">Reasoning models</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0eb270efc341" width="1" height="1" alt=""><hr><p><a href="https://medium.com/bimser-tech/openai-o1-and-how-it-works-0eb270efc341">OpenAI o1 and How It Works</a> was originally published in <a href="https://medium.com/bimser-tech">bimser.tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Diffusion Models Explained]]></title>
            <link>https://medium.com/bimser-tech/diffusion-models-explained-5f4a83bf4037?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f4a83bf4037</guid>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[image-generation]]></category>
            <category><![CDATA[midjourney]]></category>
            <category><![CDATA[text-to-image-generation]]></category>
            <category><![CDATA[diffusion-models]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Fri, 27 Sep 2024 10:45:38 GMT</pubDate>
            <atom:updated>2024-09-27T10:45:38.028Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OddFC79NsFfulxe9sPVsPQ.jpeg" /></figure><p>Everyone is talking about the ownership of AI-generated art, with artists questioning about their work being used to train these models. But have you ever wondered how these models actually work? It all starts from the concepts of thermodynamics.</p><p>Diffusion in thermodynamics, is simply movement of particles from an area of higher concentration to lower concentration, as long as there is a non-equilibrium. Diffusion models are inspired by this process. Image generation with diffusion models can be viewed as a Markov chain of diffusion steps, where each step represents a transformation of the image. Each transformation is only influenced by the previous step. This allows model to gradually refine the image from random noise to desired image.</p><p>There are several popular implementations of diffusion models, including OpenAI’s DALL-E, Stable Diffusion, and Midjourney. While all three utilize diffusion processes to generate images from textual descriptions, each one takes a different approach to meet their user needs and artistic visions. Now let’s dive into the step by step process of how diffusion models generate images from textual descriptions.</p><h4>Three main steps of Text-to-Image generation:</h4><ol><li>Text Representation Generation</li><li>Image Representation Refining</li><li>Image Upscaling</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3VFPD1-2ya47vDNaJf_uDw.png" /><figcaption>Text-to-image generation with diffusion models</figcaption></figure><h4>Text Representation Generation</h4><p>In diffusion models, text prompts broken down into sequence of tokens, similar to the process used in other large language models (LLMs). These tokens can represent words, subwords, characters, or even entire phrases, depending on the tokenization method used. By breaking text into manageable pieces, models can better understand patterns, relationships, and meanings in the data.</p><p>Once the text prompt is broken down into tokens, each token is converted into a numerical representation. This encoding process assigns unique identifier to each token, turning text into a sequence of numbers that the model can understand.</p><h4>Image Representation Refining</h4><p>Starting with random noise, diffusion models refine the image representation through multiple steps, with the guidance of the text prompt. The process typically has two main stages: first, a neural network predicts noise in the current state of the image representation, second noise removal is gradually applied to produce sharper images. After total number of refinements, model produces a clear and detailed image representation.</p><h4>Image Upscaling</h4><p>Image upscaling is the final step of the diffusion model process, where the goal is to boost the generated image to a higher resolution by using a neural network decoder.</p><h4>Image Generation Example With Python</h4><p>For this part, I created an example script to generate images from text prompts. I chose the <a href="https://huggingface.co/openskyml/midjourney-mini">Midjourney Mini</a> model as the diffusion model because it is available on Hugging Face, licensed under MIT, and can run at reasonable speeds without a GPU. My system features i7–8850H CPU and 32 GB of RAM.</p><pre>from diffusers import DiffusionPipeline<br><br><br>pipeline = DiffusionPipeline.from_pretrained(&quot;midjourney-community/midjourney-mini&quot;)<br><br>image = pipeline(<br> &quot;stained glass of darth vader, backlight, centered composition, masterpiece, photorealistic, 8k&quot;<br>).images[0]<br><br>image.save(&#39;generated_image.png&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*quMRVtG3K7NHwmxg73PJsw.png" /><figcaption>Generated image</figcaption></figure><p>The image generation process took about 4–5 mins on my system. I was impressed by the quality of the generated images, both in terms of how well they captured the textual descriptions and their overall resolution. You can experiment with this script to further evaluate this model or explore other text-to-image generation models available on Hugging Face.</p><h4>References</h4><p><a href="https://arxiv.org/abs/2112.10752">High-Resolution Image Synthesis with Latent Diffusion Models</a></p><p><a href="https://arxiv.org/abs/1503.03585">Deep Unsupervised Learning using Nonequilibrium Thermodynamics</a></p><p><a href="https://arxiv.org/abs/2006.11239">Denoising Diffusion Probabilistic Models</a></p><p><a href="https://arxiv.org/abs/2404.07771">An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization</a></p><p><a href="https://lilianweng.github.io/posts/2021-07-11-diffusion-models/">What are Diffusion Models?</a></p><p><a href="https://poloclub.github.io/diffusion-explainer/">What is Stable Diffusion?</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f4a83bf4037" width="1" height="1" alt=""><hr><p><a href="https://medium.com/bimser-tech/diffusion-models-explained-5f4a83bf4037">Diffusion Models Explained</a> was originally published in <a href="https://medium.com/bimser-tech">bimser.tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Search Engine Comparison for RAG Applications: Qdrant vs Elasticsearch]]></title>
            <link>https://medium.com/bimser-tech/search-engine-comparison-for-rag-applications-qdrant-vs-elasticsearch-3709f4d6c840?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/3709f4d6c840</guid>
            <category><![CDATA[search-engines]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[retrieval-augmented]]></category>
            <category><![CDATA[qdrant]]></category>
            <category><![CDATA[comparison]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Fri, 05 Jul 2024 06:07:04 GMT</pubDate>
            <atom:updated>2024-07-05T06:07:04.713Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hCYgxkVklKc36MOIbtx-pg.jpeg" /></figure><p>With the current state of Large Language Models (LLMs), applications are endless. One of the more popular application is powering your chatbot with a LLM instead of rule based approaches. Nowadays chatbots are even changing how we interact with applications, from graphical user interfaces to conversational interfaces with the power of AI.</p><p>Apart from that, even with the most powerful LLM’s there are downsides such as hallucinations, becoming outdated, not being aware of enterprise context. RAG allows LLMs to become more contextually correct with source attribution by accessing relevant context from a vector search.</p><p>With the use of locally deployed Small Language Models (SLMs) such as Microsoft’s Phi-3, event though Phi-3’s strong reasoning abilities, this model is struggling to store enough factual knowledge due to its smaller size. This makes RAG implementation even more important.</p><p>To implement RAG into your AI powered chatbot, you need a search engine to retrieve relevant context from your knowledge base. There are many free and open search engine options in the market. In this article we compared Qdrant vs Elasticsearch.</p><h4>Why compare Qdrant with Elasticsearch?</h4><p>Even though Qdrant is built to support multidimensional vector data, it is still a newer product against the more mature option of Elasticsearch. This means your organization might be already using Elasticsearch for other purposes. In this article we compared Qdrant with Elasticsearch in a scenario as if it makes sense to spend resources on a newer technology over the one we already familiar with.</p><h3>Qdrant</h3><p>Qdrant is a cutting-edge vector similarity search engine known for its high throughput, minimal latency, rapid indexing, and precise control. It elevates search functionalities by providing semantic, similarity, multimodal, and hybrid search capabilities, among other features.</p><h4>Pros</h4><ul><li>Purpose-built for Vectors</li><li>Better trade-off between performance and precision</li><li>Highest RPS and lowest latencies</li><li>Open Source</li></ul><h4>Cons</h4><ul><li>Authentication only</li><li>Newer product, smaller community</li></ul><h3>Elasticsearch</h3><p>Elasticsearch is a distributed, RESTful search and analytics engine designed to handle an increasing array of use cases. It allows you to execute and integrate various types of searches, including structured, unstructured, geo, metric, and those from a piped query language.</p><h4>Pros</h4><ul><li>Role-based Access Control (RBAC)</li><li>Support for both stream and batch of vector data</li><li>Binary Vector support</li><li>Excels in text-based search and analytics applications</li></ul><h4>Cons</h4><ul><li>Struggles with indexing time efficiency</li><li>Dealing with high-dimensional vector data</li><li><a href="https://www.elastic.co/elasticsearch/elasticsearch-relevance-engine">Elasticsearch Relevance Engine’s capabilities come with Elastic Enterprise Search Platinum and Enterprise plans</a></li></ul><h3>Conclusion: Qdrant vs Elasticsearch</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mwT8orm4On3NqUhjh2zKBw.png" /><figcaption><a href="https://qdrant.tech/benchmarks/">https://qdrant.tech/benchmarks/</a></figcaption></figure><p>In consideration of choosing one of this search engines to use it in a RAG application, being built to support multidimensional vector data with high performance, Qdrant is the clear winner as you can see in the <a href="https://qdrant.tech/benchmarks/">benchmarks</a>. But if you are already using Elasticsearch and maybe other tools from Elastic Stack, you can definitely choose to go with Elasticsearch for its rich ecosystem, versatility, and proven track record in traditional text-based search domains.</p><h3>References</h3><p><a href="https://qdrant.tech/benchmarks/">Vector Database Benchmarks</a></p><p><a href="https://qdrant.tech/qdrant-vector-database/">Qdrant Vector Database</a></p><p><a href="https://qdrant.tech/rag/">Retrieval Augmented Generation (RAG) With Qdrant</a></p><p><a href="https://www.elastic.co/elasticsearch/vector-database">World’s most used vector database — Elasticsearch</a></p><p><a href="https://www.elastic.co/elasticsearch/elasticsearch-relevance-engine">Elasticsearch Relevance Engine</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3709f4d6c840" width="1" height="1" alt=""><hr><p><a href="https://medium.com/bimser-tech/search-engine-comparison-for-rag-applications-qdrant-vs-elasticsearch-3709f4d6c840">Search Engine Comparison for RAG Applications: Qdrant vs Elasticsearch</a> was originally published in <a href="https://medium.com/bimser-tech">bimser.tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A review on Microsoft Phi-3: Pocket-sized GPT-3.5]]></title>
            <link>https://medium.com/bimser-tech/a-review-on-microsoft-phi-3-pocket-sized-gpt-3-5-31e468329a4b?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/31e468329a4b</guid>
            <category><![CDATA[review]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[phi-3]]></category>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[phi-3-mini]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Fri, 05 Jul 2024 06:06:34 GMT</pubDate>
            <atom:updated>2024-07-05T06:06:34.908Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3rEJDCWHQMHMUq_sOberRg.jpeg" /></figure><p>In today’s rapidly growing AI race, language models have become the most intriguing advancement in the field. With the very high processing power requirements of the Large Language Models (LLMs), Small Language Models (SLMs) become more and more popular among AI community. After the success of Google’s Gemma and Microsoft’s Phi-2 models, Microsoft recently published their latest models: The Phi-3 series models.</p><p>At Bimser we are also taking advantage of Microsoft’s latest Phi-3 models to develop AI tools for our privacy sensitive use cases. In this article we are going to share our experience with Phi-3 models and we are going to reveal some of our use cases.</p><h3>About Phi-3 Series Models</h3><p>As we write this review there are various different options of Microsoft’s latest Phi-3 series models as you can see below. There are small and large context size options with different parameter sizes such as 3.8B for mini, 7B for small and 14B for medium. Also one can not ignore Microsoft’s most intriguing addition to this list is the vision model with image understanding capabilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*Mfgc5kj00ot7qcbo36OTCQ.png" /><figcaption><a href="https://huggingface.co/collections/microsoft/phi-3-6626e15e9585a200d2d761e3">Phi-3</a></figcaption></figure><p>From the mini version to rest of the Phi-3 models, all of them outperforms their similar sized alternatives in addition to that, these models are even comparable to OpenAI’s latest models for it’s reasoning capabilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*nz3Ifh02kzsBEmIuavUUsQ.png" /><figcaption><a href="https://huggingface.co/papers/2404.14219">Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone</a></figcaption></figure><h3>Advantages Of Using Phi-3-mini Over Its Larger Sized Opponents</h3><p>In this part we are going to focus on the smallest variant of Phi-3 series models: Phi-3-mini, due to it’s lower processing power requirement.</p><p>It is always easier to maintain and, in many cases, it is even cheaper to use LLMs as paid third party services, like using OpenAI’s API. But in some cases you may not be comfortable sending any data over the internet. And this is where smaller language models with strong reasoning capabilities like Phi-3-mini shines because of its lower processing power requirement to be able to deploy this model locally.</p><p>But Phi-3-mini is not perfect either. Even all these benefits we discussed above, this model is limited by its size store too much factual knowledge. But this issue can be resolved by using a search engine to retrieve context and feeding the model with context infused prompts.</p><h3>Some Use Cases With Comparisons To GPT-3.5</h3><h4>Translator</h4><p>In this case we compared Phi-3-mini to GPT-3.5-turbo-1106 on a Turkish to English translation task. We gave the model some UI component names from Bimser’s Synergy CSP application to translate from Turkish to English as an example as shown in the prompt below:</p><pre>Translate following flow item names from Turkish to English:<br>Akış Başlangıcı<br>Ünvan<br>Akış Bitişi<br>Akış Devam Ettirici<br>Akış Tetikleyici<br>Atama<br>Karşılaştırma<br>Departman<br>Değişken<br>Akış Durumu<br>Paralel Akış<br>Ve Operatörü<br>Veya Operatörü<br>Düğüm Girişi<br>Düğüm Çıkışı<br>Yorum</pre><p><strong><em>Answer of GPT-3.5-turbo-1106:</em></strong></p><pre>Flow Start<br>Title<br>Flow End<br>Flow Continuer<br>Flow Trigger<br>Assignment<br>Comparison<br>Department<br>Variable<br>Flow Status<br>Parallel Flow<br>And Operator<br>Or Operator<br>Node Input<br>Node Output<br>Comment</pre><p><strong><em>Answer of Phi-3-mini-4k-instruct:</em></strong></p><pre>Source flow item names in Turkish:<br>Akış Başlangıcı - Start of Flow<br>Ünvan - Title/Section<br>Akış Bitişi - End of Flow<br>Akış Devam Ettirici - Continuation of Flow<br>Akış Tetikleyici - Pause in Flow<br>Atama - Decision<br>Karşılaştırma - Comparison<br>Departman - Department/Section (in the context of a flow diagram)<br>Değişken - Variable<br>Akış Durumu - State<br>Paralel Akış - Parallel Flow<br>Ve Operatörü - And Operator<br>Veya Operatörü - Or Operator<br>Düğüm Girişi - Loop Start<br>Düğüm Çıkışı - Loop End<br>Yorum - Comment/Note<br><br>Please note that &quot;Departman&quot; in this context is likely referring to a section or module within the flow diagram, rather than an actual department. The translation can vary based on specific usage scenarios and domain-specific jargon.</pre><h4>Coding</h4><p>In this case we compared Phi-3-mini to GPT-3.5-turbo-1106 on a code editing task. We gave the model a piece of Python code and described the changes we wanted as an example shown in the prompt below:</p><pre>Edit this Python code to use list comprehension instead of a for loop:<br>numbers = [1,2,3,4,5,6,7,8]<br>odd_numbers = []<br>for item in numbers:<br>  if item % 2 == 1:<br>    odd_numbers.append(item)<br>print(odd_numbers)</pre><p><strong><em>Answer of GPT-3.5-turbo-1106:</em></strong></p><pre>Here&#39;s the Python code using list comprehension instead of a for loop:<br><br>```python<br>numbers = [1, 2, 3, 4, 5, 6, 7, 8]<br>odd_numbers = [item for item in numbers if item % 2 == 1]<br>print(odd_numbers)<br>```</pre><p><strong><em>Answer of Phi-3-mini-4k-instruct:</em></strong></p><pre>You can use list comprehension to achieve the same result with a more concise and efficient code as follows:<br><br>```python<br>numbers = [1,2,3,4,5,6,7,8]<br>odd_numbers = [item for item in numbers if item % 2 == 1]<br>print(odd_numbers)<br>```</pre><h4>Json Outputs</h4><p>In this case we compared Phi-3-mini to GPT-3.5-turbo-1106 on a json outputting task. We combined this task with the Translator task and asked the model to give the translations in json format as an example shown in the prompt below:</p><pre>Translate following flow item names from Turkish to English create a json with translations as answer:<br>Akış Başlangıcı<br>Ünvan<br>Akış Bitişi<br>Akış Devam Ettirici<br>Akış Tetikleyici<br>Atama<br>Karşılaştırma<br>Departman<br>Değişken<br>Akış Durumu<br>Paralel Akış<br>Ve Operatörü<br>Veya Operatörü<br>Düğüm Girişi<br>Düğüm Çıkışı<br>Yorum</pre><p><strong><em>Answer of GPT-3.5-turbo-1106:</em></strong></p><pre>{<br>  &quot;Akış Başlangıcı&quot;: &quot;Flow Start&quot;,<br>  &quot;Ünvan&quot;: &quot;Title&quot;,<br>  &quot;Akış Bitişi&quot;: &quot;Flow End&quot;,<br>  &quot;Akış Devam Ettirici&quot;: &quot;Flow Continuer&quot;,<br>  &quot;Akış Tetikleyici&quot;: &quot;Flow Trigger&quot;,<br>  &quot;Atama&quot;: &quot;Assignment&quot;,<br>  &quot;Karşılaştırma&quot;: &quot;Comparison&quot;,<br>  &quot;Departman&quot;: &quot;Department&quot;,<br>  &quot;Değişken&quot;: &quot;Variable&quot;,<br>  &quot;Akış Durumu&quot;: &quot;Flow Status&quot;,<br>  &quot;Paralel Akış&quot;: &quot;Parallel Flow&quot;,<br>  &quot;Ve Operatörü&quot;: &quot;And Operator&quot;,<br>  &quot;Veya Operatörü&quot;: &quot;Or Operator&quot;,<br>  &quot;Düğüm Girişi&quot;: &quot;Node Input&quot;,<br>  &quot;Düğüm Çıkışı&quot;: &quot;Node Output&quot;,<br>  &quot;Yorum&quot;: &quot;Comment&quot;<br>}</pre><p><strong><em>Answer of Phi-3-mini-4k-instruct:</em></strong></p><pre>{<br>  &quot;StartTransaction&quot;: &quot;Akış Başlangıcı&quot;,<br>  &quot;Section&quot;: &quot;Ünvan&quot;,<br>  &quot;EndTransaction&quot;: &quot;Akış Bitişi&quot;,<br>   &quot;RollbackTransaction&quot;: &quot;Akış Devam Ettirici&quot;,<br>  &quot;Commit Transaction&quot;: &quot;Akış Tetikleyici&quot;,<br>  &quot;Abort&quot;: &quot;Atama&quot;,<br>  &quot;Search for Matches&quot;: &quot;Karşılaştırma&quot;,<br>  &quot;Department&quot;: &quot;Departman&quot;,<br>  &quot;Variable&quot;: &quot;Değişken&quot;,<br>  &quot;Transaction Status&quot;: &quot;Akış Durumu&quot;,<br>  &quot;Parallel Transaction&quot;: &quot;Paralel Akış&quot;,<br>  &quot;And Operator&quot;: &quot;Ve Operatörü&quot;,<br>  &quot;Or Operator&quot;: &quot;Veya Operatörü&quot;,<br>  &quot;Start Rollback&quot;: &quot;Düğüm Girişi&quot;,<br>  &quot;End Rollback&quot;: &quot;Düğüm Çıkışı&quot;<br>}</pre><h4>Chatting With Your Local Knowledge Base</h4><p>In this case we compared Phi-3-mini to GPT-3.5-turbo-1106 on a Q/A task. We gave the model a question and a context to answer from as an example shown in the prompt below:</p><pre>Context:<br>The flow start object is the object that specifies the starting point of the flow of the designed process. This object should be at the top of every flow design. Other objects to be used in the flow design are ordered after this flow start object and relate to each other with the connection object. Only 1 in-stream object is allowed on a flow screen.<br>While the authoring interface is active screen flow, the Toolbox panel lists flow objects. Sliding a Start Stream object from the Toolbox panel and dropping it onto the flow screen adds the object to the stream. Clicking the start stream object will list the properties that the object has in the Property Viewer panel.<br><br>Answer questions below using the context above:<br>Q1: How to use the Flow Start object?</pre><p><strong><em>Answer of GPT-3.5-turbo-1106:</em></strong></p><pre>To use the Flow Start object, you can drag it from the Toolbox panel and drop it onto the flow screen. This will add the object to the stream. Once added, you can click on the start stream object to view its properties in the Property Viewer panel. The Flow Start object should be placed at the top of every flow design as it specifies the starting point of the flow.</pre><p><strong><em>Answer of Phi-3-mini-4k-instruct:</em></strong></p><pre>A1: To use the Flow Start Object, you should follow these steps while working within the active screen flow and Toolbox panel of an authoring interface:<br><br>1. Ensure that your design process has a single in-stream (start stream) on each flow screen as per the guidelines provided. This start stream object will serve as the starting point for your designed process.<br>2. Locate the Flow Start Object within the Toolbox panel, which lists all available flow objects you can use in your design.<br>3. Select the Flow Start Object by clicking on it to add it to a flow screen. Once selected, drag and drop it onto an empty area of a flow screen, effectively adding it as the starting point for your process.<br>4. After placing the Flow Start Object on the flow screen, click on it to open its properties in the Property Viewer panel. This will allow you to review or modify any settings related to this object according to your design needs.<br>5. Continue designing your process by adding other objects listed in the Toolbox panel, which can be connected using Connection Objects and must follow the Flow Start Object as their starting point.<br><br>Remember that each flow screen should only contain one in-stream (Flow Start) object to ensure a well-organized and properly functioning designed process.</pre><h3>Performance of Locally Deployed Phi-3-mini Model On Legacy Hardware</h3><p>We deployed int4 quantized version of Phi-3-mini model to a machine with a Intel® Core™ i7–8850H CPU using llama.cpp Python bindings. For the example use cases above token counts and response times are shown in the Response Time Table below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/376/1*b1hE36YBMb8W59cOmytcLg.png" /><figcaption>Response Time Table</figcaption></figure><h3>Conclusion</h3><p>As seen in the use case example results, Phi-3-mini is not performing as good as GPT-3.5 at Turkish to English Translation task. This can indicate lack of Turkish examples in the training data and results may be improved by fine-tuning the model with more Turkish examples.</p><p>Even though the coding example is fairly simple we can’t ignore that Phi-3-mini’s answer is just same as GPT-3.5’s and this is very promising.</p><p>In the json output example, last item is missing but this can be resolved easily by prompt tuning.</p><p>We received reasonably detailed and on point answer from Phi-3 this answer can be improved by prompt tuning too.</p><p>As seen in the Response Time Table, response times are very reasonable considering generations are done by only using CPU.</p><h3>References</h3><p><a href="https://huggingface.co/papers/2404.14219">Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31e468329a4b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/bimser-tech/a-review-on-microsoft-phi-3-pocket-sized-gpt-3-5-31e468329a4b">A review on Microsoft Phi-3: Pocket-sized GPT-3.5</a> was originally published in <a href="https://medium.com/bimser-tech">bimser.tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting Started with the OpenAI API (ChatGPT) in Python]]></title>
            <link>https://gokhangerdan.medium.com/getting-started-with-the-openai-api-chatgpt-in-python-d689eecbbd37?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/d689eecbbd37</guid>
            <category><![CDATA[code-refactoring]]></category>
            <category><![CDATA[gpt-3-vs-gpt-4]]></category>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[openai-api]]></category>
            <category><![CDATA[openai-chatgpt]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Thu, 26 Oct 2023 01:21:46 GMT</pubDate>
            <atom:updated>2023-10-26T01:21:46.018Z</atom:updated>
            <content:encoded><![CDATA[<h4>Unlocking the Power of the OpenAI API: Models, Chat Completions, Function Calls, Embeddings, Token Management, and Best Practices — A Blend of Official Documentation and My Personal Insights</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jVBBycBK5znLN2rMGqet7Q.jpeg" /></figure><h3>Models</h3><p>OpenAI offers a range of models tailored to various tasks, making it essential to choose the right one for your needs. Let’s explore my top choices:</p><ul><li>gpt-3.5-turbo: This is the cream of the crop when it comes to GPT-3 models. As the very same model powering the free version of ChatGPT, it’s my go-to for straightforward completion tasks. Its ease of use and impressive performance make it a dependable choice.</li><li>gpt-3.5-turbo-16k: Building upon the strengths of its predecessor, this version offers an expanded context limit. Whenever I require more extensive context for my prompts, especially in scenarios where historical context matters, this model shines.</li><li>gpt-4: As OpenAI’s latest and most advanced chat completion model, gpt-4 takes center stage when complexity is the name of the game. With its enhanced understanding and generation capabilities, it excels in delivering in-depth and contextually rich responses.</li></ul><h3>A Practical Comparison: gpt-3.5-turbo vs. gpt-4 in Code Refactoring</h3><p>In this section, we conduct a practical comparison between two OpenAI models, gpt-3.5-turbo and gpt-4, using a simple code refactoring task. Code refactoring is a fundamental aspect of software development, aimed at improving code structure and readability while preserving functionality. Let’s explore how these models perform in this context.</p><p>Before we dive into the results, let’s introduce the Python code we used for this refactoring task:</p><pre>while True:<br>  mass = int(input(&quot;Enter the mass value: &quot;))<br>  if mass &gt; 0:<br>    break<br>while True:<br>  acceleration = int(input(&quot;Enter the acceleration: &quot;))<br>  if acceleration &gt; 0:<br>    break<br>print(&quot;The Force is&quot;, mass * acceleration)</pre><p>Our goal was to refactor this code for better clarity and maintainability.</p><h4>Response from gpt-3.5-turbo</h4><p>gpt-3.5-turbo suggested the following refactoring:</p><pre>while True:<br>    mass = int(input(&quot;Enter the mass value: &quot;))<br>    if mass &gt; 0:<br>        break<br><br>while True:<br>    acceleration = int(input(&quot;Enter the acceleration: &quot;))<br>    if acceleration &gt; 0:<br>        break<br><br>force = mass * acceleration<br>print(&quot;The Force is&quot;, force)</pre><p>This response offers improved readability by introducing a ‘force’ variable, making the code more self-explanatory.</p><h4>Response from gpt-4</h4><p>gpt-4 suggested a more structured approach:</p><pre>def get_positive_input(prompt):<br>    while True:<br>        value = int(input(prompt))<br>        if value &gt; 0:<br>            return value<br><br>mass = get_positive_input(&quot;Enter the mass value: &quot;)<br>acceleration = get_positive_input(&quot;Enter the acceleration: &quot;)<br>print(&quot;The Force is&quot;, mass * acceleration)</pre><p>gpt-4’s response introduces a function for obtaining positive input, enhancing code modularity.</p><h4>Comparison</h4><p>In comparing these responses, gpt-4’s approach stands out for its modularity and structured design, which can be advantageous in larger codebases. However, gpt-3.5-turbo’s response maintains simplicity and readability, which may be preferable for smaller projects or when speed is essential.</p><h4>Conclusion</h4><p>The choice between gpt-3.5-turbo and gpt-4 for code refactoring depends on the project’s size, complexity, and specific needs. Understanding their strengths and differences can help developers make informed decisions when it comes to improving their codebase.</p><h3>Running Prompts and Receiving Completions with Python</h3><p>In this section, I’ll walk you through the process of running a prompt and receiving completions using Python and the OpenAI API. I’ll use the same code refactoring prompt from the previous section as an example.</p><h4>Step 1: Setting Up Your API Key</h4><p>First, make sure you have your OpenAI API key ready. You should store this key securely, such as in an environment variable, before proceeding. If you haven’t obtained your API key yet, visit the OpenAI website to get one.</p><pre>import os<br>import openai<br><br>openai.api_key = os.getenv(&quot;OPENAI_API_KEY&quot;)</pre><h4>Step 2: Crafting Your Prompt</h4><p>To run a prompt, create a message that specifies the user’s role and content. In this example, we’re using the “gpt-3.5-turbo” model, and our prompt is the Python code snippet that we want to refactor.</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>    {<br>      &quot;role&quot;: &quot;user&quot;,<br>      &quot;content&quot;: &quot;Refactor the following Python code:\n\n&#39;&#39;&#39;\nwhile True:\n  mass = int(input(\&quot;Enter the mass value: \&quot;))\n  if mass &gt; 0:\n    break\nwhile True:\n  acceleration = int(input(\&quot;Enter the acceleration: \&quot;))\n  if acceleration &gt; 0:\n    break\nprint(\&quot;The Force is\&quot;, mass * acceleration)\n&#39;&#39;&#39;&quot;<br>    }<br>  ],<br>  temperature=0,<br>  max_tokens=1000,<br>  top_p=1,<br>  frequency_penalty=0,<br>  presence_penalty=0<br>)</pre><h4>Step 3: Handling the Model’s Response</h4><p>After running the prompt, you’ll receive a response from the model. You can access the generated completion by using:</p><pre>response[&quot;choices&quot;][0][&quot;message&quot;][&quot;content&quot;]</pre><p>This content will contain the model&#39;s refactored version of the Python code.</p><h4>Step 4: Customizing Parameters</h4><p>In the code, you can customize various parameters such as temperature, max_tokens, top_p, frequency_penalty, and presence_penalty to adjust the model&#39;s behavior according to your specific requirements.</p><p>By following these steps, you can effectively run prompts and receive completions using Python and OpenAI’s API. This process is versatile and can be adapted for various text generation tasks beyond code refactoring.</p><h3>Function Calls: An Exciting New Feature</h3><p>The OpenAI API now offers a captivating feature: function calls. This feature allows you to receive function call responses with auto-generated parameters, creating a new dimension of interaction with the model. All you need to do is define your functions and describe them to the model. Here’s how it works:</p><h4>Step 1: Import Modules and Set Your API Key</h4><p>Begin by importing the necessary modules and setting your OpenAI API key. If you don’t have an API key, obtain one from OpenAI’s website.</p><pre>import requests<br>import json<br>import openai<br><br>openai.api_key = os.getenv(&quot;OPENAI_API_KEY&quot;)</pre><h4>Step 2: Create Your Function</h4><p>Define a function that you want to use in the interaction with the model. In this example, we’ll create a function called get_exchange for currency conversion.</p><pre>def get_exchange(amount, currency_from, currency_to):<br>    response = requests.get(&quot;https://open.er-api.com/v6/latest/&quot; + currency_from)<br>    exchange_rate = response.json()[&quot;rates&quot;][currency_to]<br>    return amount*exchange_rate</pre><h4>Step 3: Describe the Function to the Model</h4><p>You need to describe your function to the model so it understands how to use it. Provide details about the function’s name, description, and expected parameters.</p><pre>functions = [<br>    {<br>        &quot;name&quot;: &quot;get_currency_exchange&quot;,<br>        &quot;description&quot;: &quot;Calculate the exchange by given currencies&quot;,<br>        &quot;parameters&quot;: {<br>            &quot;type&quot;: &quot;object&quot;,<br>            &quot;properties&quot;: {<br>                &quot;amount&quot;: {<br>                    &quot;type&quot;: &quot;number&quot;,<br>                    &quot;description&quot;: &quot;The amount of the currency&quot;,<br>                },<br>                &quot;currency_from&quot;: {&quot;type&quot;: &quot;string&quot;, &quot;enum&quot;: [&quot;USD&quot;, &quot;EUR&quot;, &quot;GBP&quot;]},<br>                &quot;currency_to&quot;: {&quot;type&quot;: &quot;string&quot;, &quot;enum&quot;: [&quot;USD&quot;, &quot;EUR&quot;, &quot;GBP&quot;]}<br>            },<br>            &quot;required&quot;: [&quot;amount&quot;, &quot;currency_from&quot;, &quot;currency_to&quot;],<br>        },<br>    }<br>]</pre><h4>Step 4: Make a Request for Completions</h4><p>Now, you’re ready to make a request to receive completions from OpenAI. This request includes a message from the user and the functions you’ve described.</p><pre>messages = [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;How much is 100 usd in euro?&quot;}]<br><br>response = openai.ChatCompletion.create(<br>    model=&quot;gpt-3.5-turbo-0613&quot;,<br>    messages=messages,<br>    functions=functions,<br>    function_call=&quot;auto&quot;<br>)</pre><h4>Step 5: Run the Function with Generated Parameters</h4><p>If the response is a function call, you can run the function with auto-generated parameters. Here’s how to do it:</p><pre>response_message = response[&quot;choices&quot;][0][&quot;message&quot;]<br><br>if response_message.get(&quot;function_call&quot;):<br>    available_functions = {<br>        &quot;get_currency_exchange&quot;: get_currency_exchange,<br>    }<br>    function_name = response_message[&quot;function_call&quot;][&quot;name&quot;]<br>    function_to_call = available_functions[function_name]<br>    function_args = json.loads(response_message[&quot;function_call&quot;][&quot;arguments&quot;])<br>    function_response = function_to_call(<br>        amount=function_args.get(&quot;amount&quot;),<br>        currency_from=function_args.get(&quot;currency_from&quot;),<br>        currency_to=function_args.get(&quot;currency_to&quot;)<br>    )<br>    print(function_response)</pre><h4>Conclusion: Elevating Conversations with Function Calls</h4><p>Function calls in the OpenAI API bring a game-changing dimension to interaction with AI models. This feature allows you to seamlessly integrate custom functions, enabling dynamic and context-aware conversations. As you define and describe functions, the model can understand and incorporate them, offering endless possibilities for automation, answering questions, and enhancing user experiences. With function calls, the OpenAI API opens doors to innovative solutions across a spectrum of applications, pushing the boundaries of AI-driven interactions.</p><h3>Exploring the Horizon: What’s Next</h3><p>In the coming posts, we’ll delve deeper into the vast capabilities of the OpenAI API. We’ll unlock the potential of embeddings, unravel the intricacies of token management for fine-tuned control, and share best practices to harness the full power of these AI models. Stay tuned for insightful discussions and hands-on tutorials as we journey through the realm of AI-driven innovation. Whether you’re a developer, a data scientist, or just curious about the future of AI, there’s something here for everyone. Join us in this exciting exploration of AI’s boundless possibilities.</p><h3>Stay Connected</h3><p>If you have questions, ideas, or simply want to chat about AI, feel free to reach out on <a href="https://twitter.com/gokhangerdan">Twitter</a>. I’m always eager to engage in meaningful conversations with fellow enthusiasts. For more AI insights and articles, don’t forget to follow me on Medium as well. Your support and engagement are greatly appreciated.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d689eecbbd37" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to add feedbacks to Docusaurus?]]></title>
            <link>https://gokhangerdan.medium.com/how-to-add-feedbacks-to-docusaurus-df18559ce59f?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/df18559ce59f</guid>
            <category><![CDATA[sqlite]]></category>
            <category><![CDATA[feedback]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[fastapi]]></category>
            <category><![CDATA[docusaurus]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Sun, 12 Feb 2023 01:35:41 GMT</pubDate>
            <atom:updated>2023-02-12T01:44:03.285Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G3Q6uh2SB5jjH0XmOcmDNg.png" /></figure><p>If you wondered how to add a simple feedback functionality to your docusaurus site, you may seen some implementations powered by github. But in this tutorial I’ll show you my way of doing it by creating a simple api with python. If you would like to have a feedback functionality in your docusaurus site too or if you just want to learn how to customize your docusaurus site further, you may enjoy this tutorial.</p><p>In this tutorial I tried to briefly explain step-by-stephow to connect my docusaurus-feedback-backend to any docusaurus site. You don’t need the backend code to follow this tutorial but after you implemented the frontend you need to run the backend to make this really work. Here is the github link for the backend:</p><p><a href="https://github.com/gokhangerdan/docusaurus_feedback_backend">https://github.com/gokhangerdan/docusaurus_feedback_backend</a></p><p>Run command below to get the code for document page to eject our custom feedback component and follow the steps below.</p><pre>npm run swizzle</pre><h4>Step 1:</h4><p>Select theme classic from the options</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/541/1*qj2PjCjFNaxPWIoWwwqnDA.png" /></figure><h4>Step 2:</h4><p>Select DocPage/Layout/Main from the options</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/472/1*jY3BhawNJmtIz5mvrBNVVA.png" /></figure><h4>Step 3:</h4><p>Select Eject from the options</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*g0VvqUgZYMPrFSLtrt-CdQ.png" /></figure><h4>Step 4:</h4><p>Select YES from the options</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*bOK6hb9Lnnfthw1hsLxihw.png" /></figure><p>You’ll see these two files (index.js, styles.module.css) created under src/theme/DocPage/Layout/Main:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*bEkaH8SMeJYNnbM5yv-xfw.png" /></figure><p>Create a folder named Feedback and a file named index.js in it as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/251/1*V0ewre-Ugvq3wQqHVqICKg.png" /></figure><p>Install react-icons library:</p><pre>npm install react-icons --save</pre><p>Copy-paste the following code to src/componnets/Feedback/index.js</p><pre>import React, { useEffect, useState } from &#39;react&#39;;<br>import { IconContext } from &quot;react-icons&quot;;<br>import { FaThumbsUp, FaThumbsDown } from &#39;react-icons/fa&#39;;<br>const fetch = require(&quot;node-fetch&quot;);<br><br>export default function Feedback() {<br>    const [feedback, setFeedback] = useState({&quot;positive_feedback&quot;: 0, &quot;negative_feedback&quot;: 0});<br><br>    let url = window.location.pathname;<br>    useEffect(() =&gt; {<br>        var myHeaders = new Headers();<br>        myHeaders.append(&quot;Content-Type&quot;, &quot;application/json&quot;);<br>    <br>        var raw = JSON.stringify({<br>        &quot;url&quot;: url<br>        });<br>    <br>        var requestOptions = {<br>        method: &#39;POST&#39;,<br>        headers: myHeaders,<br>        body: raw,<br>        redirect: &#39;follow&#39;<br>        };<br>    <br>        fetch(&quot;http://localhost:8000/api/notes/&quot;, requestOptions)<br>        .then(response =&gt; response.json())<br>        .then(result =&gt; {<br>            console.log(result)<br>            setFeedback(result)<br>        })<br>        .catch(error =&gt; console.log(&#39;error&#39;, error));<br>    }, [url]);<br>    url = window.location.pathname;<br><br>    function newFeedback(val) {<br>        var myHeaders = new Headers();<br>        myHeaders.append(&quot;Content-Type&quot;, &quot;application/json&quot;);<br><br>        var raw = JSON.stringify({<br>        &quot;url&quot;: url,<br>        &quot;feedback&quot;: val<br>        });<br><br>        var requestOptions = {<br>        method: &#39;POST&#39;,<br>        headers: myHeaders,<br>        body: raw,<br>        redirect: &#39;follow&#39;<br>        };<br><br>        fetch(&quot;http://localhost:8000/api/notes/new&quot;, requestOptions)<br>        .then(response =&gt; response.json())<br>        .then(result =&gt; {<br>            console.log(result)<br>            setFeedback({&quot;positive_feedback&quot;: result.positive_feedback, &quot;negative_feedback&quot;: result.negative_feedback})<br>        })<br>        .catch(error =&gt; console.log(&#39;error&#39;, error));<br>    }<br><br>    return (<br>        &lt;&gt;<br>            &lt;IconContext.Provider value={{ color: &quot;blue&quot;, size: &quot;2em&quot;, style: {margin: &#39;.5em&#39;} }}&gt;<br>                &lt;FaThumbsUp onClick={() =&gt; newFeedback(1)}/&gt;<br>                &lt;span&gt;{feedback.positive_feedback}&lt;/span&gt;<br>            &lt;/IconContext.Provider&gt;<br>            &lt;IconContext.Provider value={{ color: &quot;red&quot;, size: &quot;2em&quot;, style: {margin: &#39;.5em&#39;} }}&gt;<br>                &lt;FaThumbsDown onClick={() =&gt; newFeedback(-1)}/&gt;<br>                &lt;span&gt;{feedback.negative_feedback}&lt;/span&gt;<br>            &lt;/IconContext.Provider&gt;<br>        &lt;/&gt;<br>    );<br>}</pre><p>And edit src/theme/DocPage/Layout/Main/index.js as shown below:</p><pre>import React from &#39;react&#39;;<br>import clsx from &#39;clsx&#39;;<br>import {useDocsSidebar} from &#39;@docusaurus/theme-common/internal&#39;;<br>import styles from &#39;./styles.module.css&#39;;<br>import Feedback from &#39;@site/src/components/Feedback&#39;<br><br>export default function DocPageLayoutMain({hiddenSidebarContainer, children}) {<br>  const sidebar = useDocsSidebar();<br>  return (<br>    &lt;&gt;<br>      &lt;Feedback /&gt;<br>      &lt;main<br>        className={clsx(<br>          styles.docMainContainer,<br>          (hiddenSidebarContainer || !sidebar) &amp;&amp; styles.docMainContainerEnhanced,<br>        )}&gt;<br>        &lt;div<br>          className={clsx(<br>            &#39;container padding-top--md padding-bottom--lg&#39;,<br>            styles.docItemWrapper,<br>            hiddenSidebarContainer &amp;&amp; styles.docItemWrapperEnhanced,<br>          )}&gt;<br>          {children}<br>        &lt;/div&gt;<br>      &lt;/main&gt;<br>    &lt;/&gt;<br>  );<br>}</pre><p>And this is it. A simple solution. Now you can run the following command to see how it looks like:</p><pre>npm run start</pre><p>Open a document in your docusaurus site and you should see up vote and down vote icons with small numbers at top left of your documents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3cwHJYTvqeeuR_ZcKF3IVw.png" /></figure><p>To make this custom compenent really work you must run the backend application too. You can use commands below:</p><pre>git clone https://github.com/gokhangerdan/docusaurus_feedback_backend<br>cd docusaurus_feedback_backend<br>python3 -m venv env<br>source env/bin/activate<br>pip install -r requirements.txt<br>uvicorn app.main:app --reload</pre><p>And now you can click on the up vote and down vote icons to give feedback to documents as you can see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ddhzcwuGtcqy6-RrWVdlcQ.png" /></figure><p>I hope this helped you for whatever you were looking for. This is just my simple solution for a problem I had. If you have any questions or recommendations feel free to ask.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df18559ce59f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Create A Text Classifier: Online/Incremental Learning With Creme-ML]]></title>
            <link>https://gokhangerdan.medium.com/how-to-create-a-text-classifier-online-incremental-learning-with-creme-ml-6aac9d869e5c?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/6aac9d869e5c</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[creme]]></category>
            <category><![CDATA[incremental-learning]]></category>
            <category><![CDATA[online-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Sun, 17 May 2020 13:14:22 GMT</pubDate>
            <atom:updated>2020-11-06T21:06:28.015Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*nMPKdrhB1cy7CCoDRiRuAg.jpeg" /></figure><p>In this tutorial we are going to train a text classifier model that is capable of continuously learn without retraining with all the data again and again. This is called online/incremental learning. This method also called batch learning as you can get it from the name it is useful for training when we have very large amount of data. We can simply divide our data into small chunks and train our model chunk by chunk incrementally.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BmG86iYckF2jiPnRy2s3UQ.png" /><figcaption>Online/Incremental Learning (a diagram just for fun)</figcaption></figure><p>In the classical way of training a machine learning model we use all the data we have and when we receive more data we combine the new data with the old data and retrain the model completely from scratch unlike incremental learning. So we can use this to train and continuously evolve our model with realtime flowing data.</p><p>In this tutorial we are going to use Python to train our model. But instead of using Scikit-Learn we are going to use <a href="https://creme-ml.github.io/"><strong><em>Creme-Ml</em></strong></a> module for incremental learning capabilities. Let’s install this module:</p><pre>pip install creme</pre><p>We are going to use <a href="https://www.kaggle.com/savasy/ttc4900"><strong><em>this dataset</em></strong></a> as an example. You can check out <a href="https://creme-ml.github.io/content/api.html"><strong><em>api reference</em></strong></a> section from Creme-ML docs for other functionalities but in this example I will guide you through what make me most excited about Creme-ML and it is <a href="https://creme-ml.github.io/content/generated/creme.compose.Pipeline.html#creme.compose.Pipeline"><strong><em>Pipeline</em></strong></a> object.</p><p>Using Pipeline object of Creme we can compose our preprocessing, learning… etc. steps in a single Python object just like this:</p><pre><strong>import pandas as pd<br>from</strong> <strong>creme</strong> <strong>import</strong> compose<br><strong>from</strong> <strong>creme</strong> <strong>import</strong> feature_extraction<br><strong>from</strong> <strong>creme</strong> <strong>import</strong> naive_bayes</pre><pre># Read dataset<br>df = pd.read_csv(&quot;7allV03.csv&quot;)</pre><pre># Convert dataframe to list of tuples<br>docs = df.to_records(index=False)</pre><pre># Creating the pipeline<br># 1st function is creating the bag of words<br># 2nd function is the naive bayes predictor<br>model = compose.Pipeline(<br>    (&#39;tokenize&#39;, feature_extraction.BagOfWords(lowercase=<strong>False</strong>)),<br>    (&#39;nb&#39;, naive_bayes.MultinomialNB(alpha=1))<br>)</pre><pre># Training the model row by row<br><strong>for</strong> sentence, label <strong>in</strong> docs:<br>    model = model.fit_one(sentence, label)</pre><p>Now we successfully trained our model. You can make predictions like this:</p><pre>model.predict_one(&quot;text to predict category&quot;)</pre><p>Or you can increment the model like this:</p><pre>model = model.fit_one(&quot;new text to increment model&quot;, &quot;label for new text&quot;)</pre><p>There you have a machine learning model to classify news. And you can increment this model without needing the old data.</p><p>I hope this tutorial works for you. Just tell me if you have any problem following.</p><p>Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6aac9d869e5c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Knime Workflowlarınızı Kendi Sunucunuza Deploy Edin — [Knime][Python][Linux]]]></title>
            <link>https://gokhangerdan.medium.com/knime-workflowlar%C4%B1n%C4%B1z%C4%B1-kendi-sunucunuza-deploy-edin-knime-python-linux-e99b656b9d40?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/e99b656b9d40</guid>
            <category><![CDATA[knime-server]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[knimepy]]></category>
            <category><![CDATA[knime]]></category>
            <category><![CDATA[deploy-knime-workflow]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Thu, 09 Jan 2020 12:14:04 GMT</pubDate>
            <atom:updated>2020-01-27T10:23:29.798Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0mZtRU8N15zQ-j5tdkLWPg.jpeg" /><figcaption><a href="https://commons.wikimedia.org/wiki/File:A_scenic_view_of_clouds_from_plane_window.jpg">https://commons.wikimedia.org/wiki/File:A_scenic_view_of_clouds_from_plane_window.jpg</a></figcaption></figure><h3>Knime Workflowlarınızı Kendi Sunucunuza Deploy Edin — [Knime][Python][Linux]</h3><p>Bu yazımda Knime ile çalışanlar için hazırladıkları workflowları kendi sunucularında hiçbir ücret ödemeden nasıl barındırabileceklerini ve bir REST API aracılığıyla nasıl kullanabileceklerini anlatacağım.</p><p>Bildiğiniz gibi Knime’ın Analytics Platform ve Server isimlerinde iki farklı yazılımı var.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*5qjBnUMr7G6B11YiAZmpHw.png" /><figcaption><a href="https://www.knime.com/software-overview">https://www.knime.com/software-overview</a></figcaption></figure><p>Knime Analytics Platform herkesin bildiği masaüstü ortamında kullanılan hiç kod yazmaya gerek olmadan görsel modeller ile ETL, ML gibi işlerinizi tasarlayıp yapabileceğiniz bir uygulamadır. Burada örneğin bir ML modelinizi eğitip PMML formatında dışa aktardıktan sonra mesela Java ile yazdığınız bir mikro servis içerisinde kullanarak istediğiniz yere deploy edip tahminlerinizi yaptırabilirsiniz. Ama bu durumda verinin temizlik, ön işleme gibi işlemlerini de bu servis içerisinde kodlamanız gerekebilir. Öyleyse ne anladık Knime’ın sıfır kod ile iş yapma avantajından. Tabi bu durumda Knime Server kullanabilirsiniz. Knime Server da, Analytics Platformda eğittiğiniz modelleri deploy edip REST API ile bulutta kullanabilmenize imkan sağlayan bir yazılımdır. Tabi bu imkanın yanında bir sürü özelliği daha vardır. Knime Analytics Platformun açık kaynak kodlu ücretsiz bir yazılım olduğunu biliyoruz. Peki ya Knime Server?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/919/1*SqfXjJL1mVNBjMmCFs6kbw.png" /><figcaption><a href="https://www.knime.com/knime-software/knime-server-pricing">https://www.knime.com/knime-software/knime-server-pricing</a></figcaption></figure><p>Malesef Knime Server bazılarımız için biraz tuzlu olabilir. En düşük paketine bile ayda ortalama 1000 dolar ödemek durumunda kalabilirsiniz.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4NzRVyhAq9KCHr5aBq11yw.png" /><figcaption><a href="https://azuremarketplace.microsoft.com/en-us/marketplace/apps/knime.knime-server-small?tab=PlansAndPrice">https://azuremarketplace.microsoft.com/en-us/marketplace/apps/knime.knime-server-small?tab=PlansAndPrice</a></figcaption></figure><p>Bu ücretleri ödemek istemeyenler için Knime Analytics Platform uygulamasını batch mod ile sunucuda çalıştırıp Python ile yazılımış bir REST API aracılığıyla nasıl kontrol edilebileceğini ve bu yöntemin avantajlarını ve dezavantajlarını adım adım anlatacağım.</p><h3>Uygulama Adımları:</h3><ol><li>Ubuntu Server 18.04 Üzerinde Knime Kurulumu</li><li>Örnek Bir Workflowun Hazırlanması</li><li>Worflowun Sunucuda Çalıştırılıp Ortamın Test Edilmesi</li><li>Uzaktan Kontrol İçin Python İle Basit Bir REST API Hazırlanması</li></ol><h4>1. Ubuntu Server 18.04 Üzerinde Knime Kurulumu</h4><p>SSH ile bağlandığımız sunucuda aşağıdaki komutu çalıştırarak Knime 4.0.0 sürümünü indirelim:</p><blockquote><strong>ÖNEMLİ UYARI:</strong> Workflowu oluşturduğunuz Knime sürümü ile sunucuya indireceğiniz Knime sürümü aynı olmak zorundadır. Bilgisayarınızdaki knime 4.0.1 ise sunucunuzdaki Knime 4.0.0 olamaz. Sunucudaki de 4.0.1 olmak zorundadır. Aksi halde workflowunuz çalışmaz ve hata alırsınız.</blockquote><pre>wget https://download.knime.org/analytics-platform/linux/knime_4.0.1.linux.gtk.x86_64.tar.gz</pre><p>İndirdiğimiz sıkıştırılmış dosyayı aşağıdaki komut ile dışarı çıkarabiliriz:</p><pre>tar -xvf knime_4.0.1.linux.gtk.x86_64.tar.gz</pre><p>Şimdi de sıkıştırılmış dosyadan kurtulabilirsiniz.</p><pre>rm knime_4.0.1.linux.gtk.x86_64.tar.gz</pre><h4>2. Örnek Bir Workflowun Hazırlanması</h4><p>Bu aşamada basit bir workflow hazırlamanız gerekiyor. Örnek olarak aşağıda gördüğünüz workflowu kullanabilirsiniz. <a href="https://yadi.sk/d/QM6qwGpaTzGwRg">Bu bağlantıdan indirebilirsiniz.</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*00MEnMjWC7HqSW4AQmTZng.png" /></figure><p>Burada önemli olan iki node var bunlar <strong><em>Container Input (Table)</em></strong> ve <strong><em>Container Output (Table). </em></strong>Bu nodlar batch modda çalıştıracağımız workflowa dışarıdan veri alıp vermemizi sağlayacak. Gerekli olduğu kadar input ve output nodları kullanılabilir.</p><h4>3. Workflowun Sunucuda Çalıştırılıp Ortamın Test Edilmesi</h4><p>Bu bölümde hazırladığımız workflowu sunucuya taşıyıp aşağıdaki script ile çalıştırarak çıktısını ekrana yazdıracağız. Knime ve workflowa ait pathleri kendinize göre düzenlemeyi unutmayın.</p><p>Bu scripti çalıştırmadan önce gerekli kütüphaneleri (<a href="https://github.com/pandas-dev/pandas"><em>pandas</em></a>, <a href="https://github.com/knime/knimepy"><em>knime</em></a>) aşağıdaki şekilde yükleyebilirsiniz. Python3 ve pip3 araçlarının yüklü olduğunu varsayıyorum.</p><pre>pip3 install pandas</pre><pre>pip3 install knime</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f8cca36d406b8dffc162489a1d62f4e9/href">https://medium.com/media/f8cca36d406b8dffc162489a1d62f4e9/href</a></iframe><p>Yukarıda da paylaştığım <a href="https://yadi.sk/d/QM6qwGpaTzGwRg">bu linkten</a> indirdiğiniz örnek workflowu scp kullanarak aşağıdaki komut ile sunucunuza gönderebilirsiniz:</p><pre>scp -r knimepy mluser@192.168.41.181:/home/mluser</pre><p>Şimdi sunucuda herşey hazır olduğuna göre yukarıdaki scripti nano ile test.py isimli bir dosya açıp sunucumuzun <em>/home/mluser/test.py</em> dizininde olacak şekilde kaydedelim.</p><p>Sunucumuz içerisinde oluşturduğumuz dizin ve dosyalar aşağıdaki gibi olacaktır:</p><pre>- home                       # Home dizini<br>    - mluser                 # Kullanıcı Dizininiz<br>        - knime_4.0.1        # Knime Analytics Platform<br>        - knimepy            # Örnek Workflow<br>        - test.py            # Ortamı Test Etmek İçin Script</pre><p>Burada <em>test.py</em> scriptini aşağıdaki şekilde çalıştıralım:</p><pre>python3 test.py</pre><p>Eğer aşağıdaki çıktıyı görüyorsanız artık sunucunuzda Python ile Knime workflowlarını çalıştırabiliyorsunuz demektir:</p><pre>  prediction<br>0       male</pre><h4>4. Uzaktan Kontrol İçin Python İle Basit Bir REST API Hazırlanması</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/615/1*TCHA2UrhBrR0nN_WADrODg.png" /><figcaption><a href="https://falconframework.org/">https://falconframework.org/</a></figcaption></figure><p>Python programlama dili ile REST API oluşturmak için hazırlanmış bir sürü modül ile karşılaşmış olabilirsiniz. Bunlardan en popülerleri olan Django-REST (Django) ve Flask-RESTful (Flask) modüllerini duymuşsunuzdur. Genelde bu tip örneklerde Flask kullanıldığını görüyorum ve bir değişiklik olması açısından aynı zamanda da daha yüksek performanslı bir framework olduğundan ben bu örneği <a href="https://falconframework.org/">Falcon Framework</a> ile hazırladım.</p><p>Hazırladığım örnek apiyi çalıştırabilmek için aşağıdaki komutlar ile gereksinimleri yükleyin:</p><pre>apt install gunicorn3</pre><pre>pip3 install falcon</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dad78b58ec4e2c099a3ea05b853563ca/href">https://medium.com/media/dad78b58ec4e2c099a3ea05b853563ca/href</a></iframe><p>Yukarıdaki örneği <em>test.py</em> dosyası ile aynı yere <em>api.py</em> olarak kaydedin içerisindeki dosya yollarını gerektiği şekilde değiştirin ve aşağıdaki komut ile çalıştırın:</p><pre>gunicorn3 --bind 0.0.0.0:8000 api:api</pre><blockquote>EK BİLGİ [27.01.2020]: Bu şekilde çalıştırdığınızda tek seferde bir isteğe cevap verecektir. ard arda gönderilen istekleri sıraya alacaktır. Gunicorn ile birden fazla isteğe aynı anda cevap verebilmek için aşağıdaki şekilde çalıştırmanız gerekecetir. Ama bu sefer de bir knime workflow dosyası birden fazla knime uygulamasında kullanılamayacağından yalnızca ilk isteğe verdiği cevapta sonuç döndürebilecektir. Bu problemi aşmak için her request için workflow dosyasını çoğaltıp boştaki workflow dosyasına yönlendirebilecek bir nevi kendi custom load balancerınızı yazmanız gerekebilir.</blockquote><pre>gunicorn3 --bind 0.0.0.0:8000 api:api --threads 3 --workers 3</pre><p>Burada pandas kütüphanesi de apiye gelen json veriyi tabloya ve knimedan dönen tabloyu da tekrar jsona dönüştürmek için kullanıldı.</p><p>Aşağıdaki gibi bir REST client (<a href="https://insomnia.rest/">Insomnia</a>) kullanarak siz de oluşturduğunuz apiyi test edebilirsiniz:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*_G-_AfhiEpbOSPRarQBwEQ.png" /></figure><h3>Sonuç</h3><h4>Avantajlar:</h4><ul><li>Hiç kod yazmadan Knime ile hazırladığınız ETL, ML, DL workflowlarınızı sunucunuza deploy edip çalıştırabilmek projeleri epey hızlandırır. Kod ile uğraşmaktansa asıl probleme odaklanabilirsiniz.</li><li>Hazırladığınız workflowlar projenizi daha anlaşılır ve sürdürülebilir hale getirir.</li><li>Knime yeterli olmadığı durumda her zaman işinizin bir kısmını Python tarafına çekebilirsiniz ve bu iki aracı hibrit olarak kullanabilirsiniz.</li></ul><h4>Dezavantajlar:</h4><ul><li>Malesef yavaş çalışıyor. Mesela bu uygulamada sonucu döndürmesi 13 saniye sürdü ki aynı işi tamamen python ile yapmış olsaydık 1 saniyenin çok altında bir sürede sonucu döndürebilirdik. Bu yavaşlığın sebebi de her requestte workflowu tekrar yüklemesi gibi görünüyor. Bu konu üzerine düşüneceğim ve en yakın zamanda hızlandırılmış verisyonu hakkındaki yazımı paylaşacağım. Beklemede kalın.</li><li>Bir diğer dezavantajı ise en başta pahalı dediğim Knime Serverdansa bu şekilde kullanıldığı durumda sunucuyu ve ihtiyaç duyulan servisleri işletmek için tecrübe ve zaman gerekecek. Eğer bir işletme iseniz ve bu tip bir uygulamayı barındırıp sürdürebilecek teknik bir ekibiniz yok ise bu sizi zorlayabilir.</li></ul><p>Bu yazımda Knime ile hazırlamış olduğunuz workflowlarınızı kendi sunucunuzda nasıl çalıştırabileceğinizi elimden geldiğince anlattım. Bu konu hakkında desteğe ihtiyaç duyarsanız aşağıdaki adresten benimle iletişime geçebilirsiniz.</p><p><a href="https://www.linkedin.com/in/gokhangerdan/">Gökhan Gerdan<br>gokhang1327@gmail.com</a></p><h3>Knime ile İlgili Diğer Yazılarım</h3><ul><li><a href="https://www.veribilimiokulu.com/knime-ve-zemberek-nlp-ile-turkce-dokuman-siniflandirma/">Knime ve Zemberek NLP ile Türkçe Döküman Sınıflandırma - Veri Bilimi Okulu</a></li><li><a href="https://www.veribilimiokulu.com/knimeda-egitilen-modelin-farkli-programlama-dillerinde-kullanilmasi-python-java/">Knime&#39;da Eğitilen Modelin Farklı Programlama Dillerinde Kullanılması (Python, Java) - Veri Bilimi Okulu</a></li><li><a href="https://medium.com/@gokhang1327/knimeda-e%C4%9Fitilen-makine-%C3%B6%C4%9Frenimi-modelinin-web-servis-olarak-yay%C4%B1nlanmas%C4%B1-749a54abb48">Knime’da Eğitilen Makine Öğrenimi Modelinin Web Servis Olarak Yayınlanması</a></li><li><a href="https://medium.com/@gokhang1327/yeni-ba%C5%9Flayanlar-i%C3%A7in-knime-ile-kaggle-yar%C4%B1%C5%9Fmalar%C4%B1na-kat%C4%B1lma-rehberi-2c8e6b5c683e">Yeni Başlayanlar için Knime ile Kaggle Yarışmalarına Katılma Rehberi</a></li><li><a href="https://medium.com/@gokhang1327/ger%C3%A7ek-hayat-uygulamalar%C4%B1yla-veri-bilimi-al%C4%B1%C5%9F-veri%C5%9F-merkezi-m%C3%BC%C5%9Fteri-segmentasyonu-knime-ile-a6dbca8457d8">Gerçek Hayat Uygulamalarıyla Veri Bilimi: Alış Veriş Merkezi Müşteri Segmentasyonu (Knime ile)</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e99b656b9d40" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Avoid Overfitting [For Beginners] [Deep Learning]]]></title>
            <link>https://gokhangerdan.medium.com/how-to-avoid-overfitting-for-beginners-deep-learning-ed2817a7e65?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed2817a7e65</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[beginners-guide]]></category>
            <category><![CDATA[overfitting]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Mon, 09 Dec 2019 10:52:41 GMT</pubDate>
            <atom:updated>2019-12-09T13:08:56.873Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*UWRgjmwN36lj7LqfBd5DxQ.png" /></figure><h3>How To Avoid Overfitting [For Beginners] [Deep Learning] [Keras]</h3><p>Best way to avoid overfitting is to avoid using deep learning and use random forest. <a href="https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#remarks"><strong>But why random forest?</strong></a> Random forest doesn’t overfit? Ofcourse random forest overfits too. But if you increase the number of trees overfitting should decrease.</p><p>Neural networks tends to overfit more than classical machine learning algorithms. But what are simple techniques to prevent overfitting when training neural networks for beginners.</p><p><a href="https://www.kaggle.com/gokhangerdan/how-to-avoid-overfitting-for-beginners-keras?scriptVersionId=24794383"><strong>[UPDATE] Check this out. I created a Kaggle kernel for this post as an example!!!</strong></a></p><h3>Less layers, less neurons =&gt; Less overfitting</h3><p>If you simplify your model by decreasing number of layers and number of neurons per layer you can decrease overfitting but prediction accuracy can also decrease.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*iJdWyPxG4t8wihoFaTEf9w.png" /></figure><h3>Add dropout layers to generalize your model</h3><p>Dropout layers randomly drops some of connections between layers and that decrease overfitting by randomness of the selected connections.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*o0n-gsPrNHu1Ly1TtYu1RA.gif" /></figure><h3>Stopping earlier than initial epochs</h3><p>Stop learning process when validation loss starts rising up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*HQlqqeS583CQWbLPAE0HhQ.png" /></figure><h3>Conclusion</h3><p>These are just basic techniques to prevent your first neural network from overfitting. You can test these techniques easily using <a href="https://keras.io/"><strong>keras</strong></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed2817a7e65" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Derin Öğrenme Notları: Convolution, Non-linearity, Pooling (numpy ile uygulama)]]></title>
            <link>https://gokhangerdan.medium.com/derin-%C3%B6%C4%9Frenme-notlar%C4%B1-convolution-non-linearity-pooling-numpy-ile-uygulama-3143ea0cf8d6?source=rss-4789f7c1f7cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/3143ea0cf8d6</guid>
            <category><![CDATA[derin-öğrenme]]></category>
            <category><![CDATA[convolution]]></category>
            <category><![CDATA[pooling]]></category>
            <category><![CDATA[numpy]]></category>
            <category><![CDATA[relu]]></category>
            <dc:creator><![CDATA[Gökhan Gerdan]]></dc:creator>
            <pubDate>Fri, 23 Aug 2019 14:39:08 GMT</pubDate>
            <atom:updated>2019-08-23T14:39:08.214Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F2Ik_XFzmu5jZF-byiAKQQ.jpeg" /><figcaption>Image source: <a href="https://miro.medium.com/max/2510/1*vkQ0hXDaQv57sALXAJquxA.jpeg">https://miro.medium.com/max/2510/1*vkQ0hXDaQv57sALXAJquxA.jpeg</a></figcaption></figure><p>Merhaba, bu yazımda CNN’in işlem adımlarından Convolution, Non-linearity ve Pooling’in aslında nasıl bir işlem yaptığından adım adım kısaca bahsedeceğim. CNN nedir ve ne işe yarar gibi sorularınız varsa eğer bu yazıda aradığınız cevapları bulamayabilirsiniz.</p><h4>Ön bilgi gereksinimleri (uygulama kısmı için):</h4><ul><li>Python (numpy)</li></ul><h4>Convolution</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c6dab3f786792ef9c335d9c47c992687/href">https://medium.com/media/c6dab3f786792ef9c335d9c47c992687/href</a></iframe><h4>Non-linearity (RELU)</h4><p>Bu adımda bir önceki adımdan elde ettiğimiz matrise RELU aktivasyon fonksiyonu uygulayacağız. Bu adım en kolay adım olduğundan kod ekleme gereği duymadım. Bu adımda yapılacak işlem bir döngü içerisinde tüm elemanları kontrol edip 0&#39;dan küçük olanların yerine 0 yazmak. İşte bu kadar kolay.</p><h4>Pooling (Downsampling)</h4><p>Anlatacaklarım içerisinde son adım olan bu adım da RELU’dan sonra ikinci en kolay adımdır. İlk adımda yaptığımız gibi bir önceki adımdan çıkan sonuç matrisi üzerinde bu sefer 2x2 bir filtre gezdirip her adımdaki maksimum değeri alıp yeni bir matris türeteceğiz ve pooling işlemi bitmiş olacak.</p><h4>Tüm adımların kodunu içeren jupyter notebook:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/249ae122d3387e44ba15899cca5e06d0/href">https://medium.com/media/249ae122d3387e44ba15899cca5e06d0/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3143ea0cf8d6" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>