<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yunus Emre DEMİRDAĞ on Medium]]></title>
        <description><![CDATA[Stories by Yunus Emre DEMİRDAĞ on Medium]]></description>
        <link>https://medium.com/@yunusemredemirdag?source=rss-5c9eeec91e57------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*c9LC7I6OPPWcUTsXiWVH4g.jpeg</url>
            <title>Stories by Yunus Emre DEMİRDAĞ on Medium</title>
            <link>https://medium.com/@yunusemredemirdag?source=rss-5c9eeec91e57------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:42:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yunusemredemirdag/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Normalization With Python ( Step-1 )]]></title>
            <link>https://medium.com/text-to-image-comprasion/normalization-with-python-step-1-5bf97601a4b2?source=rss-5c9eeec91e57------2</link>
            <guid isPermaLink="false">https://medium.com/p/5bf97601a4b2</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[normalization]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Yunus Emre DEMİRDAĞ]]></dc:creator>
            <pubDate>Sat, 09 Dec 2023 21:52:44 GMT</pubDate>
            <atom:updated>2023-12-24T13:46:00.218Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Normalization?</h3><p>Normalization is a data preprocessing technique used to adjust the values of features in a dataset to a common scale. Normalizasyonun amacı modellerin daha performans göstermesini sağlamaktır. Normalizasyon, verilerin varyansını azaltır ve verilerin dağılımını daha eşit hale getirir.</p><p>There are two main purposes of Normalization:</p><ul><li><strong>Increase the model’s learning speed: </strong>Normalization reduces the variance of the data, allowing the model to learn faster. This is especially important for models trained on large datasets.</li><li><strong>Increase the model’s stability:</strong> Normalization makes the distribution of the data more uniform, making the models more stable. This is especially important for sensitive models.</li></ul><p>There are two main normalization techniques:</p><ul><li><strong>Min-Max normalization:</strong> In this method, each data point is rescaled at between the minimum and maximum points.</li><li><strong>Standarization:</strong> In this method, each data point is rescaled with a mean of 0 and a standard deviation of 1.</li></ul><p>Min-Max normalization effectively reduces variance of the data, but doesn’t change the distribution shape of the data. In addition to reducing the variance of the data, standarization also normalizes the distribution shape of the data.</p><p>In machine learning, which normalization technique will use, it depends to used model type and dataset attributes. Generally, Min-Max normalization more easy and more faster than standarization. Standarization is better option when it is also necessary to normalize the distribution of the data.</p><p>This is an important step to show best performance of Model.</p><p>Normalization also makes the training process less sensitive to the scale of the features. This results in getting better coefficients after training. This process of making features more suitable for training by rescaling is called feature scaling.</p><p>In this article, we will examine the use of standarization methods.</p><h3>STANDARIZATION</h3><p>Standarization is another scaling method where the values are centered around the mean with a unit standard deviation. This means that the mean of the attribute becomes zero, and the resultant distribution has a unit standard deviation.</p><p>Sklearn library will be used for standarization.</p><p>Standarizaiton of datasets is a common requirement for many machine learning estimators implemented in scikit-learn; they might behave badly if the individual features do not more or less look like standard normally distributed data: Gaussian with zero mean and unit variance.</p><p>In practice we often ignore the shape of the distribution and just transform the data to center it by removing the mean value of each feature, then scale it by dividing non-constant features by their standard deviation.</p><p>What is intended to be done in standarization is to make the mean of the data 0 and the standard deviation 1.</p><p>Below is an example code.</p><p>Firstly, the X_train variable is defined. Then tests will do on this array.</p><pre>X_train = np.array([[1., -1., 2.],<br>                    [2., 0., 0.],<br>                    [0., 1., -1.]])</pre><pre>scaler = preprocessing.StandardScaler().fit(X_train)</pre><p>Scaling parameters are learned on the X_train data with StandardScaler.</p><p>Some of the parameters of this scaler to be used in the conversion are shown in the below.</p><pre># Scaling Mean<br>scaler.mean_<br><br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([1., 0., 0.33333333])<br>&quot;&quot;&quot;<br></pre><pre># Scaling parameters<br>scaler.scale_<br><br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([0.81649658, 0.81649658, 1.24721913])<br>&quot;&quot;&quot;<br></pre><p>Using this parameters, a transformation is performed on the given data.</p><p>Let’s scale the X_train data used for teaching.</p><pre>x_scaled = scaler.transform(X_train)<br>x_scaled<br><br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([[ 0.        , -1.22474487,  1.33630621],<br>       [ 1.22474487,  0.        , -0.26726124],<br>       [-1.22474487,  1.22474487, -1.06904497]])<br>&quot;&quot;&quot;</pre><p>The output has been scaled with the scaler and the new scaled output is shown. if we calculate the mean and standard deviation of this data, the results as follows:</p><pre>x_scaled.mean(axis=0)<br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([0., 0., 0.])<br>&quot;&quot;&quot;</pre><pre>x_scaled.std(axis=0)<br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([1., 1., 1.])<br>&quot;&quot;&quot;</pre><p>As seen in the outputs, the mean is 0 and standard deviation 1.</p><h4>Scaling features to a range</h4><p>An alternative standarization is scaling features to lie between a given minimum and maximum value, often between zero and one value, or so that the maximum absolute value of each feature is scaled unit size. This can be achieved using MinMaxScaler or MaxAbsScaler, respectivly.</p><pre>min_max_scaler = preprocessing.MinMaxScaler()<br>x_train_minmax = min_max_scaler.fit_transform(X_train)<br>x_train_minmax<br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([[0.5       , 0.        , 1.        ],<br>       [1.        , 0.5       , 0.33333333],<br>       [0.        , 1.        , 0.        ]])<br><br>---<br>StandardScaler OUTPUT:<br><br>array([[ 0.        , -1.22474487,  1.33630621],<br>       [ 1.22474487,  0.        , -0.26726124],<br>       [-1.22474487,  1.22474487, -1.06904497]])<br>&quot;&quot;&quot;</pre><p>If we take a look at the MinMaxScaler parameters, it is seen in the figure below.</p><pre>min_max_scaler.scale_<br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([0.5       , 0.5       , 0.33333333])<br>&quot;&quot;&quot;<br><br>min_max_scaler.mean_<br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([0.        , 0.5       , 0.33333333])<br>&quot;&quot;&quot;</pre><p>When the test is applied with this scaler, the output is as follows.</p><pre>x_test = np.array([[-3., -1., 4.]])<br>x_test_minmax = min_max_scaler.transform(x_test)<br>x_test_minmax<br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([[-1.5       ,  0.        ,  1.66666667]])<br>&quot;&quot;&quot;</pre><p>If scaling is done with MaxAbsScaler on the same data, the following outputs are obtained.</p><pre>max_abs_scaler = preprocessing.MaxAbsScaler()<br>x_train_maxabs = max_abs_scaler.fit_transform(X_train)<br>x_train_maxabs<br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([[ 0.5, -1. ,  1. ],<br>       [ 1. ,  0. ,  0. ],<br>       [ 0. ,  1. , -0.5]])<br>&quot;&quot;&quot;<br><br>x_test_maxabs = max_abs_scaler.transform(x_test)<br>x_test_maxabs<br>&quot;&quot;&quot;<br>OUTPUT:<br><br>array([[-1.5, -1. ,  2. ]])<br>&quot;&quot;&quot;</pre><p>As a result, these standardization techniques do not improve the performance of the model when not applied in the right place, but rather worsen it. Therefore, it is important to use the right technique in the right place. More about this will be explained in future posts. Happy reading…</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5bf97601a4b2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/text-to-image-comprasion/normalization-with-python-step-1-5bf97601a4b2">Normalization With Python ( Step-1 )</a> was originally published in <a href="https://medium.com/text-to-image-comprasion">Research Technologies</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Comprasion of TextToImage Models]]></title>
            <link>https://medium.com/text-to-image-comprasion/comprasion-of-texttoimage-models-7f88f7b4809f?source=rss-5c9eeec91e57------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f88f7b4809f</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[text-to-image-generation]]></category>
            <dc:creator><![CDATA[Yunus Emre DEMİRDAĞ]]></dc:creator>
            <pubDate>Tue, 05 Dec 2023 06:42:38 GMT</pubDate>
            <atom:updated>2025-10-10T06:29:03.702Z</atom:updated>
            <content:encoded><![CDATA[<h3>Comparison of TextToImage Models</h3><p>Hello there,</p><p>In this paper, a comprasion of some of the most widely used models for text-to-image will be made. These models were tested by applying them to different models on the same prompts at various levels of difficulty (short, long and complex).</p><h3>Text-To-Image</h3><p>A text-to-image model is a machine learning model which takes an input natural language description and produces an image matching that description[1]. The state of the art text-to-image models are;</p><ul><li>OpenAI’s DALL-E 3</li><li>Google Brain’s Imagen</li><li>Stability’s Stable Diffusion</li></ul><p>Text-to-image models generally combine a language models, which transforms the input text into a latent representation, and a <a href="https://en.wikipedia.org/wiki/Generative_model">g</a>enerative image model, which produces an image conditioned on that representation. The most effective models have generally been trained on massive amounts of image and text data scraped from the web.[2]</p><p>Text-to-image models are commonly used in:</p><ol><li>Art and Design</li><li>Entertainment and Gaming</li><li>Education and Learning Materials</li><li>Product Design</li><li>Data Visualization</li><li>Content Creation for Marketing</li></ol><h3>Comprasion of Models</h3><p>Comprasion of <a href="https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0">Stable-Diffusion-XL-base-1.0</a> and <a href="https://huggingface.co/runwayml/stable-diffusion-v1-5">Stable-Diffusion-v1–5</a> models with prompts from simple to complex.</p><p>Following prompts are was used for comprasion of models.</p><pre>&quot;View&quot;: A mysterious scenery deep within the shadows of a mountainous forest by a lake.<br>&quot;Futuristic City&quot;: A futuristic cityscape filled with tall buildings, light reflections, and flying vehicles.<br>&quot;Fantastic Beast&quot;: A large, flying fantastical creature with golden scales and colorful feathers.<br>&quot;Mars Exploration&quot;: A spacecraft on the surface of the red planet, amidst the stars in the sky.<br>&quot;Wild Nature&quot;: A vast, green valley filled with wildlife along the banks of a river.<br>&quot;Science Fiction World&quot;: The barren surface of an unknown planet in a distant galaxy.<br>&quot;Ancient Temple&quot;: An ancient temple on the summit of a mountain within a forest.<br>&quot;Night City&quot;: A city sparkling with lights, stars, and the moon in the night sky.<br>&quot;World Under the Ocean&quot;: Fantastic sea creatures in the deep ocean waters glowing under blue lights.<br>&quot;Enchanted Forest&quot;: A forest filled with colorful mushrooms, glowing insects, and magical trees.<br>&quot;Parallel Universe&quot;: The tale of a traveler lost among parallel universes beyond the bounds of time.<br>&quot;Unknown Planet&quot;: A landscape on an unknown planet with massive crystals and vibrating air.<br>&quot;Virtual Reality World&quot;: The adventure of an avatar lost among colorful code blocks in a virtual reality world.<br>&quot;Dream Catcher&quot;: The journey of a dream hunter in a world of giant doors and colorful pathways.<br>&quot;Ancient Robot King&quot;: The story of a colossal robot king made of gold and silver in an ancient world.<br>&quot;Quantum Maze&quot;: The mental adventure of a scientist lost in the mazes of the quantum world.<br>&quot;Infinite Library&quot;: The dance through time between the pages for a reader lost in an infinite library.<br>&quot;Sky City&quot;: A world floating above the clouds with flying cars and skywaterfalls.<br>&quot;Magical Library&quot;: Hidden among moss-covered stones in a mystical forest, doors of a magical library reveal a world filled with enchanted maps and ancient books where light dances.<br>&quot;Time Traveler&quot;: A time traveler embarks on an adventure questioning existence, carrying the poignant memories of the past with the ticking of an old clock.<br>&quot;Mysterious Submarine City&quot;: In the unknown depths, an underwater city with golden lights and sea creatures, housing remnants of a forgotten civilization.<br>&quot;Unresolved Dream&quot;: A person lost in a night&#39;s dream seeks a solution amidst tangled memories and colorful images, navigating between dream and reality.<br>&quot;Infinite Machine Maze&quot;: In the future, a brave explorer faces challenges in an infinite machine labyrinth filled with complex gears and massive turbines to discover an unknown energy source.<br>&quot;Garden of Dreams&quot;: A garden woven with the texture of dreams, where colorful flowers bloom, and branches dance melodically in the enchanting world of dreams.<br>&quot;Rainbow Valley&quot;: In a valley reflecting the colors of the rainbow, crystal-clear waters flow, and flying creatures glide in a peaceful natural paradise.<br>&quot;Mystic Mountain Peak&quot;: At the summit of a mystical mountain shrouded in mist, ancient temples reach towards the sky, and a mystical energy pervades, freezing time.<br>&quot;Surrealistic Sky&quot;: In a surreal world, flying fish swim in the sky, and colorful clouds dance in a dreamy atmosphere where time is an abstract concept.<br>&quot;Advanced Technology Metropolis&quot;: In an advanced technology metropolis, towering buildings reach towards the sky, neon lights illuminate the city, and humans and robots coexist in a future setting.<br>&quot;Broken Dreams&quot;: In the realm of shattered dreams, the story of a dream hunter navigating among colorful shards of glass.<br>&quot;Chaos Tower&quot;: In the depths of the subconscious, a tower of chaos rises, and consciousness navigates among abstract forms.</pre><p>Using this prompts the results were as follows.</p><p>Topic: View</p><p>Prompt: A mysterious scenery deep within the shadows of a mountainous forest by a lake.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*USX8WvD5z6fCq98KCPW3rQ.png" /><figcaption>Stable-diffusion-v1–5 | Stable-diffusion-base-xl</figcaption></figure><p>Topic: Fantastic Beast</p><p>Prompt: A large, flying fantastical creature with golden scales and colorful feathers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*dJlJmH91mXmIskJ61jTesw.png" /><figcaption>Stable-diffusion-v1–5 | Stable-diffusion-base-xl</figcaption></figure><p>Topic: Antique Temple</p><p>Prompt: An ancient temple on the summit of a mountain within a forest.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*qwZRyvLZyTT3cU70Lel2Ew.png" /><figcaption>Stable-diffusion-v1–5 | Stable-diffusion-base-xl</figcaption></figure><p>Topic: Night City</p><p>Prompt: A city sparkling with lights, stars, and the moon in the night sky.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*_5uqZpNV52MMFVUwN9Xmlg.png" /><figcaption>Stable-diffusion-v1–5 | Stable-diffusion-base-xl</figcaption></figure><p>Topic: Parallel Universe</p><p>Prompt: The tale of a traveler lost among parallel universes beyond the bounds of time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*2qIywYYsdSJbfO9dKchR6g.png" /><figcaption>Stable-diffusion-v1–5 | Stable-diffusion-base-xl</figcaption></figure><p>Topic: The Time Traveler</p><p>Prompt: A time traveler embarks on an adventure questioning existence, carrying the poignant memories of the past with the ticking of an old clock.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*-yuZVTnbvyt2FwW11pNGdg.png" /><figcaption>Stable-diffusion-v1–5 | Stable-diffusion-base-xl</figcaption></figure><p>As seen in the models, the Stable-diffusion-base xl model gives better results. When analyzed in terms of resolution, it is seen that the XL model produces better quality images.</p><p>This paper was written together <a href="https://medium.com/u/93a91f7b21f">Furkankestane</a>. In the future, different models will be used and added to the comparison.</p><h4>BIBLIOGRAPHY</h4><ol><li><a href="https://en.wikipedia.org/wiki/Text-to-image_model#:~:text=Text%2Dto%2Dimage%20models%20generally,image%20conditioned%20on%20that%20representation">https://en.wikipedia.org/wiki/Text-to-image_model#:~:text=Text%2Dto%2Dimage%20models%20generally,image%20conditioned%20on%20that%20representation</a>.</li><li>Vincent, James (May 24, 2022). <a href="https://www.theverge.com/2022/5/24/23139297/google-imagen-text-to-image-ai-system-examples-paper">“All these images were generated by Google’s latest text-to-image AI”</a>. <em>The Verge</em>. Vox Media. Retrieved May 28, 2022.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f88f7b4809f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/text-to-image-comprasion/comprasion-of-texttoimage-models-7f88f7b4809f">Comprasion of TextToImage Models</a> was originally published in <a href="https://medium.com/text-to-image-comprasion">Research Technologies</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pull Data with Selenium on the Dynamic Websites]]></title>
            <link>https://medium.com/@yunusemredemirdag/pull-data-with-selenium-on-the-dynamic-websites-f35c9d255efe?source=rss-5c9eeec91e57------2</link>
            <guid isPermaLink="false">https://medium.com/p/f35c9d255efe</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[selenium]]></category>
            <category><![CDATA[collect-data]]></category>
            <category><![CDATA[crawling]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Yunus Emre DEMİRDAĞ]]></dc:creator>
            <pubDate>Sat, 11 Nov 2023 13:07:21 GMT</pubDate>
            <atom:updated>2023-12-09T21:53:42.752Z</atom:updated>
            <content:encoded><![CDATA[<p>Good days, Today, we are going to learn to extract data with selenium on the dynamic websites. For extract data, We use javascript scripts.</p><p>Firstly, we are connecting to website with selenium. Then, We are run javascript codes on website and then we are taking outputs. This processes is useful particularly that run as dynamic, while pulling with beautifulsoup, we don’t pull this datas. So we are using selenium.</p><p>I am going to tell this processes at basic level by pull Python PEP Rules.</p><h4><strong>IMPORT LIBRAIES</strong></h4><p>We are adding below libraries to connect to website.</p><pre>from selenium import webdriver<br>import time</pre><h4>CREATE DRIVER FOR CONNECTION</h4><p>After Libraries are added, we are creating a driver with specific options.</p><pre>options = webdriver.ChromeOptions()<br>options.add_argument(&#39; - headless&#39;)<br>driver = webdriver.Chrome(options=options)<br>base_url = &quot;https://peps.python.org/&quot;</pre><p>We are using Chroma for connection. So firstly we are creating <strong>ChromaOptions</strong>, then we are adding argument to options object.</p><p>While processes is applying, “ — headless” argument is using to don’t open Chroma Browser. Then We are creating Chroma driver.</p><pre>driver.get(base_url)<br>time.sleep(2)</pre><p>After Chroma driver is created, We are connecting to website. And then we are waiting 2 seconds for exactly uploaded all site objects.</p><p>Then we are preparing javascript script.</p><pre>script = &quot;&quot;&quot;<br><br>var data = []<br>var rows = document.querySelectorAll(&quot;#numerical-index &gt; div &gt; table &gt; tbody &gt; tr&quot;);<br><br>rows.forEach(function(row) {<br>    var columns = row.querySelectorAll(&quot;td&quot;);<br>    <br>    var sub_data = [];<br>    columns.forEach(function(column) {<br>        var attr = column.textContent;<br>        <br>        sub_data.push(attr);<br>        });<br>    sub_data[0] = columns[0].textContent + &quot;, &quot; + columns[0].querySelector(&quot;abbr&quot;).title;<br>    <br>    data.push({info: sub_data});<br>})<br><br>return data;<br>&quot;&quot;&quot;</pre><p>This script extract data from website. “<strong>return data;”</strong> command is return expected data.</p><pre>results = driver.execute_script(script)<br>print(results)<br>driver.close()</pre><p>Finally we are running this script with <strong>driver.execute_script(..)</strong> command.</p><p>And we take outputs like below.</p><pre>[<br>  {<br>    &#39;info&#39;: [<br>      &#39;PA, Process, Active&#39;,<br>      &#39;1&#39;,<br>      &#39;PEP Purpose and Guidelines&#39;,<br>      &#39;Barry Warsaw, Jeremy Hylton, David Goodger, Alyssa Coghlan&#39;,<br>      &#39;&#39;<br>    ]<br>  },<br>  {<br>    &#39;info&#39;: [<br>      &#39;PA, Process, Active&#39;,<br>      &#39;2&#39;,<br>      &#39;Procedure for Adding New Modules&#39;,<br>      &#39;Brett Cannon, Martijn Faassen&#39;,<br>      &#39;&#39;<br>    ]<br>  },<br>  {<br>    &#39;info&#39;: [<br>      &#39;PW, Process, Withdrawn&#39;,<br>      &#39;3&#39;,<br>      &#39;Guidelines for Handling Bug Reports&#39;,<br>      &#39;Jeremy Hylton&#39;,<br>      &#39;&#39;<br>    ]<br>  },<br>  ...<br>]</pre><h4>ALL SCRIPT</h4><pre>from selenium import webdriver<br>import time<br><br><br>options = webdriver.ChromeOptions()<br>options.add_argument(&#39;--headless&#39;)<br>driver = webdriver.Chrome(options=options)<br>base_url = &quot;https://peps.python.org/&quot;<br><br><br>driver.get(base_url)<br><br>time.sleep(2)<br><br><br>script = &quot;&quot;&quot;<br><br>var data = []<br>var rows = document.querySelectorAll(&quot;#numerical-index &gt; div &gt; table &gt; tbody &gt; tr&quot;);<br><br>rows.forEach(function(row) {<br>    var columns = row.querySelectorAll(&quot;td&quot;);<br>    <br>    var sub_data = [];<br>    columns.forEach(function(column) {<br>        var attr = column.textContent;<br>        <br>        sub_data.push(attr);<br>        });<br>    sub_data[0] = columns[0].textContent + &quot;, &quot; + columns[0].querySelector(&quot;abbr&quot;).title;<br>    <br>    data.push({info: sub_data});<br>})<br><br>return data;<br>&quot;&quot;&quot;<br><br>results = driver.execute_script(script)<br>print(results)<br>    <br>driver.close()</pre><p>Thank you for listening to me.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f35c9d255efe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python ile RabbitMQ Üzerinde Virtual Host Oluşturularak Görev Kuyruklarına Yönlendirme Yapılması]]></title>
            <link>https://medium.com/@yunusemredemirdag/python-ile-rabbitmq-%C3%BCzerinde-virtual-host-olu%C5%9Fturularak-g%C3%B6rev-kuyruklar%C4%B1na-y%C3%B6nlendirme-yap%C4%B1lmas%C4%B1-71ff347cfd88?source=rss-5c9eeec91e57------2</link>
            <guid isPermaLink="false">https://medium.com/p/71ff347cfd88</guid>
            <dc:creator><![CDATA[Yunus Emre DEMİRDAĞ]]></dc:creator>
            <pubDate>Sun, 04 Sep 2022 15:19:16 GMT</pubDate>
            <atom:updated>2022-09-04T15:20:44.347Z</atom:updated>
            <content:encoded><![CDATA[<p>Bu Yazıda, RabbitMQ ile kullanıcı oluşturlarak kimlik doğrulaması yapılması ve bu bilgiler ile kuyruk oluşturulması anlatılmaktadır.</p><p>İlk olarak RabbitMQ ile bir kullanıcı oluşturuyoruz. Daha sonra çalışma ortamı olarak bir <strong>virtual host ( vhost )</strong> oluşturuyoruz. Bu vhost üzerinde bir kuyruk oluşturarak işlemlerimizi yapıyoruz.</p><h3>VIRTUAL HOST OLUŞTURULMASI</h3><p>İlk olarak rabbitmq ile ilgili işlemleri yapmak için root kullanıcısına geçiyoruz.</p><blockquote><strong>sudo su</strong></blockquote><p>Sistemde default olarak guest kullanıcısı bulunur. Bu kullanıcı <strong>administrator</strong> etiketine sahiptir. Biz kendi kullanıcımızı ekleyerek, bu kullanıcıya bir şifre verebiliriz. Bu şekilde mesaj kuyruklarımızın güvenliğini ilk etapta sağlamış oluruz. Kullanıcı oluşturmak için şu komut uygulanır.</p><blockquote><strong>rabbitmqctl add_user “test_user” “test_123”</strong></blockquote><p>Bu komut satırında<strong> “test_user”</strong> ile belirtilen değişken kullanıcı adını belirtir. şifre olarak ise <strong>“test_123”</strong> olarak belirtilen değişken şifremizi belirlediğimiz yerdir. Bu işlem sonucunda şu şekilde bir çıktı alırız.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*aMS8k76IHrqTh55usTx8eA.png" /><figcaption>Kullanıcı Ekledikten Sonra Oluşan Mesaj</figcaption></figure><p>Çıktı kullanıcı eklediğimizi fakat kullanıcının yetkisinin bulunmadığını söylemektedir. Kullanıcın eklendiğini şu komut ile doğrulayabiliriz.</p><blockquote><strong>rabbitmqctl list_users</strong></blockquote><p>Bu komutun çıktısı şu şekilde olacaktır.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/468/1*B8dYxdXEMh3tUumscb8kUQ.png" /><figcaption>RabbitMQ Kullanıcıları</figcaption></figure><p>Yukarıdaki Görselde görüldüğü gibi <strong>test_user</strong> kullanıcımız oluşmuştur. Şimdi bu kullanıcıya bir tag ataması yapmamız geremektedir. Tag ataması için şu komut kullanılmalıdır.</p><blockquote><strong>rabbitmqctl set_user_tags test_user administrator</strong></blockquote><p>Bu komut ile <strong>test_user</strong> kullanıcısında <strong>administrator</strong> Tag’i atanmıştır. Kullanıcılar tekrar listelenerek kullanıcıya Tag ataması yapıldığı doğrulanabilir. Kullanıcı oluşturulduktan sonra bir vhost ( virtual host ) oluşturmamız gerekmektedir.</p><blockquote><strong>rabbitmqctl add_vhost test_vhost</strong></blockquote><p>Bu komut ile <strong>test_vhost</strong> adında bir virtual host oluşturulmuştur. Sistemde default olarak <strong>‘/’</strong> virtual host bulunmaktadır. vhost’ları görüntülemek için şu komut kullanılabilir.</p><blockquote><strong>rabbitmqctl list_vhosts</strong></blockquote><p>Bu komutun çıktısı şu şekilde olacaktır.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/1*fyw-noz-pLrvn274bCvO-g.png" /><figcaption>Virtual Hosts</figcaption></figure><p>Çıktıda görüldüğü gibi <strong>test_vhost</strong> vhost’unun oluştuğu görülmektedir.</p><p>Bu işlemler tamamlandıktan sonra oluşturulan virtual host için oluşturduğumuz kullanıcıya izin tanımlamamız gerekir. Bunun için şu komut kullanılabilir.</p><blockquote><strong>rabbitmqctl set_permissions -p “test_vhost” “test_user” “.*” “.*” “.*”</strong></blockquote><p>Bu komutta <strong>-p </strong>parametresinden sonra ilk olarak oluşturulan vhost adı belirtilir. Daha sonra izin verilmek istenen kullanıcı adı belirtilir. Sonrasında gelen<strong> “.*”</strong> parametrelerinin anlamları şu şekildedir.</p><ul><li>İlk<strong> “.*”</strong> Her bir varlık için konfigürasyon yapabilme yetkisi tanımlamaktadır.</li><li>İkinci<strong> “.*”</strong> Her bir varlık için yazma yetkisi tanımlamaktadır.</li><li>Üçüncü<strong> “.*”</strong> Her bir varlık için okuma yetkisi tanımlamaktadır.</li></ul><p>Yetkiler tanımlandıktan sonra kullanıma hazır hale gelmektedir.</p><h3>PYTHON RABBITMQ BAĞLANTISI</h3><p>RabbitMQ ile bağlantı kurulması için pika kütüphanesine ihtiyaç duyulmaktadır.</p><blockquote><strong>pip install pika</strong></blockquote><p>komutu ile <strong>pika</strong> kütüphanesi indirilir. Oluşturulacak örneğin Dosya yapısı şu şekildedir;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*BQaIPDsrx5fLuHbdPFP1EQ.png" /><figcaption>Dizin Yapısı</figcaption></figure><p>Kimlik Doğrulama ile bağlantı kurulması için <strong>configuration.py</strong> dosyasında gerekli bağlantılar yapılmıştır.</p><pre># configuration.py<br>import pika<br><br><br>credintals = pika.PlainCredentials(&#39;test_user&#39;, &#39;test_123&#39;)<br>parameters = pika.ConnectionParameters(&#39;localhost&#39;, 5672, &#39;test_vhost&#39;, credintals)<br><br>connection = pika.BlockingConnection(parameters)</pre><p>Burada ilk olarak kullanıcı adı ve şifrenin alınması için kütüphanenin <strong><em>PlainCredentials</em></strong> fonksiyonu kullanılıyor. Daha sonra bağlantı için gerekli parametreler veriliyor. <strong><em>ConnectionParameters</em></strong> fonksiyonunda ilk parametre host adresimizi belirtir. şuan yerel bilgisayarda çalıştğımızdan dolayı <em>‘localhost’</em> olarak girilir. Daha sonra ikinci parametre port numarasını belirtmektedir. rabbitmq <strong>5672</strong> portunda dinleme yapmaktadır. Üçüncü parametre oluşturduğumuz virtual host ismini almaktadır. Bu virtual host üzerinde kuyruklar oluşturulacaktır. Son olarak kullanıcı bilgileri verilerek parametreler tamamlanır ve bağlantı kurulur.</p><p>Bu örnekte log kayıtlarının farklı görev dinleyicileri tarafından alınıp işleme alınması gerçekleştirilmiştir. ilk olarak info log kayıtları için bir dinleyici oluşturuyoruz.<strong> receiver_info.py</strong> dosyasında temel bağlantılar yapılmış ve işlemler gerçekleştirilmiştir.</p><p>ilk olarak <strong>configuration.py</strong> dosyasında connection alınır.</p><pre>from configuration import connection<br><br><br>channel = connection.channel()</pre><p>connection ile bağlantı alınarak bir kanala bağlanılır. Daha sonra info log kayıtlarını tutmak için temel konfigürasyon yapılır.</p><pre>logging.basicConfig(level=logging.INFO,filename=&quot;logs/info_queue.log&quot;)</pre><p>Daha sonra kuyruklara görevleri dağıtmak için bir <strong>exchange</strong> belirtilir. Kuyruk belirlenir. <strong>Exchange</strong> ve <strong>kuyruk</strong> birbirine bağlanarak dinlemeye hazır hale getirilir. Ve istek gelmesi beklenir.</p><pre>channel.exchange_declare(exchange=&#39;logs&#39;, exchange_type=&#39;direct&#39;)<br>channel.queue_declare(queue=&#39;info&#39;)<br>channel.queue_bind(queue=&#39;info&#39;, exchange=&#39;logs&#39;)<br><br>print(&quot; [x] Waiting Info Log. To exit press CTRL+C&quot;)</pre><p>exchange ismi olarak ‘logs’ belirlenmiştir. ve exchange_type olarak ‘direct’ belirlenmiştir. info bilgilerini dinlemek için kuyruk adı olarak ‘info’ verilmiştir. kuyruk ve exchange bağlantısı kurulmuştur.</p><pre>def callback(ch, method, properties, body):<br>    print(f&quot; [*] Info: {body}&quot;)<br>    logging.info(f&quot; [*] Info: {body}&quot;)<br><br><br>channel.basic_consume(queue=&#39;info&#39;, on_message_callback=callback, auto_ack=True)<br><br>channel.start_consuming()</pre><p>Daha sonra bir callback fonksiyonu yazılarak gönderilen mesajlar log dosyasına kaydedilmiştir. <strong><em>basic_consume</em></strong> metodu ile dinleyeceğimiz kuyruk belirttilmiştir. <strong><em>on_message_callback</em></strong> fonksiyonu ile kuyruktan alınan görevlerin tabi tutulacağı işlemlerin yapılacağı fonksiyon belirtilmektedir. Burada callback fonkiyonu ile info kuyruğuna gelen mesajların info log dosyasına yazılması gerçekleştirilmiştir. <strong><em>start_consuming</em></strong> ile dinlemeye başlamıştır.</p><p>Bu işlemler warning ve error log kayıtları için de yapıldıktan sonra çalıştırılmaya hazır hale gelecektir. Kodların tamamı aşağıda yer almaktadır. Bu dinleyicilere istek atılması için worker.py dosyası oluşturulmuştur.</p><pre># worker.py<br>from configuration import connection<br><br><br>channel = connection.channel()<br><br>channel.exchange_declare(exchange=&#39;logs&#39;, exchange_type=&#39;direct&#39;)<br><br>levels = [&quot;info&quot;, &quot;warning&quot;, &quot;error&quot;]<br><br>for level in levels:<br>    channel.queue_declare(queue=level)<br>    channel.basic_publish(exchange=&#39;logs&#39;, routing_key=level, body=&quot;Hello, World!&quot;)<br>    print(f&quot; [x] Sent &#39;Hello World!&#39; to {level}&quot;)<br><br>connection.close()</pre><p>Bu kod bloğunda her bir log için dizi kuyruk belirlenmiş ve istekler kuyruklara iletilmiştir. İşlemler tamamlandıktan sonra farklı terminallerde receiver kodları çalıştırılır. Sonra <strong>worker.py</strong> dosyası çalıştırılarak işlem yapılması beklenir. Çalışma sonucunda terminallerde aşağıdaki çıktıların alındığı görülür ve sonuçlar log dosyalarına kaydedilir.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/503/1*xQH9N_alnchDUBW9ODIR2g.png" /><figcaption>INFO Kuyruğu ( Komut : python receiver_info.py )</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*INC9mA1uo14UupTzYokBvA.png" /><figcaption>WARNING Kuyruğu ( Komut : python receiver_warning.py )</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/362/1*SiemLKgQCs1ptafQqnpZQg.png" /><figcaption>ERROR Kuyruğu ( Komut : python receiver_error.py )</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/334/1*v4odTZ-t5BwN2uFCZrCv-g.png" /><figcaption>Kuyruklara Mesaj Gönderilmesi ( Komut : python worker.py )</figcaption></figure><p>Bu şekilde farklı kuyruklar oluşturularak işlemler gerçekleştirilir ve Güvenli bir şekilde işlemler kuyruğa alınarak işletilmesi gerçekleştirilir.</p><h3>KOD</h3><pre># configuration.py<br>import pika<br><br><br>credintals = pika.PlainCredentials(&#39;test_user&#39;, &#39;test_123&#39;)<br>parameters = pika.ConnectionParameters(&#39;localhost&#39;, 5672, &#39;test_vhost&#39;, credintals)<br><br>connection = pika.BlockingConnection(parameters)</pre><pre>--------------------------------------------------------------------</pre><pre># worker.py<br>from configuration import connection<br><br><br>channel = connection.channel()<br><br>channel.exchange_declare(exchange=&#39;logs&#39;, exchange_type=&#39;direct&#39;)<br><br>levels = [&quot;info&quot;, &quot;warning&quot;, &quot;error&quot;]<br><br>for level in levels:<br>    channel.queue_declare(queue=level)<br>    channel.basic_publish(exchange=&#39;logs&#39;, routing_key=level, body=&quot;Hello, World!&quot;)<br>    print(f&quot; [x] Sent &#39;Hello World!&#39; to {level}&quot;)<br><br>connection.close()</pre><pre>--------------------------------------------------------------------</pre><pre># receiver_info.py<br>import logging<br>from configuration import connection<br><br><br>channel = connection.channel()<br><br>logging.basicConfig(level=logging.INFO, filename=&quot;logs/info_queue.log&quot;)<br><br>channel.exchange_declare(exchange=&#39;logs&#39;, exchange_type=&#39;direct&#39;)<br>channel.queue_declare(queue=&#39;info&#39;)<br>channel.queue_bind(queue=&#39;info&#39;, exchange=&#39;logs&#39;)<br><br>print(&quot; [x] Waiting Info Log. To exit press CTRL+C&quot;)<br><br><br>def callback(ch, method, properties, body):<br>    print(f&quot; [*] Info: {body}&quot;)<br>    logging.info(f&quot; [*] Info: {body}&quot;)<br><br><br>channel.basic_consume(queue=&#39;info&#39;, on_message_callback=callback, auto_ack=True)<br><br>channel.start_consuming()</pre><pre>--------------------------------------------------------------------</pre><pre># receiver_warning.py<br>import logging<br>from configuration import connection<br><br>logging.basicConfig(level=logging.WARNING, filename=&quot;logs/warning_queue.log&quot;)<br><br>channel = connection.channel()<br><br>channel.exchange_declare(exchange=&#39;logs&#39;, exchange_type=&#39;direct&#39;)<br>channel.queue_declare(queue=&#39;warning&#39;)<br>channel.queue_bind(queue=&#39;warning&#39;, exchange=&#39;logs&#39;)<br><br>print(&quot; [x] Waiting Info Log. To exit press CTRL+C&quot;)<br><br><br>def callback(ch, method, properties, body):<br>    print(f&quot; [*] Warning: {body}&quot;)<br>    logging.warning(f&quot; [*] Warning: {body}&quot;)<br><br><br>channel.basic_consume(queue=&#39;warning&#39;, on_message_callback=callback, auto_ack=True)<br><br>channel.start_consuming()</pre><pre>--------------------------------------------------------------------</pre><pre># receiver_error.py<br>import logging<br>from configuration import connection<br><br>logging.basicConfig(level=logging.ERROR, filename=&quot;logs/error_queue.log&quot;)<br><br>channel = connection.channel()<br><br>channel.exchange_declare(exchange=&#39;logs&#39;, exchange_type=&#39;direct&#39;)<br>channel.queue_declare(queue=&#39;error&#39;)<br>channel.queue_bind(queue=&#39;error&#39;, exchange=&#39;logs&#39;)<br><br>print(&quot; [x] Waiting Info Log. To exit press CTRL+C&quot;)<br><br><br>def callback(ch, method, properties, body):<br>    print(f&quot; [*] Error: {body}&quot;)<br>    logging.error(f&quot; [*] Error: {body}&quot;)<br><br><br>channel.basic_consume(queue=&#39;error&#39;, on_message_callback=callback, auto_ack=True)<br><br>channel.start_consuming()</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=71ff347cfd88" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>