<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mustafa Tırnova on Medium]]></title>
        <description><![CDATA[Stories by Mustafa Tırnova on Medium]]></description>
        <link>https://medium.com/@mustafa.tirnova?source=rss-b1d0810d5c2f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*mmF5SPZ5qAZ12O8hZyY0Xw.jpeg</url>
            <title>Stories by Mustafa Tırnova on Medium</title>
            <link>https://medium.com/@mustafa.tirnova?source=rss-b1d0810d5c2f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 22:36:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mustafa.tirnova/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Moving S3 Data Between AWS Accounts with DataSync]]></title>
            <link>https://medium.com/@mustafa.tirnova/moving-s3-data-between-aws-accounts-with-datasync-0b2daad8911c?source=rss-b1d0810d5c2f------2</link>
            <guid isPermaLink="false">https://medium.com/p/0b2daad8911c</guid>
            <category><![CDATA[aws-datasync]]></category>
            <category><![CDATA[amazon-s3]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud-migration]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Mustafa Tırnova]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 17:57:03 GMT</pubDate>
            <atom:updated>2026-10-25T17:57:03.733Z</atom:updated>
            <content:encoded><![CDATA[<p>To copy an S3 bucket into another AWS account with DataSync, you need separate service roles for the source and destination buckets, a bucket policy on the destination, and locations created in each bucket’s own Region.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ucchx1XC0tovdfIyjsug4g.png" /></figure><h3>Why DataSync fits this job</h3><p>DataSync fits a one-off copy between two AWS accounts when you want logs and verification without writing or maintaining scripts.</p><p>A typical case is moving a bucket into a different account, for example during an account consolidation or a move to a new landing zone, where you also need evidence that every object arrived. I have used DataSync to move very large datasets between AWS accounts, and this guide walks through the setup step by step.</p><p>There are three common options for a copy like this.</p><ul><li><strong>aws s3 sync</strong> is good for small, quick copies from one machine. The trade-off is that you own retries, credentials and checking the result.</li><li><strong>S3 Replication</strong> is good for keeping buckets in step over time. The trade-off is that it needs versioning on both buckets and does not copy existing objects on its own.</li><li><strong>AWS DataSync</strong> is good for one-off or scheduled bucket to bucket copies. It has few trade-offs here, since it gives a managed task, CloudWatch logs and built-in verification.</li></ul><p>My rule of thumb is simple. Pick S3 Replication when new objects must keep flowing between the buckets. Pick DataSync for a one-off or scheduled migration where you need logs and proof that everything arrived, which is what the rest of this guide covers. Use aws s3 sync only when the data is small and you can watch it run.</p><h3>Architecture and the eight steps</h3><p>In this setup DataSync runs from the source account, reads the source bucket through one service role, and writes to the destination bucket through a second one that the destination bucket policy explicitly trusts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zab9xOYRuILOAd4EjXt5Tg.png" /><figcaption>Figure 1, the eight steps and where each one lives across the two accounts.</figcaption></figure><p><strong>The steps below match the numbers in the diagram.</strong></p><ol><li>Create the destination bucket in the destination account with ACLs disabled.</li><li>Create two DataSync service roles in the source account, one per bucket.</li><li>Add a bucket policy to the destination bucket.</li><li>Assume an admin role in the source account that is allowed to create the locations.</li><li>Create the source and destination locations, each in its own bucket’s Region.</li><li>Create a CloudWatch log group that DataSync can write to.</li><li>Create the task.</li><li>Start the task and verify the result.</li></ol><p>These placeholders appear in every command and policy.</p><ul><li><strong>SOURCE_ACCOUNT_ID</strong> is the account that owns the source bucket and runs DataSync.</li><li><strong>DEST_ACCOUNT_ID</strong> is the account that owns the destination bucket.</li><li><strong>SOURCE_BUCKET and DEST_BUCKET</strong> are the bucket names.</li><li><strong>SOURCE_REGION and DEST_REGION</strong> are the Regions of the two buckets, for example eu-west-1 and eu-central-1.</li></ul><h3>Steps 1 to 4, the permission setup</h3><p>Most cross-account DataSync problems come from this part, so it is worth doing slowly.</p><h3>Step 1, create the destination bucket</h3><p>Create the bucket in the destination account with ACLs disabled, which is the default for new buckets. Every copied object is then owned by the destination account and you avoid object ownership surprises. Run the command with credentials for the destination account.</p><pre>aws s3api create-bucket \<br>  --bucket DEST_BUCKET \<br>  --region DEST_REGION \<br>  --create-bucket-configuration LocationConstraint=DEST_REGION</pre><h3>Step 2, create the two service roles in the source account</h3><p>Create two roles that DataSync can assume. This guide calls them DataSyncSourceRole and DataSyncDestRole. Both use the same trust policy.</p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Principal&quot;: { &quot;Service&quot;: &quot;datasync.amazonaws.com&quot; },<br>      &quot;Action&quot;: &quot;sts:AssumeRole&quot;,<br>      &quot;Condition&quot;: {<br>        &quot;StringEquals&quot;: { &quot;aws:SourceAccount&quot;: &quot;SOURCE_ACCOUNT_ID&quot; }<br>      }<br>    }<br>  ]<br>}</pre><p>Attach this inline policy to DataSyncDestRole. It grants access to the destination bucket only.</p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Action&quot;: [<br>        &quot;s3:GetBucketLocation&quot;,<br>        &quot;s3:ListBucket&quot;,<br>        &quot;s3:ListBucketMultipartUploads&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;arn:aws:s3:::DEST_BUCKET&quot;<br>    },<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Action&quot;: [<br>        &quot;s3:AbortMultipartUpload&quot;,<br>        &quot;s3:DeleteObject&quot;,<br>        &quot;s3:GetObject&quot;,<br>        &quot;s3:ListMultipartUploadParts&quot;,<br>        &quot;s3:PutObject&quot;,<br>        &quot;s3:GetObjectTagging&quot;,<br>        &quot;s3:PutObjectTagging&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;arn:aws:s3:::DEST_BUCKET/*&quot;<br>    }<br>  ]<br>}</pre><p>To limit the role to buckets owned by the destination account, add this condition to both statements. The current AWS tutorial does the same.</p><pre>&quot;Condition&quot;: {<br>  &quot;StringEquals&quot;: { &quot;aws:ResourceAccount&quot;: &quot;DEST_ACCOUNT_ID&quot; }<br>}</pre><p>DataSyncSourceRole gets the same policy pointed at SOURCE_BUCKET. For a copy only task you can trim it down to the read actions. If your source objects use SSE-KMS, this role also needs permission to use the key.</p><h3>Step 3, add the bucket policy on the destination bucket</h3><p>The destination account has to opt in. Create the admin role from step 4 before you save this policy, because S3 rejects a bucket policy that names a role that does not exist yet. This bucket policy lets DataSyncDestRole read and write, and gives the admin role from step 4 permission to list the bucket. The current AWS tutorial lists only the DataSync role, so this second statement matters only if creating the destination location is denied without it.</p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Sid&quot;: &quot;DataSyncCreateS3LocationAndTaskAccess&quot;,<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Principal&quot;: {<br>        &quot;AWS&quot;: &quot;arn:aws:iam::SOURCE_ACCOUNT_ID:role/DataSyncDestRole&quot;<br>      },<br>      &quot;Action&quot;: [<br>        &quot;s3:GetBucketLocation&quot;,<br>        &quot;s3:ListBucket&quot;,<br>        &quot;s3:ListBucketMultipartUploads&quot;,<br>        &quot;s3:AbortMultipartUpload&quot;,<br>        &quot;s3:DeleteObject&quot;,<br>        &quot;s3:GetObject&quot;,<br>        &quot;s3:ListMultipartUploadParts&quot;,<br>        &quot;s3:PutObject&quot;,<br>        &quot;s3:GetObjectTagging&quot;,<br>        &quot;s3:PutObjectTagging&quot;<br>      ],<br>      &quot;Resource&quot;: [<br>        &quot;arn:aws:s3:::DEST_BUCKET&quot;,<br>        &quot;arn:aws:s3:::DEST_BUCKET/*&quot;<br>      ]<br>    },<br>    {<br>      &quot;Sid&quot;: &quot;DataSyncCreateS3Location&quot;,<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Principal&quot;: {<br>        &quot;AWS&quot;: &quot;arn:aws:iam::SOURCE_ACCOUNT_ID:role/DataSyncAdminRole&quot;<br>      },<br>      &quot;Action&quot;: &quot;s3:ListBucket&quot;,<br>      &quot;Resource&quot;: &quot;arn:aws:s3:::DEST_BUCKET&quot;<br>    }<br>  ]<br>}</pre><h3>Step 4, assume the admin role in the source account</h3><p>The identity that creates the locations and the task is not the same as the service roles. This guide uses a role called DataSyncAdminRole in the source account. It needs the DataSync actions you plan to run, such as datasync:CreateLocationS3, datasync:CreateTask, datasync:StartTaskExecution and datasync:DescribeTaskExecution. It also needs iam:PassRole on the two service roles.</p><p>Instead of exporting temporary keys into the shell, a named profile that assumes the role is cleaner and easier to repeat.</p><pre>[profile datasync-admin]<br>role_arn = arn:aws:iam::SOURCE_ACCOUNT_ID:role/DataSyncAdminRole<br>source_profile = default<br>region = DEST_REGION</pre><p>Confirm that you are using the right identity before you continue.</p><pre>aws sts get-caller-identity --profile datasync-admin</pre><h3>Steps 5 to 7, locations, logging and the task</h3><p>With the permissions in place, the rest is three short CLI commands. Run all of them with the admin profile from step 4.</p><h3>Step 5, create the two locations</h3><p>A location has to be created in the Region of its own bucket. Each location uses the service role that has access to its own bucket.</p><pre># Source location, in the source bucket&#39;s Region<br>aws datasync create-location-s3 \<br>  --s3-bucket-arn arn:aws:s3:::SOURCE_BUCKET \<br>  --s3-config BucketAccessRoleArn=arn:aws:iam::SOURCE_ACCOUNT_ID:role/DataSyncSourceRole \<br>  --region SOURCE_REGION \<br>  --profile datasync-admin</pre><pre># Destination location, in the destination bucket&#39;s Region<br>aws datasync create-location-s3 \<br>  --s3-bucket-arn arn:aws:s3:::DEST_BUCKET \<br>  --s3-config BucketAccessRoleArn=arn:aws:iam::SOURCE_ACCOUNT_ID:role/DataSyncDestRole \<br>  --region DEST_REGION \<br>  --profile datasync-admin</pre><p>Each command returns a LocationArn, so keep both. You can add --s3-storage-class to choose the storage class at the destination and --subdirectory to limit the copy to a prefix.</p><h3>Step 6, create the log group</h3><p>DataSync writes its task logs to CloudWatch Logs, and it needs permission to do so. Create the log group and give the DataSync service a resource policy on it. This step applies to Basic mode tasks.</p><pre>aws logs create-log-group \<br>  --log-group-name /aws/datasync/s3-cross-account \<br>  --region DEST_REGION \<br>  --profile datasync-admin</pre><pre>aws logs put-resource-policy \<br>  --policy-name DataSyncLogs \<br>  --region DEST_REGION \<br>  --profile datasync-admin \<br>  --policy-document &#39;{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [{<br>      &quot;Sid&quot;: &quot;DataSyncLogsToCloudWatchLogs&quot;,<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Principal&quot;: { &quot;Service&quot;: &quot;datasync.amazonaws.com&quot; },<br>      &quot;Action&quot;: [&quot;logs:CreateLogStream&quot;, &quot;logs:PutLogEvents&quot;],<br>      &quot;Resource&quot;: &quot;arn:aws:logs:DEST_REGION:SOURCE_ACCOUNT_ID:log-group:/aws/datasync/s3-cross-account:*&quot;<br>    }]<br>  }&#39;</pre><h3>Step 7, create the task</h3><p>The task ties the two locations together. When the buckets are in different Regions, create the task in the destination Region, and the log group has to live in the same Region as the task. The Region rules are described in the DataSync cross-account tutorial listed in the references.</p><p>AWS now recommends Enhanced task mode for this kind of transfer. With Basic mode, a transfer that crosses both accounts and Regions can fail with a network connection error, unless the task is created in the Region of the destination location, which is why the Region rule above matters. Add --task-mode ENHANCED to the command below to use Enhanced mode. The options each mode supports differ, so use the DataSync task mode documentation as the reference for the exact option list. Enhanced mode tasks also send their logs to a log group named /aws/datasync automatically, so the log group from step 6 is only needed for Basic mode.</p><pre>aws datasync create-task \<br>  --source-location-arn SOURCE_LOCATION_ARN \<br>  --destination-location-arn DEST_LOCATION_ARN \<br>  --cloud-watch-log-group-arn arn:aws:logs:DEST_REGION:SOURCE_ACCOUNT_ID:log-group:/aws/datasync/s3-cross-account \<br>  --name s3-cross-account-copy \<br>  --options VerifyMode=ONLY_FILES_TRANSFERRED,LogLevel=TRANSFER \<br>  --region DEST_REGION \<br>  --profile datasync-admin</pre><p>Two options are worth knowing. LogLevel=TRANSFER logs every file that is transferred, which helps when you need to prove what moved. VerifyMode=ONLY_FILES_TRANSFERRED checks only the files that were copied, while the default scans both locations at the end and takes longer on large buckets.</p><h3>Step 8, start the task and verify the result</h3><p>A finished task is not proof by itself, so check the result three ways before you call the migration done.</p><p>Start the task and keep the execution ARN that comes back.</p><pre>aws datasync start-task-execution \<br>  --task-arn TASK_ARN \<br>  --region DEST_REGION \<br>  --profile datasync-admin</pre><p>To copy only part of the bucket, add --includes FilterType=SIMPLE_PATTERN,Value=&quot;/prefix/*&quot; or an --excludes filter to the same command.</p><p>Watch the execution until its status reaches SUCCESS. The output also reports the files and bytes that were transferred.</p><pre>aws datasync describe-task-execution \<br>  --task-execution-arn TASK_EXECUTION_ARN \<br>  --region DEST_REGION \<br>  --profile datasync-admin</pre><p>Then compare the two buckets directly. The last two lines of this command show the total object count and size.</p><pre>aws s3 ls s3://SOURCE_BUCKET --recursive --summarize --human-readable | tail -2<br>aws s3 ls s3://DEST_BUCKET --recursive --summarize --human-readable | tail -2</pre><p>Finally, open the CloudWatch log group and look for errors and skipped files. If the counts differ, the log is where the reason will be. DataSync transfers the current version of each object, so a versioned source bucket can legitimately hold more objects than the destination. Objects encrypted with customer-provided keys, known as SSE-C, are not transferred either, so they can also explain a difference.</p><p>Keep the task execution output and both bucket totals together, since they are the record that the migration was complete.</p><h3>What goes wrong and how to fix it</h3><p>Cross-account setups fail in a small number of predictable places, and almost all of them are permission or Region mistakes.</p><ul><li><strong>Creating the destination location is denied.</strong> Likely cause, the admin role is missing from the second statement of the bucket policy. Fix, add s3:ListBucket for the admin role on the destination bucket.</li><li><strong>Creating a location or task is denied on iam:PassRole.</strong> Likely cause, the admin role cannot pass the service roles to DataSync. Fix, add iam:PassRole for both service roles to the admin role.</li><li><strong>The location is rejected or the task cannot use it.</strong> Likely cause, the location was created in a different Region than its bucket. Fix, recreate the location with the bucket’s Region.</li><li><strong>The task runs but no logs appear.</strong> Likely cause, the log group has no resource policy for DataSync, or it sits in another Region than the task. Fix, add the policy from step 6 and keep the log group in the task’s Region.</li><li><strong>Access is denied on encrypted objects.</strong> Likely cause, the role or the key policy does not allow the KMS key. Fix, allow the DataSync role in the key policy and in the role policy.</li><li><strong>Copied objects cannot be read in the destination account.</strong> Likely cause, the destination bucket has ACLs enabled and ownership stayed with the source. Fix, use a bucket with ACLs disabled.</li></ul><h3>Cost, cleanup and wrap up</h3><p>DataSync charges per gigabyte copied, and you also pay for S3 requests and for data transfer when the buckets are in different Regions. Check the current prices before you estimate, because they vary by Region.</p><p>When the migration is verified, remove what you created so nothing keeps running or stays open to another account.</p><pre>aws datasync delete-task --task-arn TASK_ARN --region DEST_REGION --profile datasync-admin<br>aws datasync delete-location --location-arn SOURCE_LOCATION_ARN --region SOURCE_REGION --profile datasync-admin<br>aws datasync delete-location --location-arn DEST_LOCATION_ARN --region DEST_REGION --profile datasync-admin<br>aws logs delete-log-group --log-group-name /aws/datasync/s3-cross-account --region DEST_REGION --profile datasync-admin</pre><p>Then delete the two service roles, remove the DataSync statements from the destination bucket policy, and delete the named profile if you no longer need it.</p><p>The pattern to remember is simple. The source account holds the roles and runs DataSync, the destination bucket grants access through its policy, and every location lives in its own bucket’s Region. Get those three right and the rest is routine.</p><h3>Checklist</h3><ol><li>The destination bucket exists with ACLs disabled.</li><li>Both service roles exist and trust the DataSync service.</li><li>The destination bucket policy names the DataSync role.</li><li>Each location was created in its own bucket’s Region.</li><li>The log group has a resource policy, and the task mode is chosen.</li><li>Object counts and sizes match, and the logs show no errors.</li><li>The task, locations, roles and policy statements are removed.</li></ol><h3>References</h3><ul><li><a href="https://docs.aws.amazon.com/datasync/latest/userguide/tutorial_s3-s3-cross-account-transfer.html">Tutorial, transferring data between S3 buckets across AWS accounts</a></li><li><a href="https://aws.amazon.com/blogs/storage/how-to-use-aws-datasync-to-migrate-data-between-amazon-s3-buckets/">How to use AWS DataSync to migrate data between Amazon S3 buckets</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0b2daad8911c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Install Graylog?]]></title>
            <link>https://medium.com/@mustafa.tirnova/how-to-install-graylog-d912aa11a05e?source=rss-b1d0810d5c2f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d912aa11a05e</guid>
            <dc:creator><![CDATA[Mustafa Tırnova]]></dc:creator>
            <pubDate>Tue, 26 Dec 2023 15:19:33 GMT</pubDate>
            <atom:updated>2023-12-26T15:26:05.773Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/964/1*1lB0T5_5ixwjgh64si1YFw.png" /></figure><h4>What is Graylog?</h4><p>Graylog is an open-source log management tool. It provides a scalable and high-performance solution for log collection, storage, searching, analytics, and reporting. Graylog is a system used to collect, store, and search log data from various sources. It supports multiple protocols for collecting log data from different systems. Additionally, Graylog enhances log data by providing features such as search, filtering, and reporting, making the data more meaningful. Due to these features, Graylog can be used for various purposes, including system security, performance monitoring, troubleshooting, etc. In this article, I will explain how to perform the installation of Graylog 5.0. <em>Note that the installation of Elasticsearch, which is a prerequisite for Graylog, is not covered in this article. Therefore, it is necessary to fulfill the prerequisites for Graylog installation. For more information, you can refer to the Graylog and MongoDB documentation in the resources section.</em></p><h4>Why does Graylog require MongoDB and Elasticsearch?</h4><p>Graylog uses MongoDB and Elasticsearch to provide efficient and powerful storage and search capabilities for log data.</p><p><strong>MongoDB</strong> is used by Graylog to store its configurations and metadata. MongoDB is a flexible and scalable document-oriented database that can store complex data structures, making it well-suited for storing Graylog’s configuration and metadata.</p><p>On the other hand, <strong>Elasticsearch</strong> serves as the primary storage for log data. Elasticsearch is a powerful search engine that allows fast and accurate searches through large volumes of log data. It can handle high write loads and can scale horizontally to handle high traffic.</p><p>In summary, MongoDB and Elasticsearch are used together in Graylog because they provide a robust and efficient solution for storing and searching log data. MongoDB provides the necessary functionality to store configurations and metadata, while Elasticsearch provides the necessary functionality to store and search log messages. Together, they offer a comprehensive and powerful log management solution.</p><p>The steps to be performed are:</p><ul><li><strong><em>First, create an Elasticsearch in the database subnet in version 7.10.2 prior to Graylog.</em></strong></li><li><strong><em>Next, create an EC2 instance in the private subnets and perform the installation of MongoDB and Graylog with version 5.</em></strong></li><li><strong><em>Then, create a Network Load Balancer (NLB) to ensure log flow and add listeners to this NLB. Set the ports of the added listeners as inputs to Graylog.</em></strong></li></ul><h3>Installation Process:</h3><p><em>Graylog 5.0 works with the following prerequisites.</em></p><p><strong>Prerequisites:</strong></p><ul><li>OpenJDK 17 (included in the Graylog 5.0 installation package)</li><li>Elasticsearch 7.10.2 or OpenSearch 2.x</li><li>MongoDB (5.x or 6.x)</li></ul><p>Before installing Graylog, we will first perform the installation of MongoDB 5.0. You can follow the steps below.</p><p><strong>MongoDB 5.0 Installation</strong></p><pre>wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -<br>// After this command, you should see an OK response on your screen. However, if you encounter an error indicating that &quot;gnupg&quot; is not installed, you can follow the 2 steps below.<br>   → sudo apt-get install gnupg<br>   → wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -<br><br>echo &quot;deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/5.0 multiverse&quot; | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list<br>sudo apt-get update<br>sudo apt-get install -y mongodb-org=5.0.14 mongodb-org-database=5.0.14 mongodb-org-server=5.0.14 mongodb-org-shell=5.0.14 mongodb-org-mongos=5.0.14 mongodb-org-tools=5.0.14</pre><p><strong>Graylog 5.0 Installation</strong></p><pre>wget https://packages.graylog2.org/repo/packages/graylog-5.0-repository_latest.deb<br>sudo dpkg -i graylog-5.0-repository_latest.deb<br>sudo apt-get update &amp;&amp; sudo apt-get install graylog-server <br>systemctl start graylog-server<br>systemctl enable graylog-server<br>systemctl status graylog-server</pre><p>When the Graylog installation is completed, and we check the status of the service, we will encounter a situation like the following. To ensure the smooth operation of the service, we need to perform the configuration settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0KTxxusskCU6TgkTrLvdWw.png" /></figure><p><strong>Editing the Graylog Configuration File:</strong></p><p>We open the /etc/graylog/server/server.conf file and make the following modifications.</p><pre>#root_username = admin → Uncomment the line.<br>http_bind_address = → Add our own machine&#39;s address.<br>password_secret = → Specify a value for password_secret. Graylog won&#39;t start without this value.<br>root_password_sha2 = → Create root_password_sha2. Graylog won&#39;t start without this value.<br>elasticsearch_hosts = → Write the Elasticsearch endpoint we created.</pre><p>Afterwards, by following the steps below, we complete the Graylog installation. Note: Since Graylog communicates over port 9000, this port needs to be allowed on the machine.</p><pre>sudo systemctl daemon-reload<br>sudo systemctl enable graylog-server.service<br>sudo systemctl start graylog-server.service<br>sudo systemctl --type=service --state=active | grep graylog</pre><p>Afterwards, Graylog becomes active as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tKXa_nYOQ2gAW4AngvEb-Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fWYLIcpKK7x5p-sadzEgvg.png" /></figure><h3>References</h3><p><em>For more information, you can refer to the following resources:</em></p><ul><li><a href="https://go2docs.graylog.org/5">https://go2docs.graylog.org/5</a> 0/downloading_and_installing_graylog/ubuntu_installation.html</li><li><a href="https://www.mongodb.com/docs/v5.0/tutorial/install-mongodb-on-ubuntu/">https://www.mongodb.com/docs/v5.0/tutorial/install-mongodb-on-ubuntu/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d912aa11a05e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python Flask ile yazdığımız uygulamayı Docker ile Containerize etme ve Environment Variables…]]></title>
            <link>https://medium.com/@mustafa.tirnova/python-flask-ile-yazd%C4%B1%C4%9F%C4%B1m%C4%B1z-uygulamay%C4%B1-docker-ile-containerize-etme-ve-environment-variables-9ec954c5551?source=rss-b1d0810d5c2f------2</link>
            <guid isPermaLink="false">https://medium.com/p/9ec954c5551</guid>
            <dc:creator><![CDATA[Mustafa Tırnova]]></dc:creator>
            <pubDate>Thu, 06 Jan 2022 07:55:58 GMT</pubDate>
            <atom:updated>2022-01-06T16:58:11.386Z</atom:updated>
            <content:encoded><![CDATA[<h3>Python Flask ile yazdığımız uygulamayı Docker ile Containerize etme ve Environment Variables kullanımı</h3><p>Arkadaşlar merhaba, Mediumda hazırlamış olduğum ilk yazımda sizlere Python Flask ile oluşturduğumuz basit bir uygulamanın Dockerfile dosyası ile imageini oluşturup ardından bir container içerisinde çalışmasını sağlayacağım.</p><p>İlk olarak “Docker Image ve Container nedir?” sorularına cevap bulduktan sonra yolumuza devam edelim.</p><h4>Docker Image Nedir?</h4><p>En basit haliyle bu soruyu yanıtlayacak olursak, docker image bir uygulamanın ve bu uygulamanın çalışması için gerekli tüm gereksinimlerin(işletim sistemi, ilgili kütüphaneler vb. gibi) bir araya getirilmesinden oluşturulan bir paket diyebiliriz.</p><h4>Docker Container Nedir?</h4><p>Bu tanıma ise vereceğimiz yanıt gayet net olacaktır. Docker Container denildiğinde, akla gelen ilk şey, Docker Image öğesinin çalışan durumuna verilen isim olarak adlandırabiliriz. Aslında bir image öğesinden birden fazla container oluşturabilir ve her bir containeri birbirinden izole bir şekilde çalıştırabiliriz.</p><h3>Uygulamamıza Geçecek Olursak…</h3><h4>1-Basit Python Uygulamasına Göz Atalım</h4><p>İlk olarak Python Flask ile yazdığımız uygulamamıza bir göz atalım. Uygulamamız gayet basit bir şekilde tasarlandı. Yaptığı işlem ise Dockerfile üzerinde “KULLANICIADI” olarak belirlediğimiz bir “environment variables” öğesini belli bir metin taslağı arasında ekrana yazdırmak olacaktır.</p><pre>from flask import Flask<br>import os<br>app = Flask(__name__)<br>@app.route(&quot;/&quot;)<br>def hello():<br>     kullanici= os.getenv(&quot;KULLANICIADI&quot;,&quot;UYARI!Herhangi bir mesaj yok.&quot;)<br>     return &quot;Merhaba, &quot;+ kullanici +&quot; hosgeldiniz.&quot;<br>if __name__ == &quot;__main__&quot;:<br>     app.run(host=&quot;0.0.0.0&quot;, port=int(&quot;5000&quot;), debug=True)</pre><p>Buradaki kodu inceleyecek olursak uygulamanın çalışması için gerekli kütüphaneler 1 ve 2. satırlarda projemize import ediliyor. Ardından sayfamızı oluşturuyoruz, burada dikkat edeceğimiz en önemli yer ise, aşağıdaki komut satırı olacaktır.</p><pre>kullanici= os.getenv(&quot;KULLANICIADI&quot;,&quot;UYARI!Herhangi bir mesaj yok.&quot;)</pre><p>Burada import ettiğimiz “os” kütüphanesinin “getenv” methodu ise, eğer bir ortam değişkeni varsa, bu değeri “kullanici” değişkeninin içerisine döndürür. Yok ise, ikinci parametrede yazılan değeri döndürür. Burada yapmış olduğumuz işlem bu şekildedir.</p><h4>2-Docker File Dosyası Oluşturalım</h4><p>Dockerfile bizim uygulamamızı image haline getirmeye yarayan bir dosyadır. Dockerfile dosyamızda bazı ayarlamalar yapıp uygulamamızı verilen talimatlara göre image haline getirebiliriz. Fakat oluşturacağımız DockerFile dosyası için bazı kurallar söz konusudur. Bulardan ilki dosyanın adı “Dockerfile” şeklinde olmalı ve herhangi bir uzantı almamalıdır. Eğer farklı isimde ya da uzantıda bir Dockerfile dosyası oluşturacaksak bu dosyayı build aşamasında belirtmemiz gereklidir. Belirtmediğimiz takdirde Docker Engine her daim “Dockerfile” dosyasını arayacaktır. İkinci dikkat etmemiz gereken yer ise, Dockerfile dosyasını build ederken, dosya ile aynı dizinde olmamız gerektiğidir. Bu basit kurallara dikkat ettikten sonra Dockerfile dosyamızı oluşturmaya geçebiliriz.</p><pre>FROM python:alpine <br>COPY . /app <br>WORKDIR /app <br>ENV KULLANICIADI=”MUSTAFA” <br>RUN pip install -r requirements.txt <br>EXPOSE 5000 <br>CMD python ./index.py</pre><p>Yukarıdaki kodda da görüldüğü gibi, Dockerfile dosyamızı bu şekilde oluşturuyoruz. Burada bazı talimatlar söz konusudur. Bu talimatlardan kısaca bahsedecek olursak;</p><p><strong>FROM </strong>talimatı ile ilk olarak uygulamamızın python üzerinde çalışacağı base imageini çekiyoruz. Burada alpine işletim sistemi üzerinde python kurulu olarak geliyor ve python ile yazılmış uygulamamızın çalışacağı ortam oluşturuluyor. Ardından <strong>COPY </strong>talimatı ile Dockerfile dosyamızın bulunduğu klasördeki tüm dosyaları app klasörüne kopyalıyoruz. <strong>WORKDIR </strong>talimatı ile ise app klasörünün içerisine geçiyoruz. <strong>ENV </strong>talimatıyla ise python flask uygulamamızda oluşturduğumuz değişkene burada değer atıyoruz. Bu ortam değişkeni olarak adlandırılır. <strong>RUN </strong>talimatı ile ise requirements.txt içerisine yazmış olduğumuz flask gereksimini burada yüklüyoruz. Bu işlem flask modülümüzün çalışması için gerekli bir işlemdir. <strong>EXPOSE </strong>talimatı ile ise container içerisine ulaşmak için 5000 portunu belirtiyoruz. Son olarak da <strong>CMD </strong>talimatında ise klasörümüz içerisindeki python uygulamamızı çalıştırıyoruz.</p><p>Şimdi ise iş, oluşturduğumuz bu Dockerfile dosyasını build ederek docker engine üzerinde bir image dosyası oluşturma sırasına geldi.</p><h4>3-Image Dosyası Oluşturma</h4><p>Bu işlem oldukça basit bir işlemdir. Yapmamız gereken tek şey docker CLI ekranında dosyaların bulunmuş olduğu klasöre gelip aşağıdaki komutu girmek olacaktır. Burada dikkat çekmek istediğim yerler ise ilk olarak imageimizin adlandırma biçimidir. Imagemizi adlandırırken dikkat etmemiz gereken bazı kurallar mevcuttur. Bu kurallar ilerleyen zamanlarda imageimizi, dockerhub vb. repolar üzerine push etmek istediğimizde karşılaşacağımız problemlerin önüne geçecektir. Bu isimlendirmede “ornekler/py1” tanımı bu imageinin hangi repository üzerinde bulunduğunu belirtiyor. “v1” ise bu imageinin tag ismini belirtiyor.</p><pre>docker image build -t ornekler/py1:v1 .</pre><p>İşte bu kadar basit bir şekilde girmiş olduğumuz talimatlar ile bir image dosyası oluşturma işlemi gerçekleştirebiliriz. Ardından aşağıdaki gibi bir çıktı alıp, Dockerfile dosyası içerisine kaydettiğimiz 7 talimatında gerçekleştiğini göreceğiz.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/949/1*9prun18yN5ERMzuf4ptOMQ.png" /></figure><p>Gerçekleşen bu talimatlar sonrasında docker engine üzerinde image dosyamızı oluşturmuş olduk. Docker üzerindeki tüm imageleri listelettirmek istersek yapmamız gereken tek şey <strong>docker image ls </strong>komutunu çalıştırmak olacaktır. Bu komut sonrasında oluşturduğumuz image dosyamızın listelendiğini görmüş olacağız.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*S_opTcX9P7WG9i7ZsMyvzw.png" /></figure><p>Sıradaki işlem ise oluşturduğumuz bu image dosyasını container haline getirip ayağa kaldırmak olacaktır. Bu sayede Python ile yapmış olduğumuz uygulamamızı çalıştırmış olacağız.</p><h4>4- Container Oluşturma</h4><p>Şimdi ise sıra, oluşturmuş olduğumuz image dosyasından container oluşturmaya geldi. Bu işlemi oldukça basit bir şekilde gerçekleştirebiliriz. Bunun için yapmamız gereken tek şey, aşağıdaki komutu çalıştırmak olacaktır.</p><pre>docker container run --name pythonUygulamasi -p 80:5000 ornekler/py1:v1 </pre><p>yazılan bu komutu açıklayacak olursak, burada dikkat çekmek istediğim en önemli yer “–p 80:5000” komutu olacaktır. Burada Dockerfile dosyasında 5000 portunu expose ettiğimiz için container üzerinde de bu port ayarlamasını gerçekleştirmemiz gerekmektedir.</p><p>Aşağıdaki görselde de görüldüğü gibi image dosyamızdan container oluşturma işlemimiz gerçekleşmiş bulunmakta ve containerımız çalışır vaziyettedir.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/949/1*VO77hMPfIVb54qGBxfWMhw.png" /></figure><p>Ve son olara şağıdaki görselde de görebileceğimiz gibi uygulamamızı container haline getirip docker üzerinde ayağa kaldırmış olduk. Ayrıca Dockerfile üzerinde tanımlamış olduğumuz environment variables(KULLANICI) değişkenine atamış olduğumuz “MUSTAFA” değerini de ekranda görmekteyiz.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/949/1*PYil52F_SzhFvriH3r38Ow.png" /></figure><p>Evet arkadaşlar medium platformunda yayımlamış olduğum ilk yazımı okuduğunuz için teşekkür ederim. Docker öğrenmeye yeni başlayan biri olarak, bu yazılarımın asıl amacı öğrenirken, aynı zamanda öğretmektir.</p><p>Sü’rçülisan ettiysem affola…</p><p>Uygulamanın kaynak kodlarına Github(<a href="https://github.com/MTirnova">https://github.com/MTirnova</a>) adresim üzerinden ulaşabilirsiniz.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tYxKjlLlZxeujqiWOVTfdw.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ec954c5551" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>