<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[aspnetrun - Medium]]></title>
        <description><![CDATA[The best path to leverage your aspnet skills. Onboarding to .Net Software Architect jobs. Developing production-ready enterprise .Net applications with applying latest architectures and best practices. - Medium]]></description>
        <link>https://medium.com/aspnetrun?source=rss----7897df73dd1c---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>aspnetrun - Medium</title>
            <link>https://medium.com/aspnetrun?source=rss----7897df73dd1c---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 14:49:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/aspnetrun" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Microservices Monitoring with Health Checks using WatchDog]]></title>
            <link>https://medium.com/aspnetrun/microservices-monitoring-with-health-checks-using-watchdog-6b16fdae0349?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/6b16fdae0349</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[resilience]]></category>
            <category><![CDATA[health]]></category>
            <category><![CDATA[dotnet]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Tue, 20 Apr 2021 12:29:12 GMT</pubDate>
            <atom:updated>2021-10-02T11:55:06.590Z</atom:updated>
            <content:encoded><![CDATA[<p><em>In this article, we are going to Developing Microservices Monitoring with Health Checks using WatchDog. We will Use the HealthChecks feature in our back-end ASP.NET microservices.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/954/1*KiPBvPkRFl-nZshn9nWEeA.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/monitor-app-health">https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/monitor-app-health</a></figcaption></figure><p>Microservice architecture have become the new model for building modern cloud-native applications. And microservices-based applications are distributed systems.</p><p>While architecting distributed microservices-based applications, it get lots of benefits like makes easy to scale and manage services, but as the same time, it is increasing interactions between those services have created a new set of problems.</p><p>So we should assume that failures will happen, and we should dealing with unexpected failures, especially in a distributed system. For example in case of network or container failures, microservices must have a strategy to retry requests again.</p><blockquote>How do you handle the complexity that comes with cloud and microservices?</blockquote><p><strong>— Health Monitoring</strong><br>Microservice should design for monitoring with health monitoring, Health monitoring is critical to multiple aspects of operating microservices. <br>By this way, we can understand for a particular microservice is alive and ready to accommodate requests. We can also provide health information to our orchestrator’s cluster, so that the cluster can act accordingly. For example Kubernetes has Liveness and Readiness probes that we can address health check urls.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*T7Ya2LJPfA9wAovap5VeEQ.png" /></figure><p>That make a good health reporting which customized for our microservices <br> like adding sub health checks for underlying database connection, <br>and by this way we can detect and fix issues for your running application much more easily.</p><p>You can see in first picture, for Ordering microservices health checks,<br> it has 3 sub health checks</p><ul><li>1 for self ordering microservices</li><li>2 for underlying database connection,</li><li>3 for underlying rabbitmq connection</li></ul><p>So since all sub conditions healthy, after that we can say ordering microservices is ready to accommodate requests.</p><p>Let’s check our big picture and see what we are going to build one by one.<br>As you can see that, we are in here and start to developing “Microservices Monitoring with Health Checks using WatchDog”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Wl-TneQ9Tbih7Mz03qGdg.png" /></figure><p>We are going to cover the;</p><ul><li>Implement<strong> health checks</strong> in ASP.NET Core services</li><li>Use the <strong>HealthChecks </strong>feature in your back-end ASP.NET microservices</li><li>This will be the aspnet health check implementation with custom health check methods which includes database availabilities — for example in basket microservices, we will add sub-hc conditions for connecting <strong>Redis</strong> and <strong>RabbitMQ</strong>.</li><li>Query microservices to report about their health status</li><li>Returning health status in <strong>JSON format</strong></li><li>Use <strong>Watchdogs </strong>which is separate service that can watch health and load across microservices, and report health about the microservices by querying with the HealthChecks methods.</li></ul><p>We will apply these microservices<strong> Monitoring Health Checks</strong> for all microservices starting with Catalog microservice and continue to add for all microservices. After that we will create a new Microservices which will be “<strong>WebStatus</strong>”. This will implement health check UI and listen all microservices <strong>health checks </strong>and <strong>visualize </strong>with <strong>WatchDog</strong>. Lastly we containerize for all microservices on docker environment and monitoring <strong>healths </strong>with checking “<strong>WebStatus</strong>” microservice.</p><h3>Background</h3><p>This is the introduction of the series. This will be the series of articles. You can follow the series with below links.</p><ul><li><a href="https://mehmetozkaya.medium.com/microservices-observability-resilience-monitoring-on-net-a5dfbdbb0fbd"><strong>0- Microservices Observability, Resilience, Monitoring on .Net</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-observability-with-distributed-logging-using-elasticsearch-and-kibana-79df919997d2"><strong>1- Microservices Observability with Distributed Logging using ElasticSearch and Kibana</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-resilience-and-fault-tolerance-with-applying-retry-and-circuit-breaker-patterns-c32e518db990"><strong>2- Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns using Polly</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-monitoring-with-health-checks-using-watchdog-6b16fdae0349"><strong>3- Microservices Monitoring with Health Checks using WatchDog</strong></a></li></ul><p>We will focus on microservices cross-cutting concerns on these article series.</p><h3>Step by Step Development w/ Udemy Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*BNqi74a9lU-6U0XBxqscvQ.png" /></figure><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=OCTOBER2021"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-aspnetcore-microservices"><strong>Get the Source Code from AspnetRun Microservices Github</strong> </a>— Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.</p><h3>Asp.Net Health Checks</h3><p>When you are working with Microservices, Health monitoring is critical to multiple aspects of operating microservices. Unhealthy services leads to a lot of revenue loss, customer confidence, and negative effects on the business. In order to avoid such situations, we should regularly monitor the status of our apps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*2MIu8isJAq1fVQO8LC6Mbw.png" /></figure><p>Using these health checks, we can be notified to customers about outages beforehand and reduce the impact of the outage by repairing it or notifying the customer about the ongoing outage, leading to trust.<br> While a simple application may require monitoring of its dependencies once a day, a critical application may need to be monitored as often as possible. <br>We are going to provide a status page to view results and add functionality to inform developers about their issues.</p><p>We will use the health check API endpoint, which will return the instant health state (Healthy / Unhealthy / Degraded) of each microservice in response to us. This endpoint will perform the necessary health check checks that we define.</p><h4>WatchDog Health Dashboard</h4><p>First, let’s try to explain the health status we mentioned above:<br><strong>Healthy: </strong>All of our ingredients are healthy.<br><strong>Unhealthy:</strong> There may be a problem with at least one of our components or an unhandled exception may have been thrown during the health check.<br><strong>Degraded: </strong>This indicates that the components are all healthy but the controls are slow or unstable. We get a successful response with a simple database query, but the execution time is higher than we expected.</p><p>Ok now we can talk about implementing basic health monitoring when developing ASP.NET Core Microservices, you can use a built-in health monitoring feature by using a nuget package “<strong>Microsoft.Extension.Diagnostic.HealthCheck</strong>”. <br>These health monitoring features can be enabled by using a set of services and middleware.</p><p>Go to -&gt;<br><a href="https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks#ui-storage-providers">https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks#ui-storage-providers</a></p><p><strong>See Health Checks</strong><br>HealthChecks packages include health checks for:</p><p>Sql Server<br>MySql<br>Oracle<br>Sqlite<br>RavenDB<br>Postgres<br>EventStore<br>…</p><p><strong>See HealthCheckUI</strong><br>The project HealthChecks.UI is a minimal UI interface that stores and shows the health checks results from the configured HealthChecks uris.<br>To integrate HealthChecks.UI in your project you just need to add the HealthChecks.UI services and middlewares available in the package: <strong>AspNetCore.HealthChecks.UI</strong></p><p>So now we are ready to implement health checks.</p><h3>Adding Health Check for Catalog.API Microservices with Checking MongoDb Connection</h3><p>Before we start, we should talk about the HealthChecks nuget packages,</p><p>We are not installment required package, it comes from aspnet framework<br> <br><strong> Nuget Package<br> Install-Package Microsoft.AspNetCore.Diagnostics.HealthChecks</strong></p><p>After that, We are going to;<br> — Add Health Check</p><pre>Startup.cs -&gt; <br> <br>ConfigureServices<br>services.AddHealthChecks();</pre><pre>Configure<br> app.UseEndpoints(endpoints =&gt;<br> {<br> endpoints.MapControllers();<br> endpoints.MapHealthChecks(“/hc”); — → ADDED<br> });</pre><p>Now we can</p><ul><li>TEST:<br><a href="http://localhost:5000/hc">http://localhost:5000/hc</a><br>Healthy</li></ul><p>But for Catalog.API project using mongodb and need to check mongodb connection is ready or not. In case of Catalog.API is working but mongodb container is not running, That means Catalog.API web application can be worked but some of some components not working. In that cases we set Degraded as a Health Check result.</p><p>In order to check mongo connection in HealthCheck</p><p><strong>Nuget Package :</strong><br> Install-Package AspNetCore.HealthChecks.MongoDb</p><p>Add new sub item</p><pre>Startup.cs -&gt; ConfigureServices</pre><pre>services.AddHealthChecks()<br> .AddMongoDb(Configuration[“DatabaseSettings:ConnectionString”], “Catalog MongoDb Health”, HealthStatus.Degraded);</pre><p>Now we can test Catalog hc with mongo db.<br>Stop mongo docker images and test again<br> <br> docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d<br> docker-compose -f docker-compose.yml -f docker-compose.override.yml down</p><ul><li>TEST:<br><a href="http://localhost:5000/hc">http://localhost:5000/hc</a><br>Degraded</li></ul><p>Start mongo docker images and test again<br> <br> docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d<br> docker-compose -f docker-compose.yml -f docker-compose.override.yml down</p><ul><li>TEST: — after run mongo<br><a href="http://localhost:5000/hc">http://localhost:5000/hc</a><br>Healthy</li></ul><h4>Adding json response</h4><p>Nuget Package<br>Install-Package AspNetCore.HealthChecks.UI.Client</p><pre>Startup.cs -&gt; Configure</pre><pre>app.UseEndpoints(endpoints =&gt;<br> {<br> endpoints.MapControllers(); <br> endpoints.MapHealthChecks(“/hc”, new HealthCheckOptions()<br> {<br> Predicate = _ =&gt; true,<br> ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse<br> });<br> });</pre><p>TEST:</p><ul><li>Run the application<br> Change Run Profile to “Catalog.API” and Run the application</li><li><a href="http://localhost:5000/hc">http://localhost:5000/hc</a></li></ul><p><strong>JSON Response :</strong></p><pre>{“status”:”Healthy”,”totalDuration”:”00:00:00.0023926&quot;,”entries”:{“MongoDb Health”:{“data”:{},”duration”:”00:00:00.0021199&quot;,”status”:”Healthy”,”tags”:[]}}}</pre><p>As you can see that, we have added Health Check for Catalog.API Microservices with Checking MongoDb Connection.</p><p><strong>For Basket Microservices</strong>, we should add Redis health check.</p><p>Add Redis HC<br> In order to check redis connection in HealthCheck</p><p><strong>Nuget Packages :<br> Install-Package AspNetCore.HealthChecks.Redis</strong></p><pre>Startup.cs -&gt; ConfigureServices</pre><pre>services.AddHealthChecks() — — — — — — — → ADDED<br> .AddRedis(Configuration[“CacheSettings:ConnectionString”], “Redis Health”, HealthStatus.Degraded);</pre><p><strong>For Shopping.Aggregator and AspnetRunBasics microservices,</strong> we can add UrlGroup in order check sub healths.</p><p>Add Uri Group HC for Shopping.Aggregator and AspnetRunBasics</p><p><strong>Nuget Package :</strong><br><strong> Install-Package AspNetCore.HealthChecks.Uris</strong></p><p>Shopping.Aggregator</p><pre>Startup.cs -&gt; ConfigureServices</pre><pre>services.AddHealthChecks()<br> .AddUrlGroup(new Uri($”{Configuration[“ApiSettings:CatalogUrl”]}/swagger/index.html”), “Catalog.API”, HealthStatus.Degraded)<br> .AddUrlGroup(new Uri($”{Configuration[“ApiSettings:BasketUrl”]}/swagger/index.html”), “Basket.API”, HealthStatus.Degraded)<br> .AddUrlGroup(new Uri($”{Configuration[“ApiSettings:OrderingUrl”]}/swagger/index.html”), “Ordering.API”, HealthStatus.Degraded);</pre><h3>Developing WebStatus App for Centralized Microservices Health Monitoring Using Watchdogs for Visualization</h3><p>First of all, We are going to start with;</p><ul><li>Create New Project<br> AspNet MVC Project<br> Name<br><strong> WebStatus</strong></li><li>Set Port<br> <a href="http://localhost:5007/">http://localhost:5007</a></li><li>Set a Startup Porject<br> Select “WebStatus” Project<br> Run Application</li></ul><p>After that, install required nuget packages</p><h4>Add Nuget Package</h4><p>Install-Package AspNetCore.HealthChecks.UI<br> Install-Package AspNetCore.HealthChecks.UI.InMemory.Storage</p><h4>Manage Startup for Adding HealthChecksUI with Watchdog</h4><p>Startup.cs</p><pre>public void ConfigureServices(IServiceCollection services)<br>{<br> //…<br> // Registers required services for health checks<br> services.AddHealthChecksUI()<br> .AddInMemoryStorage();<br>}<br>//…<br>public void Configure(IApplicationBuilder app, IHostingEnvironment env)<br>{<br> //…<br> app.UseEndpoints(endpoints =&gt;<br> {<br> endpoints.MapHealthChecksUI(); — — — -&gt; ADDED<br> //…<br>}</pre><p>As you can see that, we have Developing WebStatus App for Centralized Microservices Health Monitoring.<br>After adding packages and startup developments, We should configure our Health Monitoring uris in order to check hc results</p><p>Configuration file for health check UI:</p><p><strong>appsettings.json</strong></p><pre>“HealthChecks-UI”: {<br> “HealthChecks”: [<br> {<br> “Name”: “Catalog Health Check”,<br> “Uri”: “<a href="http://localhost:5000/hc">http://localhost:5000/hc</a>”<br> },<br> {<br> “Name”: “Basket Health Check”,<br> “Uri”: “<a href="http://localhost:5001/hc">http://localhost:5001/hc</a>”<br> }<br> ]<br> },</pre><p>For testing purpose, this time I am adding only Catalog and Basket hc. <br> But after test I will add for all microservices with docker urls.</p><p>Of course we need to <br> — Redirect Home Page to HC</p><pre>HomeController</pre><pre>public IActionResult Index()<br> {<br> return Redirect(“/healthchecks-ui”); — — — → ADDED<br> //return View();<br> }</pre><p>Default UI Path comes below library — <a href="https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks/blob/master/src/HealthChecks.UI/Configuration/Options.cs">https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks/blob/master/src/HealthChecks.UI/Configuration/Options.cs</a><br> public string UIPath { get; set; } = “/healthchecks-ui”;</p><p>Now we can test the application</p><ul><li>Set Multiple Run Profile<br> Catalog.API<br> Basket.API<br> WebStatus</li><li>Run Application :</li></ul><p>localhost:5007/healthchecks-ui</p><p>See dashboard worked !</p><h4>Adding All Microservices HC urls with docker uris<br> — Add All HC uri s into Configuration</h4><p>appsettings.json file</p><pre>“HealthChecks-UI”: {<br> “HealthChecks”: [<br> {<br> “Name”: “Catalog Health Check”,<br> “Uri”: “<a href="http://localhost:8000/hc">http://localhost:8000/hc</a>&quot;<br> },<br> {<br> “Name”: “Basket Health Check”,<br> “Uri”: “<a href="http://localhost:8001/hc">http://localhost:8001/hc</a>&quot;<br> },<br> {<br> “Name”: “Discount Health Check”,<br> “Uri”: “<a href="http://localhost:8002/hc">http://localhost:8002/hc</a>&quot;<br> },<br> {<br> “Name”: “Ordering Health Check”,<br> “Uri”: “<a href="http://localhost:8004/hc">http://localhost:8004/hc</a>&quot;<br> },<br> {<br> “Name”: “Shopping Aggregator Health Check”,<br> “Uri”: “<a href="http://localhost:8005/hc">http://localhost:8005/hc</a>&quot;<br> },<br> {<br> “Name”: “AspnetRunBasics WebMVC Health Check”,<br> “Uri”: “<a href="http://localhost:8006/hc">http://localhost:8006/hc</a>&quot;<br> }<br> ],</pre><p>We have configured for docker urls. But now that microservices docker images not included hc developments. So We will test with building again full images. Because we have added hc developments.</p><p>As you can see that, we have finished to Development of <strong>WebStatus </strong>App for Centralized Microservices Health Monitoring Using Watchdogs for Visualization.</p><h3>Containerize WebStatus Health Monitoring Microservices using Docker Compose</h3><p>We are going to Containerize WebStatus Health Monitoring Microservices using Docker Compose. We are going to add Docker-Compose File for WebStatus Application Microservices Solution.</p><p>First, We should go <strong>WebStatus </strong>project. In order to create DockerFile and docker compose file we use Visual Studio Container Orchestrator Support.</p><p>In the Discount.API project, <br>Right Click <br> choose Add &gt; ..Container Orchestrator Support. <br> The Docker Support Options dialog appears.<br> Choose Docker Compose.<br> Choose Linux.</p><p>DockerFile and docker-compose created.</p><p>Visual Studio creates DockerFile and update the docker-compose.yml file with newly added <strong>WebStatus </strong>images.</p><h4>Examine Docker-Compose yaml file</h4><p>We should Format docker-compose yaml files with tabs. These yaml files is very strict to indentation of the commands.</p><h4>docker-compose.yml</h4><pre>version: ‘3.4’<br>..</pre><pre>webstatus:<br> image: ${DOCKER_REGISTRY-}webstatus<br> build:<br> context: .<br> dockerfile: WebApps/WebStatus/Dockerfile</pre><h4>docker-compose.override.yml</h4><pre>version: ‘3.4’<br>…</pre><pre>webstatus:<br> container_name: webstatus<br> environment:<br> — ASPNETCORE_ENVIRONMENT=Development<br> — HealthChecksUI__HealthChecks__0__Name=Catalog Health Check<br> — HealthChecksUI__HealthChecks__0__Uri=<a href="http://catalog.api/hc">http://catalog.api/hc</a><br> — HealthChecksUI__HealthChecks__1__Name=Basket Health Check<br> — HealthChecksUI__HealthChecks__1__Uri=<a href="http://basket.api/hc">http://basket.api/hc</a><br> — HealthChecksUI__HealthChecks__2__Name=Discount Health Check<br> — HealthChecksUI__HealthChecks__2__Uri=<a href="http://discount.api/hc">http://discount.api/hc</a><br> — HealthChecksUI__HealthChecks__3__Name=Ordering Health Check<br> — HealthChecksUI__HealthChecks__3__Uri=<a href="http://ordering.api/hc">http://ordering.api/hc</a><br> — HealthChecksUI__HealthChecks__4__Name=Shopping Aggregator Health Check<br> — HealthChecksUI__HealthChecks__4__Uri=<a href="http://shopping.aggregator/hc">http://shopping.aggregator/hc</a><br> — HealthChecksUI__HealthChecks__5__Name=AspnetRunBasics WebMVC Health Check<br> — HealthChecksUI__HealthChecks__5__Uri=<a href="http://aspnetrunbasics/hc">http://aspnetrunbasics/hc</a> <br> ports:<br> — “8007:80”</pre><p>Overriding existing appsettings on docker container. Overriding array appsettings on Docker we used double __ characters.</p><p>You can see that we have override ocelot HealthChecks api url with docker container names.</p><p>As a summary, we have created Dockerfile and docker-compose files for WebStatus Health Monitoring Application configuration.<br> So now we can test all microservices with in docker-compose environment.</p><h4>Test on Docker environment — WebStatus Health Monitoring Microservices into Docker Compose for Visulize WatchDog HC</h4><p>Close all dockers and run with below command on that location;<br> <br> Start/Stop Docker compose :</p><pre>docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d<br> docker-compose -f docker-compose.yml -f docker-compose.override.yml down</pre><ul><li>Check the WebStatus</li><li>WebStatus — HC List<br><a href="http://localhost:8007/">http://localhost:8007</a><br><a href="http://host.docker.internal:8007/">http://host.docker.internal:8007</a></li></ul><p>As you can see that, we have tested all microservices on docker environment with heath check features on WebStatus Health Monitoring Microservices.</p><h3>Step by Step Development w/ Udemy Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*BNqi74a9lU-6U0XBxqscvQ.png" /></figure><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=OCTOBER2021"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b16fdae0349" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/microservices-monitoring-with-health-checks-using-watchdog-6b16fdae0349">Microservices Monitoring with Health Checks using WatchDog</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns…]]></title>
            <link>https://medium.com/aspnetrun/microservices-resilience-and-fault-tolerance-with-applying-retry-and-circuit-breaker-patterns-c32e518db990?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/c32e518db990</guid>
            <category><![CDATA[circuit-breaker]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[fault-tolerance]]></category>
            <category><![CDATA[resilience]]></category>
            <category><![CDATA[retry]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Tue, 20 Apr 2021 12:29:02 GMT</pubDate>
            <atom:updated>2021-10-02T11:54:18.466Z</atom:updated>
            <content:encoded><![CDATA[<h3>Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns using Polly</h3><p><em>In this article, we are going to Developing “Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns using Polly”.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qwb7-X9__R3X7Mp859boNQ.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/application-resiliency-patterns">https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/application-resiliency-patterns</a></figcaption></figure><p>Microservice architecture have become the new model for building modern cloud-native applications. And microservices-based applications are distributed systems.</p><p>While architecting distributed microservices-based applications, it get lots of benefits like makes easy to scale and manage services, but as the same time, it is increasing interactions between those services have created a new set of problems.</p><p>So we should assume that failures will happen, and we should dealing with unexpected failures, especially in a distributed system. For example in case of network or container failures, microservices must have a strategy to retry requests again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PrOy0jg7EPqYt5Fou4uzIQ.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/handle-partial-failure">https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/handle-partial-failure</a></figcaption></figure><p>But What happens when the machine where the microservice is running fails?</p><p>For instance, a single microservice can fail or might not be available to respond for a short time. Since clients and services are separate processes, a service might not be able to respond in a timely way to a client’s request. <br>The service might be overloaded and responding very slowly to requests or might simply not be accessible for a short time because of network issues.</p><blockquote>How do you handle the complexity that comes with cloud and microservices?</blockquote><p><strong>— Microservices Resilience</strong></p><p>Microservice should design for resiliency, A microservice needs to be resilient to failures and must accept partial failures. We should Design microservices to be resilient for these partial failures. They should ability to recover from failures and continue to function.</p><p>We should accepting failures and responding to them without any downtime or data loss. The main purpose of resiliency microservices is to return the application to a fully functioning state after a failure.</p><p>So when we are architecting distributed cloud applications, we should assume that failures will happen and design our microservices for resiliency. We accept that Microservices are going to fail at some point, that&#39;s why we need to learn embracing failures.</p><p>Let’s check our big picture and see what we are going to build one by one.<br>As you can see that, we are in here and start to developing “Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns using Polly”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*axQ9LUNyHJJA_knHhp56hg.png" /></figure><p>We are going to cover the;</p><ul><li>Start with Microservices Resilience Patterns</li><li>Definition of Retry pattern</li><li>Definition of Circuit-Breaker patterns</li><li>Definition of Bulkhead pattern</li><li>Appling Retry patterns using Polly</li><li>Appling Circuit-Breaker patterns using Polly</li></ul><p>We will apply these microservices resilience patterns on our microservices communications which specifically</p><ul><li>Shopping Client microservices</li><li>Ocelot Api GW microservices</li><li>Shopping.Aggregator microservices</li></ul><p>These are communicating internal microservices in order to perform their operations, so these communications should be resilient with applying retry and circuit-breaker patterns.</p><p>Also Applying <strong>Retry pattern</strong> for <strong>Database migrate operations.</strong></p><ul><li>When Discount microservices startup, we will create database, table and seed table in PostreSQL database in order to perform migration. This will retry by applying retry pattern with polly.</li><li>When Ordering microservices startup, we will create database, table and seed table in SQLServer database in order to perform migration. This will retry by applying retry pattern with polly.</li></ul><p>So in this article, we are going to Develop our “Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns using Polly”.<br>By the end of the article, we will make stronger for our microservices reference application with applying microservices resilience patterns.</p><h3>Background</h3><p>This is the introduction of the series. This will be the series of articles. You can follow the series with below links.</p><ul><li><a href="https://mehmetozkaya.medium.com/microservices-observability-resilience-monitoring-on-net-a5dfbdbb0fbd"><strong>0- Microservices Observability, Resilience, Monitoring on .Net</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-observability-with-distributed-logging-using-elasticsearch-and-kibana-79df919997d2"><strong>1- Microservices Observability with Distributed Logging using ElasticSearch and Kibana</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-resilience-and-fault-tolerance-with-applying-retry-and-circuit-breaker-patterns-c32e518db990"><strong>2- Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns using Polly</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-monitoring-with-health-checks-using-watchdog-6b16fdae0349"><strong>3- Microservices Monitoring with Health Checks using WatchDog</strong></a></li></ul><p>We will focus on microservices cross-cutting concerns on these article series.</p><h3>Step by Step Development w/ Udemy Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*BNqi74a9lU-6U0XBxqscvQ.png" /></figure><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=OCTOBER2021"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-aspnetcore-microservices"><strong>Get the Source Code from AspnetRun Microservices Github</strong> </a>— Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.</p><h3>Microservices Resilience Patterns</h3><p>In order to provide unbroken microservice, the architecture must be designed correctly, and the developers must develop applications that are suitable for this architecture and will not cause errors. <br>Unfortunately, we may not always achieve this, but fortunately there are some approaches that we can apply to provide uninterrupted service, the name of these approaches is “<strong>Resilience Pattern</strong>”.</p><p>Resilience Patterns can be divided into different categories according to the problem area they solve, and it is possible to examine them under a separate topics, but in this article, we will look at the most needed and essential design patterns.</p><p>Ensuring the durability of services in microservice architecture is relatively difficult compared to a monolithic architecture. In the microservice architecture, the communication between services is distributed and many internal or external network traffic is created for a transaction. As the communication need between services and dependence on external services increases, the possibility of occurring errors will increase.</p><h4>Polly Policies</h4><p>We will use Polly when implementing microservices resilience patterns.</p><p>Polly is a .NET resilience and transient-fault-handling library that allows developers to express policies such as Retry, Circuit Breaker, Timeout, Bulkhead Isolation, and Fallback in a fluent and thread-safe manner.</p><p>Go to;<br> <a href="https://github.com/App-vNext/Polly">https://github.com/App-vNext/Polly</a></p><p>You can check Resilience policies. Polly offers multiple resilience policies:</p><ul><li><strong>Retry</strong><br>Configures retry operations on designated operations</li><li><strong>Circuit Breaker</strong><br>Blocks requested operations for a predefined period when faults exceed a configured threshold.</li><li><strong>Timeout</strong><br>Places limit on the duration for which a caller can wait for a response.</li><li><strong>Bulkhead</strong><br>Constrains actions to fixed-size resource pool to prevent failing calls from swamping a resource.</li><li><strong>Cache</strong><br>Stores responses automatically.</li><li><strong>Fallback</strong><br>Defines structured behavior upon a failure.</li></ul><h3>Retry Pattern</h3><p>Microservices Inter-service communication can be performed by HTTP or gRPC and can be managed by developments. But when in comes to network, server, and such physical interruptions may be unavoidable.</p><p>If one of the services returns http 500 during the transaction, <br>the transaction may be interrupted and an error may be returned to the user, <br>but when the user restarts the same transaction, it may work.</p><p>In such a case, it would be more logical to repeat the request made to the 500 returned service, so that the roll-backs of the realized transactions will not having to work, and the user will be able to perform the transaction successfully, even if it is late.</p><p>So we can say that a distributed cloud-native environment, <br>microservices communications can fail because of transient failures, <br>but this failures happen in a short-time and can fix after that time.<br>For that cases, we should implementing retry pattern.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kdAwkkfqeGOLY253wSzi4A.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/application-resiliency-patterns">https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/application-resiliency-patterns</a></figcaption></figure><p>In the image, the client requested to api gateway in order to call internal microservices. Api Gateway Service needs to access internal Microservice and creates a request, so if internal microservice has failed for a short-time and returns http 500 or gets an error, the transaction will be terminated,</p><p>But when the Retry design pattern is applied, the specified number of transactions will be tried and may be performed successfully.</p><p>For this image, api gateway get 2 times 500 response from internal microservice and finally the third call got succeed, so that means the transaction completed successfully without termination.</p><p>In order to allow the microservice time to fix itself with self-correct, <br>it is important to extend the back-off time before retrying the call.<br>In Most usages the back-off period should be exponentially incremental withdrawal to allow sufficient correction time.</p><h3>Circuit Breaker Pattern</h3><p>Circuit Breakers pattern, is a method in electronic circuits that is constructed like circuit breaker switchgear, as the name suggested. Circuit breakers stop the load transfer in case of a failure in the system in order to protect the electronic circuit.</p><p>When the Circuit Breakers pattern is applied, it is constructed to include communication between services. It monitors the communication between the services and follows the errors that occur in the communication. An example of this error is that an API end that has been requested returns an HTTP 500 error code.</p><p>When the error in the system exceeds a certain threshold value, <br>Circuit Breakers turn on and cut off communication, and returning previously determined error messages.</p><p>While Circuit Breakers is open, it continues to monitor the communication traffic and if the requested service starts to return successful results, it becomes closed.</p><p>If you look at the image, the circuit breaker has 3 basic modes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/1*RwwP-WYYDG7EiyvJiHuK3A.png" /><figcaption><a href="https://www.gokhan-gokalp.com/en/resiliency-patterns-in-microservice-architecture/">https://www.gokhan-gokalp.com/en/resiliency-patterns-in-microservice-architecture/</a></figcaption></figure><p><strong>Closed:</strong> In this mode, the circuit breaker is not open and all requests are executed.</p><p><strong>Open</strong>: the circuit breaker is open and it prevents the application from repeatedly trying to execute an operation while an error occurs.</p><p><strong>Half-Open:</strong> In this mode, the circuit breaker executes a few operations <br> to identify if an error still occurs. If errors occur, then the circuit breaker will be opened, if not it will be closed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qwb7-X9__R3X7Mp859boNQ.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/application-resiliency-patterns">https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/application-resiliency-patterns</a></figcaption></figure><p>In the image, the client requested to api gateway in order to call internal microservices. And we have implemented retry pattern on this communication. But what if internal microservice is totally down, and it can’t be ready for a long time.</p><p>For that cases if we repeated retries on an unresponsive service, <br>this will consume resources such as memory, threads, and database connections and cause irrelevant failures.</p><p>So for that reason, we have applied Circuit Breaker pattern over the retry pattern. As you can see that, in here, after 100 failed requests in 30 seconds, <br> the circuit breakers opens and no longer allows calls to the internal microservice from api gateway. That means circuit breaker pattern applied in the api gateway when calling internal microservices. After 30 second, api gateway try to call microservices again and If that call succeeds, the circuit closes and the service is once again available to traffic.</p><p>The Circuit Breaker pattern is prevent broken communications from repeatedly trying to send request that is mostly to fail. We set the pre-defined number of failed calls and time period of this fails, if fails happen in that time interval than it blocks all traffic to the service.</p><p>After that it is periodically try to call the target service in order to determine its up or down. If the service call succeed than close the circuit and continue to accept requests again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*GVtGhi3PUR4pH2YuNeKm3Q.png" /><figcaption><a href="https://www.gokhan-gokalp.com/en/resiliency-patterns-in-microservice-architecture/">https://www.gokhan-gokalp.com/en/resiliency-patterns-in-microservice-architecture/</a></figcaption></figure><h4>Retry + Circuit Breaker Pattern</h4><p>If we compare with the Retry pattern, Retry pattern only retry an operation in the expectation calls since it will get succeed. But The Circuit Breaker pattern is prevent broken communications from repeatedly trying to send request that is mostly to fail.</p><p>So we are going to apply both Retry and Circuit-Breaker Pattern in our microservices reference application with using Polly library.<br> <a href="https://github.com/App-vNext/Polly">https://github.com/App-vNext/Polly</a></p><p>We will examine examples of github readme file and develop for our microservices.</p><p>For further explanations on this topic, you should check microsoft architecture book which name is — Architecting Cloud Native .NET Applications for Azure written from Rob Vettor and Steve Smith</p><p>Go to<br> <a href="https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/">https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/</a></p><p>If you come to topic of Cloud-native resiliency. You can get additional details of this topic. Also our images comes from this book, this is free book from Microsoft.</p><h3>Bulkhead Pattern</h3><p>The Bulkhead design pattern aims to isolate the occurring error which is in one location does not affect other services by isolating the services. <br>The main purpose is isolated error place and secure the rest of services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*aw0JjmzoEEeKAU_HOJLY6Q.png" /></figure><p>The inspiration for this design pattern is naval architecture. While ships or submarines are being built, they are made not as a whole, but by shielding from certain areas, so if there is happens a flood or fire, the relevant compartment is closed and isolated and the ship / submarine can continue its mission.</p><p>If we think of our microservices as a ship, we must be able to respond to user requests by isolating the relevant service so the error that will occur in a service does not affect other services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*h-UjfCCEA2bmAsoX2h94bA.png" /><figcaption><a href="https://speakerdeck.com/slok/resilience-patterns-server-edition?slide=33">https://speakerdeck.com/slok/resilience-patterns-server-edition?slide=33</a></figcaption></figure><p>See image here, we can better understand why we need the bulkhead design pattern. In case of the http requests from users comes too high, physical machines will not be able to process all requests at the same time and will naturally put them in order.</p><p>Let’s assume that users make a request to service A, service A doesn’t respond due to a software bug, and user requests start to accumulate in the queue as shown in number two. After a certain time, the requests will start to use all CPU threads of the machines and after a while, all new requests will start to wait in the queue.</p><p>While all requests to service A that are not working are busy with resources, <br>requests to service B that are running successfully will continue to wait in the queue, in this case, none of the demanding users will be satisfied.</p><p>In order to solve this problem, the bulkhead design pattern also suggests that <br>when a service is unresponsive, instead of using all resources to process requests to that service, some of the resources are allocated in a separate process and other requests are processed with the remaining server resource.</p><p>To implement this design pattern, resource limits can be defined for the application server where the services are running, for example, the maximum number of threads and memory limit that the service can use.</p><h3>Apply Retry Pattern with Polly policies on HttpClientFactory for Shopping.Aggregator Microservices</h3><p>We are going to Apply Retry Pattern with Polly policies on HttpClientFactory for Shopping.Aggregator Microservices. As you remember that when we developed Shopping.Aggregator microservices, this will consume internal microservices with using HttpClientFactory and agregate Catalog, Basket and Ordering microservices data in one web service.<br>If one of the internal microservices is temporary down or not accessible, we will apply retry pattern with using Polly policies on HttpClientFactory for Shopping.Aggregator Microservices.</p><blockquote>So how Polly policies can apply retry pattern on HttpClientFactory ?</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*FxTyWiId8CqdALofZG3w3Q.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests">https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests</a></figcaption></figure><p>You can see in this picture, we are going to</p><ul><li>Use IHttpClientFactory to implement resilient HTTP requests</li><li>Implement HTTP call retries with exponential backoff with IHttpClientFactory and Polly policies</li></ul><p>You can find the main article of this picture,<br> <a href="https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests">https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests</a></p><p>Before we start, we should go to Shopping.Aggregator Microservices<br>In this microservices we Use <strong>IHttpClientFactory </strong>to implement resilient HTTP requests. <br>We have Implemented <strong>Typed Client </strong>classes that use the injected and configured <strong>HttpClient </strong>with <strong>IHttpClientFactory</strong></p><p>Go to<strong> Shopping.Aggregator</strong><br><strong>Startup.cs — ConfigureServices</strong></p><pre>services.AddHttpClient&lt;ICatalogService, CatalogService&gt;(c =&gt;<br> c.BaseAddress = new Uri(Configuration[“ApiSettings:CatalogUrl”]))<br> .AddHttpMessageHandler&lt;LoggingDelegatingHandler&gt;();<br> services.AddHttpClient&lt;IBasketService, BasketService&gt;(c =&gt;<br> c.BaseAddress = new Uri(Configuration[“ApiSettings:BasketUrl”]))<br> .AddHttpMessageHandler&lt;LoggingDelegatingHandler&gt;();<br> services.AddHttpClient&lt;IOrderService, OrderService&gt;(c =&gt;<br> c.BaseAddress = new Uri(Configuration[“ApiSettings:OrderingUrl”]))<br> .AddHttpMessageHandler&lt;LoggingDelegatingHandler&gt;();</pre><p>We have 3 api connection with <strong>HttpClientFactory</strong>. So we are going to Develop Policies for <strong>Retry </strong>and <strong>Circuit Breaker Pattern</strong> with <strong>Polly policies </strong>on <strong>HttpClientFactory</strong>.<br>Let’s take an action.</p><p><strong>Install Nuget Package</strong><br> Install-Package Microsoft.Extensions.Http.Polly</p><pre>See that<br> &lt;PackageReference Include=”Microsoft.Extensions.Http.Polly” Version=”5.0.1&quot; /&gt;</pre><p>Before we start, we should check Polly repository on github</p><h4>Polly Policies :</h4><p><a href="https://github.com/App-vNext/Polly">https://github.com/App-vNext/Polly</a></p><p>// Retry a specified number of times, using a function to <br>// calculate the duration to wait between retries based on <br>// the current retry attempt (allows for exponential backoff)<br>// In this case will wait for<br>// 2 ^ 1 = 2 seconds then<br>// 2 ^ 2 = 4 seconds then<br>// 2 ^ 3 = 8 seconds then<br>// 2 ^ 4 = 16 seconds then<br>// 2 ^ 5 = 32 seconds<br>Policy<br> .Handle&lt;SomeExceptionType&gt;()<br> .WaitAndRetry(5, retryAttempt =&gt; <br> TimeSpan.FromSeconds(Math.Pow(2, retryAttempt))</p><p>If you check Retry operation, you can see example of usage with custom wait strategy.</p><h4>Developing Retry and Circuit Breaker Pattern</h4><p>Lets develop custom policy methods with detail policies.</p><p>Go to Shopping.Aggregator<br><strong>Startup.cs</strong></p><pre>private static IAsyncPolicy&lt;HttpResponseMessage&gt; GetRetryPolicy()<br> {<br> // In this case will wait for<br> // 2 ^ 1 = 2 seconds then<br> // 2 ^ 2 = 4 seconds then<br> // 2 ^ 3 = 8 seconds then<br> // 2 ^ 4 = 16 seconds then<br> // 2 ^ 5 = 32 seconds</pre><pre>return HttpPolicyExtensions<br> .HandleTransientHttpError()<br> .WaitAndRetryAsync(<br> retryCount: 5,<br> sleepDurationProvider: retryAttempt =&gt; TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)),<br> onRetry: (exception, retryCount, context) =&gt;<br> {<br> Log.Error($”Retry {retryCount} of {context.PolicyKey} at {context.OperationKey}, due to: {exception}.”);<br> });<br> }</pre><pre>private static IAsyncPolicy&lt;HttpResponseMessage&gt; GetCircuitBreakerPolicy()<br> {<br> return HttpPolicyExtensions<br> .HandleTransientHttpError()<br> .CircuitBreakerAsync(<br> handledEventsAllowedBeforeBreaking: 5,<br> durationOfBreak: TimeSpan.FromSeconds(30)<br> );<br> }</pre><p>After that, we should refactor our policies. <br> This time we will use “<strong>AddPolicyHandler</strong>” method.</p><h4>Change ConfigureServices</h4><p>Go to<strong> Shopping.Aggregator</strong><br><strong>Startup.cs — ConfigureServices</strong></p><pre>public void ConfigureServices(IServiceCollection services)<br> {<br> services.AddTransient&lt;LoggingDelegatingHandler&gt;();</pre><pre>services.AddHttpClient&lt;ICatalogService, CatalogService&gt;(c =&gt;<br> c.BaseAddress = new Uri(Configuration[“ApiSettings:CatalogUrl”]))<br> .AddHttpMessageHandler&lt;LoggingDelegatingHandler&gt;()<br><strong> .AddPolicyHandler(GetRetryPolicy()) — -&gt; ADDED<br> .AddPolicyHandler(GetCircuitBreakerPolicy()); — -&gt; ADDED</strong></pre><pre>services.AddHttpClient&lt;IBasketService, BasketService&gt;(c =&gt;<br> c.BaseAddress = new Uri(Configuration[“ApiSettings:BasketUrl”]))<br> .AddHttpMessageHandler&lt;LoggingDelegatingHandler&gt;()<br><strong> .AddPolicyHandler(GetRetryPolicy()) — -&gt; ADDED<br> .AddPolicyHandler(GetCircuitBreakerPolicy()); — -&gt; ADDED</strong></pre><pre>services.AddHttpClient&lt;IOrderService, OrderService&gt;(c =&gt;<br> c.BaseAddress = new Uri(Configuration[“ApiSettings:OrderingUrl”]))<br> .AddHttpMessageHandler&lt;LoggingDelegatingHandler&gt;()<br><strong> .AddPolicyHandler(GetRetryPolicy()) — -&gt; ADDED<br> .AddPolicyHandler(GetCircuitBreakerPolicy()); —-&gt; ADDED</strong></pre><h4>TEST Polly :</h4><p>Make sure that your docker microservices working fine.</p><p>Run Docker compose :</p><pre>docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d<br>docker-compose -f docker-compose.yml -f docker-compose.override.yml down</pre><p>Now we can test application retry operations.</p><p>Set a Startup Project<br><strong> Shopping.Aggregator</strong></p><p><strong>Change App url</strong></p><pre> “ApiSettings”: {<br> “CatalogUrl”: “<a href="http://localhost:8000/">http://localhost:8000</a>&quot;,<br> “BasketUrl”: “<a href="http://localhost:8009/">http://localhost:8009</a>&quot;,-- CHANGE 8001 to 8009</pre><ul><li>Run Application<br> test with <br> swn</li><li><strong>See logs</strong><br> As you can see that retry 3 times</li></ul><p>As you can see that, we have Developed Policies for Retry and Circuit Breaker Pattern with Polly policies on HttpClientFactory.</p><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=RESILIENCE"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><p>For the next articles -&gt;</p><ul><li><a href="https://mehmetozkaya.medium.com/microservices-monitoring-with-health-checks-using-watchdog-6b16fdae0349"><strong>3- Microservices Monitoring with Health Checks using WatchDog</strong></a></li></ul><h3>Step by Step Development w/ Udemy Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*BNqi74a9lU-6U0XBxqscvQ.png" /></figure><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=OCTOBER2021"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c32e518db990" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/microservices-resilience-and-fault-tolerance-with-applying-retry-and-circuit-breaker-patterns-c32e518db990">Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns…</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Microservices Observability with Distributed Logging using ElasticSearch and Kibana]]></title>
            <link>https://medium.com/aspnetrun/microservices-observability-with-distributed-logging-using-elasticsearch-and-kibana-79df919997d2?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/79df919997d2</guid>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[kibana]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[serilog]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Tue, 20 Apr 2021 12:28:51 GMT</pubDate>
            <atom:updated>2021-10-02T11:52:27.987Z</atom:updated>
            <content:encoded><![CDATA[<p><em>In this article, we are going to Developing “Microservices Observability with Distributed Logging using ElasticSearch and Kibana”.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-j8W5EW-YONtl9BSrzJaAA.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/observability-patterns">https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/observability-patterns</a></figcaption></figure><p>Microservice architecture have become the new model for building modern cloud-native applications. And microservices-based applications are distributed systems.</p><p>How do you handle the complexity that comes with cloud and microservices? — <strong>Observability</strong></p><p>Microservice should have a strategy for <strong>monitoring </strong>and <strong>managing </strong>the complex dependencies on microservices. That means we need to implement <strong>microservices observability with using distributed logging features.</strong><br>Microservices Observability gives us greater operational insight and leading to understand incidents on our microservices architecture.</p><p>Let’s check our big picture and see what we are going to build one by one.<br>As you can see that, we are in here and start to developing “Microservices Observability with Distributed Logging”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EVdSa1tEI6i9G_19zb5JtQ.png" /></figure><p>We are going to cover the;</p><ul><li>Start with Asp.Net logging basics and understand how to logging works on asp.net</li><li>Applying <strong>Elastic Stack</strong> which includes <strong>Elasticsearh + Logstach + Kibana</strong></li><li>Use <strong>SeriLog nuget package </strong>for creating structured logs in .net microservices</li><li>Use <strong>docker-compose Kibana image</strong> from docker hub and feed <strong>Kibana </strong>with <strong>elastic stack</strong></li><li>That means we Containerize <strong>ElasticSearch </strong>and <strong>Kibana </strong>on our docker environment with using Docker Compose</li></ul><p>So in this article, we are going to Develop our “<strong>Microservices Observability with Distributed Logging implementing</strong> <strong>ElasticStack </strong>on Asp.Net Microservices”. By the end of the article, we will have ElasticSearch and Kibana docker images and feeding this systems from our Asp.net microservices with creating structure logs using SeriLog nuget package.</p><h3>Background</h3><p>This is the introduction of the series. This will be the series of articles. You can follow the series with below links.</p><ul><li><a href="https://mehmetozkaya.medium.com/microservices-observability-resilience-monitoring-on-net-a5dfbdbb0fbd"><strong>0- Microservices Observability, Resilience, Monitoring on .Net</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-observability-with-distributed-logging-using-elasticsearch-and-kibana-79df919997d2"><strong>1- Microservices Observability with Distributed Logging using ElasticSearch and Kibana</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-resilience-and-fault-tolerance-with-applying-retry-and-circuit-breaker-patterns-c32e518db990"><strong>2- Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns using Polly</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-monitoring-with-health-checks-using-watchdog-6b16fdae0349"><strong>3- Microservices Monitoring with Health Checks using WatchDog</strong></a></li></ul><p>We will focus on microservices cross-cutting concerns on these article series.</p><h3>Step by Step Development w/ Udemy Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*BNqi74a9lU-6U0XBxqscvQ.png" /></figure><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=OCTOBER2021"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-aspnetcore-microservices"><strong>Get the Source Code from AspnetRun Microservices Github</strong> </a>— Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.</p><h4>What is Elastic Search ?</h4><p>e have said that, we should have a strategy for monitoring and managing the complex dependencies on microservices. That means we need to implement microservices observability with using distributed logging features.<br>Elastic Search is one of the best tools for distributed logging on microservices architectures.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*dOBAESRY95I78ZcKrPVbuw.png" /></figure><h4>Official — ElasticSearch</h4><p>Elasticsearch is a distributed, open source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. Elasticsearch is built on Apache Lucene and was first released in 2010 by Elasticsearch N.V. (now known as Elastic)</p><p>So we can say that; <strong>Elasticsearch </strong>is the preferred full-text search search engine in processes such as content search, data analysis, queries and suggestions, especially due to its performance capabilities, powerful and flexible features. It is developed in Java and is based on Lucene.</p><p>In other words, Elasticsearch is an open source database that is well suited for indexing logs and analytical data.</p><h4>Why is ElasticSearch so popular?</h4><p>Besides being a requirement for almost every application, ElasticSearch solves a number of problems and does it really well:</p><ul><li><strong>Free and open source<br></strong>Basic features are mostly free. If you need security and alert features with Kibana, then it is required to purchase commercial pack.</li><li><strong>RESTful API</strong><br>ElasticSearch has a RESTful API. Query results are returned in JSON format, which means working with results is easy. Querying and adding data through the RESTful API means it is easy to use any programming language to work with ElasticSearch.</li><li><strong>Easy To Query</strong><br>ElasticSearch has a built-in full-text search engine based on Apache Lucene. It is very easy to writing queries on Elastic Search, no required advanced programming knowledge.</li><li><strong>It’s very fast</strong><br>Elasticsearch is fast. Because Elasticsearch engine based on Lucene, it excels at full-text search. <br>Elasticsearch is also a near real-time search platform, meaning the its property of high speed and high availability, it is also a popular tool to save big data today.</li><li><strong>Scalable</strong><br>It is easy to scale. it’s open source and containerized on docker hub and that means it means it’s easy work on docker containers.</li><li><strong>Easy to Install</strong><br>There is docker images on DockerHub for ElasticSearch and Kibana containers. So that means we can easly add these services into our docker-compose files and you are ready to start logging and searching.</li></ul><h4>What is Kibana ?</h4><p>Kibana is an open source data visualization and user interface for Elasticsearch. We can think of Elasticsearch as a database and Kibana as the web user interface that we can use to create graphs, queries, indexes in Elasticsearch. <br>It provides search and visualization capabilities for data indexes in the ElasticSearch. You can query the data on Elasticsearch and create graphics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GA09jxxfLF-MO3lDCQQrqw.png" /></figure><p>Also you can see the image,</p><ul><li>Logstash is collecting logs and transform</li><li>Elasticsearch is searching and analysing logs</li><li>Kibana is visualize ana manage logs</li></ul><h4>Why logging with ElasticSearch and Kibana?</h4><p>In microservices applications, logging is critical to implement in order to identify problems in distributed architecture. ElasticSearch makes any kind of logging easy, accessible and searchable.<br>When using ElasticSearch, It makes logging easily accessible <br>and searchable using a simple query language coupled with Kibana interface.<br>ElasticSearch’s incredible speed and simple query language coupled with Kibana’s interface.</p><h4>What is Serilog?</h4><p>Serilog is a library for ASP.NET that makes logging easy. There are several Sinks available for Serilog, for example, File, SQL and Elasticsearch Sinks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*aZOVjC1H3fiZT3hJeJD8ag.png" /></figure><h3>Adding ElasticSearch and Kibana image into Docker-Compose File for Multi-Container Docker Environment</h3><p>we are going to add ElasticSearch and Kibana image into Docker-Compose File for Multi-Container Docker Environment.</p><p>Implementing Centralized Distributed Logging for Microservices</p><p><strong>Elastic Stack (ELK)</strong> E-elasticsearch, L-logstash (log shipping), K-kibana<br>SeriLog -send logs to ElasticSearch<br>Kibana — query logs — data visualization<br>Enrich Logs</p><p>Before we start coding in .NET Core, it’s important to first spin up the Elasticsearch and Kibana containers. <br>The easiest way to spin up these containers is to updating our docker-compose.yml file.</p><p>Before we start, we should check ElasticSearch and Kibana images from docker hub.</p><p>Docker compose setup ElasticSearch and Kibana</p><p><a href="https://hub.docker.com/_/elasticsearch/">https://hub.docker.com/_/elasticsearch/</a><br><a href="https://hub.docker.com/_/kibana">https://hub.docker.com/_/kibana</a></p><p>Now we can add ElasticSearch and Kibana image into our docker-compose.yml files</p><h4>docker-compose.yml</h4><pre>elasticsearch:<br> image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2</pre><pre>kibana:<br> image: docker.elastic.co/kibana/kibana:7.9.2</pre><pre>volumes:<br> mongo_data:<br> portainer_data:<br> postgres_data:<br> pgadmin_data:<br> elasticsearch-data: — — — — — — → ADDED</pre><p>We have added 2 image, 1 volume</p><h4>docker-compose.override.yml</h4><pre>elasticsearch:<br> container_name: elasticsearch<br> environment:<br> — xpack.monitoring.enabled=true<br> — xpack.watcher.enabled=false<br> — “ES_JAVA_OPTS=-Xms512m -Xmx512m”<br> — discovery.type=single-node<br> ports:<br> — “9200:9200”<br> volumes:<br> — elasticsearch-data:/usr/share/elasticsearch/data</pre><pre>kibana:<br> container_name: kibana<br> environment: <br> — ELASTICSEARCH_URL=<a href="http://localhost:9200/">http://localhost:9200</a><br> depends_on:<br> — elasticsearch<br> ports:<br> — “5601:5601”</pre><p>We have added elasticsearch and kibana<br>We set elasticsearch and kibana configuration as per environment variables.</p><p>Finally, we can create ElasticSearch and Kibana image for Implementing Centralized Distributed Logging for Microservices.</p><h4>Open In Terminal</h4><p>RUN with below command on that location;</p><pre>docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d<br> docker-compose -f docker-compose.yml -f docker-compose.override.yml down</pre><p>This will pull images for elasticsearch and kibana.</p><p>The first time you run the docker-compose command, it will download the images for ElasticSearch and Kibana from the docker registry, <br>so it might take a few minutes depending on your connection speed.</p><p>Once you’ve run the docker-compose up command, check that ElasticSearch and Kibana are up and running.</p><h4>RUN on DOCKER</h4><p>Verify that Elasticsearch is up and running</p><p>Elasticsearch is up and running<br> <a href="http://localhost:9200/">http://localhost:9200</a></p><p>Check the indexes<br> <a href="http://localhost:9200/_aliases">http://localhost:9200/_aliases</a></p><p>So right now there is no any indexes defined in elastic search. I am not going to create indexes or data, because we have already generated logs and logs comes from different microservices, those are will be indexes and data for elasticsearh.</p><p>But its good to know, you can create data with sending POST request to elastic search. For example you can send set of json product data to this url and elastic search will cover data.<br> <a href="http://localhost:9200/products/_bulk">http://localhost:9200/products/_bulk</a><br> <a href="http://localhost:9200/products/_search">http://localhost:9200/products/_search</a><br>But these are not our topic for now we are focusing on distributed logging feature on microservices.</p><p>Verify that Kibana is up and running</p><p>Navigate to <a href="http://localhost:5601/">http://localhost:5601</a> to ensure Kibana is up and running</p><p>Kibana is up and running<br> <a href="http://localhost:5601/">http://localhost:5601</a></p><p>Normally it takes some time to run Kibana, we should wait. But If you get error , you should INCREASE DOCKER RAM.</p><p>DOCKER SETTINGS</p><p>RESOURCES<br> MEMORY <br> INCREASE 2 -&gt; 4 GB</p><p>As you can see that, we have successfully added ElasticSearch and Kibana image into Docker-Compose File for Multi-Container Docker Environment.</p><h3>Install and Configure SeriLog For ElasticSearch and Kibana Sink Integration</h3><p>We are going to Install and Configure SeriLog For ElasticSearch and Kibana Sink Integration.</p><p>Let’s develop Serilog in AspnetRunBasics Shopping Web Application and test it.</p><p>Go to <br> AspnetRunBasics</p><h4>Install Packages</h4><p>Install-Package Serilog.AspNetCore<br> Install-Package Serilog.Enrichers.Environment<br> Install-Package Serilog.Sinks.Elasticsearch</p><p>Check item group project file;</p><pre> &lt;ItemGroup&gt;<br> &lt;PackageReference Include=”Microsoft.VisualStudio.Azure.Containers.Tools.Targets” Version=”1.10.9&quot; /&gt;<br> &lt;PackageReference Include=”Serilog.AspNetCore” Version=”3.4.0&quot; /&gt;<br> &lt;PackageReference Include=”Serilog.Enrichers.Environment” Version=”2.1.3&quot; /&gt;<br> &lt;PackageReference Include=”Serilog.Sinks.Elasticsearch” Version=”8.4.1&quot; /&gt;<br> &lt;/ItemGroup&gt;</pre><p>After the related packages are installed, we organize the logging section in our appsettings.json file according to the serilog section that we will use:</p><h4>Change the Logging Settings on appsettings.json file</h4><p><strong>Existing</strong></p><pre>{<br> “ApiSettings”: {<br> “GatewayAddress”: “<a href="http://localhost:8010/">http://localhost:8010</a>&quot;<br> },<br> “Logging”: {<br> “LogLevel”: {<br> “Default”: “Information”,<br> “Microsoft”: “Warning”,<br> “Microsoft.Hosting.Lifetime”: “Information”, <br> “AspnetRunBasics”: “Debug”<br> }<br> },<br> “AllowedHosts”: “*”<br> }</pre><p><strong>New One :</strong></p><pre>{<br> “ApiSettings”: {<br> “GatewayAddress”: “<a href="http://localhost:8010/">http://localhost:8010</a>&quot;<br> },<br> “Serilog”: {<br> “MinimumLevel”: {<br> “Default”: “Information”,<br> “Override”: {<br> “Microsoft”: “Information”,<br> “System”: “Warning”<br> }<br> }<br> },<br> “ElasticConfiguration”: {<br> “Uri”: “<a href="http://localhost:9200/">http://localhost:9200</a>”<br> },<br> “AllowedHosts”: “*”<br>}</pre><p>We have added the Serilog section and the section where our Elasticsearch endpoint is located.<br>Remove the Logging section in appsettings.json and replace it with the following configuration so that we can tell Serilog what the minimum log level should be, and what url to use for logging to Elasticsearch.</p><h3>Configure SeriLog</h3><p>After that, we will configure logging in <strong>Program.cs</strong> by adding the following details on Main method.</p><p>What we want to do is set up logging before creating the host. This way, if the host does not start, we can log any errors. After that we will add the <strong>ConfigureLogging </strong>and <strong>ElasticsearchSinkOptions </strong>methods to the Program.cs file.</p><p>AspnetRunBasics</p><pre>Program.cs — UseSerilog — — ADDED</pre><pre>public static IHostBuilder CreateHostBuilder(string[] args) =&gt;<br> Host.CreateDefaultBuilder(args) <br> .UseSerilog((context, configuration) =&gt;<br> {<br> configuration<br> .Enrich.FromLogContext()<br> .Enrich.WithMachineName()<br> .WriteTo.Console()<br> .WriteTo.Elasticsearch(<br> new ElasticsearchSinkOptions(new Uri(context.Configuration[“ElasticConfiguration:Uri”]))<br> {<br> IndexFormat = $”applogs-{Assembly.GetExecutingAssembly().GetName().Name.ToLower().Replace(“.”, “-”)}-{context.HostingEnvironment.EnvironmentName?.ToLower().Replace(“.”, “-”)}-logs-{DateTime.UtcNow:yyyy-MM}”,<br> AutoRegisterTemplate = true,<br> NumberOfShards = 2,<br> NumberOfReplicas = 1<br> })<br> .Enrich.WithProperty(“Environment”, context.HostingEnvironment.EnvironmentName)<br> .ReadFrom.Configuration(context.Configuration);<br> })<br> .ConfigureWebHostDefaults(webBuilder =&gt;<br> {<br> webBuilder.UseStartup&lt;Startup&gt;();<br> });</pre><p>We have configured Serilog according to writing Console and Elastic Search. When Configure for the Elastic Search we provide some sink options.</p><p>As you can see that, we have Install and Configured SeriLog For ElasticSearch and Kibana Sink Integration, next video we are going to test it.</p><h3>Test SeriLog For ElasticSearch and Kibana Sink Integration in Shopping Web Microservices</h3><p>We are going to Test SeriLog For ElasticSearch and Kibana Sink Integration in Shopping Web Microservices.</p><p>So far we have finished to development part of configurations, but Kibana will not show any logs at the moment. We need to create an index for Kibana to show your logs. <br>As a standard, I will create a pattern with ‘*’. We can give a specific pattern here, but I found it appropriate to give a general pattern as an example. <br>Let’s continue the log process by showing how to create an example index pattern on Kibana.</p><p>Before we start, let me Verify Docker compose worked :</p><pre>docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d<br></pre><p>Run Application<br> Set a Startup Project and Run<br> AspnetRunBasics</p><p>See logs are colorfully</p><pre>[15:02:50 INF] Now listening on: <a href="http://localhost:5006/">http://localhost:5006</a><br> [15:02:50 INF] Application started. Press Ctrl+C to shut down.<br> [15:02:50 INF] Hosting environment: Development<br> [15:02:50 INF] Content root path: C:\Users\ezozkme\source\repos\mutank\src\WebApps\AspnetRunBasics<br> [15:02:51 INF] Request starting HTTP/1.1 GET <a href="http://localhost:5006/">http://localhost:5006/</a> — -<br> [15:02:52 INF] Executing endpoint ‘/Index’<br> [15:02:52 INF] Route matched with {page = “/Index”}. Executing page /Index<br> [15:02:52 INF] Executing handler method AspnetRunBasics.Pages.IndexModel.OnGetAsync — ModelState is Valid<br> [15:02:57 INF] Executed handler method OnGetAsync, returned result Microsoft.AspNetCore.Mvc.RazorPages.PageResult.<br> [15:02:57 INF] Executed page /Index in 5374.8521ms<br> [15:02:57 INF] Executed endpoint ‘/Index’<br> [15:02:57 INF] Request finished HTTP/1.1 GET <a href="http://localhost:5006/">http://localhost:5006/</a> — — — 200 — text/html;+charset=utf-8 5688.4302ms<br> [15:03:29 INF] Request starting HTTP/1.1 GET <a href="http://localhost:5006/Product">http://localhost:5006/Product</a> — -<br> [15:03:29 INF] Executing endpoint ‘/Product’</pre><p>When we go to the Kibana interface with the localhost: 5601 url on the browser, it will meet us with a screen as follows:</p><p>Elasticsearch is up and running<br> <a href="http://localhost:9200/">http://localhost:9200</a></p><p>Kibana is up and running<br> <a href="http://localhost:5601/">http://localhost:5601</a></p><p><a href="http://localhost:5601/app/home#/">http://localhost:5601/app/home#/</a></p><h3>Create an Index Pattern in Kibana to Show Data</h3><p>Kibana will not show the logs yet. Before we can view the logged data, we must specify a index pattern. <br>In order to do this, click the Explore by myself link on the default Kibana page <br>and then click the Discover link in the navigation section.</p><ul><li>Use Elasticsearch data<br> Connect to your Elasticsearch index</li><li>Index patterns<br> Create Index Pattern</li></ul><p>my index seems here<br> aspnetrunbasics-development-2021–02</p><p>Then, type in an index pattern. It will show the index pattern that was just created. You can type in the entire index, or use wildcards.</p><p>put this <br><strong>aspnetrunbasics-*</strong></p><p>On the next page, select the <a href="http://twitter.com/timestamp">@timestamp</a> field as the time filter field name and click the Create index pattern button.</p><ul><li>Next Step<br> timestamp<br> Create Index Pattern</li></ul><p>See Fields</p><p>You can now view the logs by clicking the Discover link in the navigation pane.</p><ul><li>Go to Main Menu<br> Discover</li></ul><p>See Logs, as you can see that we can see logs on Kibana Dashboard.</p><h4>Change LogInformation — add new customPropery</h4><p>Now I am going to add new log into my application.</p><pre>AspnetRunBasics.Services<br>CatalogService</pre><pre>public async Task&lt;IEnumerable&lt;CatalogModel&gt;&gt; GetCatalog()<br> {<br> _logger.LogInformation(“Getting Catalog Products from url : {url} and custom property: {customPropery}”, _client.BaseAddress, 6);</pre><p>Now that we’ve logged a message, refresh the application on <a href="http://localhost:5000/">http://localhost:5000</a> again. Then search for the log message for customPropery text.</p><ul><li>Run Again<br> See <br><strong> fields.customPropery — 6</strong></li><li>Open Kibana<br>Index Pattern<br> Refresh</li><li>Now you can search by<br> fields.customPropery — 6</li></ul><p><strong>Go to Discover<br> fields.customPropery : 6</strong></p><p><strong>Search</strong></p><p>As you can see that, we have Tested SeriLog For ElasticSearch and Kibana Sink Integration in Shopping Web Microservices.<br> But this is only 1 microservices logs, but we have more that 10 microservices need to centralized log in order to trace log exceptions.</p><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=RESILIENCE"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><p>For the next articles -&gt;</p><ul><li><a href="https://mehmetozkaya.medium.com/microservices-resilience-and-fault-tolerance-with-applying-retry-and-circuit-breaker-patterns-c32e518db990"><strong>2- Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns using Polly</strong></a></li></ul><h3>Step by Step Development w/ Udemy Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*BNqi74a9lU-6U0XBxqscvQ.png" /></figure><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=OCTOBER2021"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79df919997d2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/microservices-observability-with-distributed-logging-using-elasticsearch-and-kibana-79df919997d2">Microservices Observability with Distributed Logging using ElasticSearch and Kibana</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Microservices Observability, Resilience, Monitoring on .Net]]></title>
            <link>https://medium.com/aspnetrun/microservices-observability-resilience-monitoring-on-net-a5dfbdbb0fbd?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/a5dfbdbb0fbd</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[resilience]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[distributed-tracing]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Tue, 20 Apr 2021 12:28:38 GMT</pubDate>
            <atom:updated>2023-04-09T12:25:08.102Z</atom:updated>
            <content:encoded><![CDATA[<h3>Microservices Observability, Resilience, Monitoring on .Net</h3><p><em>Building Cross-Cutting Concerns — Microservices Observability with Distributed Logging, Health Monitoring, Resilient and Fault Tolerance with using Polly.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MobM2Ec4_iFkqi1juYV6nA.png" /></figure><p>In this article we will show <strong>how to perform Microservices Observability, Microservices Resilience and Monitoring</strong> principles on .Net microservices.</p><p>So When you are developing projects in microservices architecture, it is crucial to following Microservices <strong>Observability</strong>, Microservices <strong>Resilience </strong>and <strong>Monitoring </strong>principles.</p><p>So, we will separate our Microservices Cross-Cutting Concerns in 4 main pillars;</p><ul><li>Microservices <strong>Observability </strong>with Distributed Logging</li><li>Microservices <strong>Resilience </strong>and <strong>Fault Tolerance</strong> with applying <strong>Retry </strong>and <strong>Circuit-Breaker patterns</strong> using <strong>Polly</strong></li><li>Microservices Monitoring with <strong>Health Checks </strong>using <strong>WatchDog</strong></li><li>Microservices Tracing with <strong>OpenTelemetry </strong>using <strong>Zipkin</strong></li></ul><p>So we are going to follow this 4 main pillars and develop our microservices reference application with using latest implementation and best practices on cloud-native microservices architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DYkvDaztKZtyNW1IH4_ZjQ.png" /></figure><p>See the our big picture what we are going to develop Microservices Architecture and Step by Step Implementation together.<br>We have already developed this microservices reference application in the microservices articles.</p><h3>Step by Step Development w/ Udemy Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*BNqi74a9lU-6U0XBxqscvQ.png" /></figure><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=APRI23"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-aspnetcore-microservices"><strong>Get the Source Code from AspnetRun Microservices Github</strong> </a>— Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.</p><h3>Microservices Cross-Cutting Concerns</h3><p>So with this article, we will extend this microservices reference application with Cross-Cutting Concerns for provide microservices resilience.</p><p>We are going to cover Cross-Cutting Concerns in<strong> 4 main parts</strong>;</p><h4><strong>1-Microservices Observability with Distributed Logging</strong></h4><p>This applying Elastic Stack which includes elasticsearh + logstach + kibana and SeriLog nuget package for .net microservices.<br>We will docker-compose Kibana image from docker hub and feed kibana with elastic stack.</p><h4><strong>2-Microservices Resilience and Fault Tolerance using Polly</strong></h4><p>This will apply retry and circuit-breaker patterns on microservices communication with creating polly policies.</p><h4><strong>3-Microservices Health Monitoring with using WatchDog</strong></h4><p>This will be the aspnet health check implementation with custom hc methods<br> which inludes database availabilities — for example in basket microservices, we will add sub-hc contitons for connecting Redis, and RabbitMQ.</p><h4><strong>4-Microservices Distributed Tracing with OpenTelemetry using Zipkin</strong></h4><p>This will be the implementation of OpenTelemetry with Zipkin.</p><p>By the end of this article, you’ll learn how to design and developing Microservices Cross-Cutting Concerns — Microservices Observability with Distributed Logging, Health Monitoring, Resilient and Fault Tolerance with using Polly on .Net Microservices.</p><h3>Background</h3><p>This is the introduction of the series. This will be the series of articles. You can follow the series with below links.</p><ul><li><a href="https://mehmetozkaya.medium.com/microservices-observability-resilience-monitoring-on-net-a5dfbdbb0fbd"><strong>0- Microservices Observability, Resilience, Monitoring on .Net</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-observability-with-distributed-logging-using-elasticsearch-and-kibana-79df919997d2"><strong>1- Microservices Observability with Distributed Logging using ElasticSearch and Kibana</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-resilience-and-fault-tolerance-with-applying-retry-and-circuit-breaker-patterns-c32e518db990"><strong>2- Microservices Resilience and Fault Tolerance with applying Retry and Circuit-Breaker patterns using Polly</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-monitoring-with-health-checks-using-watchdog-6b16fdae0349"><strong>3- Microservices Monitoring with Health Checks using WatchDog</strong></a></li></ul><p>Also you can find whole microservices series on below article;</p><p><a href="https://medium.com/aspnetrun/microservices-architecture-on-net-3b4865eea03f"><strong>Check for the previous article which explained overall microservice architecture of this repository.</strong></a></p><p>We will focus on microservices cross-cutting concerns on these article series.</p><h3>Prerequisites</h3><ul><li>Install the .NET Core 5 or above SDK</li><li>Install Visual Studio 2019 v16.x or above</li><li>Docker Desktop</li></ul><h3>Microservices = Distributed Systems</h3><p>Microservice architecture have become the new model for building modern cloud-native applications. And microservices-based applications are distributed systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oE2FY2uHGN4qbFwh3TNxog.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/handle-partial-failure">https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/handle-partial-failure</a></figcaption></figure><p>While architecting distributed microservices-based applications, it get lots of benefits like makes easy to scale and manage services, but as the same time, it is increasing interactions between those services have created a new set of problems.</p><h3>Distributed Dependencies</h3><p>So we should assume that failures will happen, and we should dealing with unexpected failures, especially in a multiple Distributed Dependencies systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PsXQLt6IlRiDpVYmsArUrQ.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/handle-partial-failure">https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/handle-partial-failure</a></figcaption></figure><p>For example in case of network or container failures, microservices must have a strategy to retry requests again.</p><h3>Dealing with Failures</h3><p>But What happens when the machine where the microservice is running fails ?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PrOy0jg7EPqYt5Fou4uzIQ.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/handle-partial-failure">https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/handle-partial-failure</a></figcaption></figure><p>For instance, a single microservice can fail or might not be available to respond for a short time. Since clients and services are separate processes, a service might not be able to respond in a timely way to a client’s request. <br>The service might be overloaded and responding very slowly to requests or might simply not be accessible for a short time because of network issues.</p><blockquote>How do you handle the complexity that comes with distributed cloud-native microservices?</blockquote><h3>Microservices Resilience</h3><p>Microservice should design for resiliency. A microservice needs to be resilient to failures and must accept partial failures. We should Design microservices to be resilient for these partial failures. microservices should ability to recover from failures and continue to function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qwb7-X9__R3X7Mp859boNQ.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/application-resiliency-patterns">https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/application-resiliency-patterns</a></figcaption></figure><p>We should accepting failures and responding to them without any downtime or data loss. The main purpose of resiliency microservices is to return the application to a fully functioning state after a failure.</p><p>So when we are architecting distributed cloud applications, we should assume that failures will happen and design our microservices for resiliency. We accept that Microservices are going to fail at some point, thats why we need to learn embracing failures.</p><h3>Microservices Observability</h3><p>Also we should have a strategy for monitoring and managing the complex dependencies on microservices. That means we need to implement microservices observability with using distributed logging features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-j8W5EW-YONtl9BSrzJaAA.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/observability-patterns">https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/observability-patterns</a></figcaption></figure><p>Microservices Observability gives us greater operational insight and leading to understand incidents on our microservices architecture.</p><h3>Microservices Monitoring</h3><p>Also we should have microservices monitoring with health monitoring, Health monitoring is critical to multiple aspects of operating microservices. By this way, we can understand for a particular microservice is alive and ready to accommodate requests. We can also provide health information to our orchestrator’s cluster, so that the cluster can act accordingly. For example Kubernetes has Liveness and Readiness probes that we can address health check urls.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7LIdw0pW5iKG_mAKNaud6A.png" /><figcaption><a href="https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/observability-patterns">https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/observability-patterns</a></figcaption></figure><p>That make a good health reporting which customized for our microservices like adding sub health checks for underlying database connection, and by this way we can detect and fix issues for our running application much more easily.</p><p>So, we will separate our Microservices Cross-Cutting Concerns in 4 main pillars;</p><ul><li>Microservices <strong>Observability </strong>with Distributed Logging</li><li>Microservices <strong>Resilience </strong>and <strong>Fault Tolerance</strong> with applying <strong>Retry </strong>and <strong>Circuit-Breaker patterns</strong> using <strong>Polly</strong></li><li>Microservices Monitoring with <strong>Health Checks </strong>using <strong>WatchDog</strong></li><li>Microservices Tracing with <strong>OpenTelemetry </strong>using <strong>Zipkin</strong></li></ul><p>We will use all steps with applying cross-cutting concerns for the next articles -&gt;</p><ul><li><a href="https://mehmetozkaya.medium.com/microservices-observability-with-distributed-logging-using-elasticsearch-and-kibana-79df919997d2"><strong>1- Microservices Observability with Distributed Logging using ElasticSearch and Kibana</strong></a></li></ul><h3>Step by Step Development w/ Udemy Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*BNqi74a9lU-6U0XBxqscvQ.png" /></figure><p><a href="https://www.udemy.com/course/microservices-observability-resilience-monitoring-on-net/?couponCode=APRI23"><strong>Get Udemy Course with discounted — Microservices Observability, Resilience, Monitoring on .Net.</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a5dfbdbb0fbd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/microservices-observability-resilience-monitoring-on-net-a5dfbdbb0fbd">Microservices Observability, Resilience, Monitoring on .Net</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automate Deployments with CI/CD pipelines on Azure Devops]]></title>
            <link>https://medium.com/aspnetrun/automate-deployments-with-ci-cd-pipelines-on-azure-devops-13a83d3dd67a?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/13a83d3dd67a</guid>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[azure-kubernetes-service]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[azure-pipelines]]></category>
            <category><![CDATA[azure-devops]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Fri, 02 Apr 2021 06:16:46 GMT</pubDate>
            <atom:updated>2023-04-09T12:23:21.151Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article, we are going to Automate our Microservices Deployments with CI/CD pipelines on Azure Devops.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*2WHIbwc-NX9uATWKmEaJGA.png" /></figure><p>See the overall picture. As you can see that, we have</p><ul><li>Created docker images,</li><li>Compose docker containers and tested them,</li><li>Deploy these docker container images on local kubernetes clusters,</li><li>Push our image to ACR,</li><li>Shifting deployment to the cloud Azure kubernetes services (AKS),</li><li>Update microservices with zero-downtime deployments.</li></ul><p>That means we have finished to this box.</p><p>So in this article, we are focusing on automation deployments with creating <strong>CI/CD pipelines</strong> on <strong>Azure Devops </strong>tool.<br>Before we start this actions, we should understand what is <strong>Azure Devops</strong> and how to use <strong>pipelines </strong>on Azure Devops.</p><h3>Background</h3><p>This is the last article of the series. You can follow the series with below links.</p><ul><li><a href="https://mehmetozkaya.medium.com/deploying-net-microservices-to-azure-kubernetes-services-aks-and-automating-with-azure-devops-c50bdd51b702"><strong>0- Deploying .Net Microservices</strong></a></li><li><a href="https://mehmetozkaya.medium.com/preparing-multi-container-microservices-applications-for-deployment-793d60f48d31"><strong>1- Preparing Multi-Container Microservices Applications for Deployment</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploying-microservices-on-kubernetes-35296d369fdb"><strong>2- Deploying Microservices on Kubernetes</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploy-microservices-into-cloud-azure-kubernetes-service-aks-with-using-azure-container-registry-b661698610b1"><strong>3- Deploy Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry (ACR)</strong></a></li><li><a href="https://mehmetozkaya.medium.com/automate-deployments-with-ci-cd-pipelines-on-azure-devops-13a83d3dd67a"><strong>4- Automate Deployments with CI/CD pipelines on Azure Devops</strong></a></li></ul><h3>Step by Step Development w/ Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/0*n0tIaZeejbncDdSJ.png" /></figure><p><a href="https://www.udemy.com/course/deploying-net-microservices-with-k8s-aks-and-azure-devops/?couponCode=APRI23"><strong>I have just published a new course — Deploying .Net Microservices with K8s, AKS and Azure DevOps.</strong></a></p><p>In this course, we’re going to learn how to <strong>Deploying .Net Microservices</strong> into <strong>Kubernetes</strong>, and moving deployments to the cloud <strong>Azure kubernetes services (AKS)</strong> with using <strong>Azure Container Registry(ACR)</strong> and last section is we will learn how to Automating Deployments with <strong>CI/CD pipeline of Azure DevOps</strong> and GitHub.</p><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-devops"><strong>Get the Source Code from AspnetRun Microservices Github</strong></a> — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.</p><h3>What is Azure Devops ?</h3><p>Azure DevOps provides developer services for support teams to plan work, collaborate on code development, and build and deploy applications. <br>Developers can work in the cloud using <strong>Azure DevOps Services</strong>.<br><strong>Azure DevOps</strong> provides integrated features that you can access through your web browser or IDE client. We will follow with using web browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T6DUwBR_60Eqwiizk_DaUg.png" /></figure><p>You can use one or more of the following services based on your business needs: <br>We are going to use Azure pipelines when automate deployment of microservices, but let me explain all services.</p><p><strong>Azure Boards</strong> delivers a suite of Agile tools to support planning and tracking work, code defects, and issues using Kanban and Scrum methods<br><strong>Azure Repos </strong>provides Git repositories for source control of your code<br><strong>Azure Pipelines </strong>provides build and release services to support continuous integration and continuous delivery of your apps<br><strong>Azure Test Plans </strong>provides several tools to test your apps, including manual/exploratory testing and continuous testing<br><strong>Azure Artifacts </strong>allows teams to share packages such as Maven, npm, NuGet and more from public and private sources and integrate package sharing into your CI/CD pipelines</p><p>Also Azure DevOps supports adding extensions and integrating with other popular services, <br> such as: Campfire, Slack, Trello, UserVoice, and more, and developing your own custom extensions.</p><h3>What is Azure Pipelines ?</h3><p>Azure Pipeline is a <strong>Continous Integration and Continous Deployment (CI/CD) </strong>tool provided by Azure under Azure DevOps. With Azure Pipelines you can build, test, and deploy your applications across all languages. <br>Azure Pipelines can pull your source code from various repositories and execute a build according to your specifications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cDePDy4qS8wV514uvJGN7g.png" /></figure><p>Let me give more detail about what is Continous Integration and Continous Deployment (CI/CD) flows;</p><p>This flow provide to Automate tests, builds, and delivery.</p><p><strong>Continuous integration</strong> automates tests and builds for your project. CI helps to catch bugs or issues early in the development cycle, when they’re easier and faster to fix. Items known as artifacts are produced from CI systems. They’re used by the continuous delivery release pipelines to drive automatic deployments.</p><p><strong>Continuous delivery</strong> automatically deploys and tests code in multiple stages to help drive quality. Continuous integration systems produce deployable artifacts, which include infrastructure and apps. Automated release pipelines consume these artifacts to release new versions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gRaaQ-6Jo22DrK5DPM2e2Q.png" /></figure><p>We can provide this flows with creating a pipeline on azure pipelines. <br>We will define our pipeline in a YAML file called <strong>azure-pipelines.yml</strong> with for the one application. But of course we will create pipelines as per microservices ci/cd flows.</p><p><strong>Pipelines </strong>are stored with our code, so we will push and commit pipeline yaml files like our code structure on github. By this way, we get validation of your changes through code reviews in pull requests.</p><p>Any change to the build process might cause a break or result in an unexpected outcome. Because the change is in version control with the rest of our codebase, we can more easily identify the issue.</p><h3>Deploy to AKS through Azure CI/CD Pipelines</h3><p>Our main target is automate deployments of Shopping Microservices into AKS with using <strong>Azure CI/CD Pipelines</strong>. So before we start let’s remember what steps we have done when manually deploying to AKS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gN4Te3KEwmHqi7VGsOBE5A.png" /></figure><p>As you can see that, we have created docker images for our microservices. Also we compose docker containers and tested them. Deploy these docker container images on local Kubernetes clusters, push our image to <strong>ACR</strong>, shifting deployment to the cloud <strong>Azure Kubernetes Services (AKS)</strong>, update microservices with <strong>zero-downtime</strong> deployments.</p><p>So now it is time to check that What steps we are going to do when automate deployments into <strong>AKS </strong>with using<strong> Azure CI/CD Pipelines.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rtyh_sEdhxcKBBP3swuf1A.png" /></figure><p>There are several steps we need to go through to set up Azure Pipelines.</p><ul><li>We will develop our project on visual studio.</li><li>Storing our codes on Github Repository.</li><li>Azure pipelines connect and triggering with Git repo changes.</li><li>After that CI/CD pipeline will start according to azure pipeline yaml definitions.</li><li>You can see the building and pushing images to ACR and deploy to AKS.</li></ul><p>Here you can find example scenario that explains automation deployments into AKS with using Azure CI/CD Pipelines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RScqJLabL17D2DjSo3kwVQ.png" /></figure><p>Whenever we pushed any new code or update any existing code on Github, <br>Azure Pipeline will pull the changes and start the automation.</p><p>First, it builds a new version of docker image from docker file which means creating a container image of our application,<br>After this step, pipeline will hit docker push command which will push to your docker image to an azure container registry with new version tag name of image, <br>After that, Azure k8s cluster will pull the image of the application and it creates and run a container from this new image inside of the k8s pods,<br>AKS will perform rollout deployment that pods replace one by one with the new version of image. So end user never effect to these zero-downtime deployments.<br>By this way, we can deploy hundreds of deployment without affecting the system in a day. Also same as the rollback deployments with no effect.</p><p>We explained the basics. Now let’s get into action.</p><h4>Sign up for Azure Pipelines</h4><p>In order to work with Azure Pipelines we need to create an organization first and then a project inside of the organization.<br>You can Sign up for an Azure DevOps organization and Azure Pipelines to begin managing CI/CD to deploy your code with pipelines.</p><h4>Create Our first pipeline with Azure Pipelines</h4><p>In your project, navigate to the Pipelines page. Then choose the action to create a new pipeline.</p><p>Give Repo of GitHub — swnzen<br>We will Walk through the steps of the wizard by first selecting GitHub as the location of your source code.<br>It will redirected to GitHub to sign in. If so, enter your GitHub credentials.<br>When the list of repositories appears, select our Shopping microservices repository.</p><h4>Configure your pipeline</h4><p>Select Below template<br><strong> Docker<br> Build and push an image to Azure Container Registry</strong></p><p>Basically this task cover our first part of deployment process.<br>We will start with build task, which is docker build and push.<br>also there is Kubernetes deployment which contains Docker Build and Push template, we will see it later when explain deployment stages.</p><p>Generating yaml file</p><p>See generated yaml file; Examine;</p><pre># Docker<br># Build and push an image to Azure Container Registry<br># <a href="https://docs.microsoft.com/azure/devops/pipelines/languages/docker">https://docs.microsoft.com/azure/devops/pipelines/languages/docker</a></pre><pre> — — This is for when start our pipeline definition, that means it starts any changes on main branch of our github repo.<br>trigger: <br>- main</pre><pre> — — This is default definition of resources<br>resources:<br>- repo: self</pre><pre> — — Variables are like normal development. You can think this yaml file as a normal code window, we can define variables on top of page and use these variables into task objects.<br>variables:</pre><pre> — — This docker connection is created when we added to task. Let me show you settings — connected services = as you can see that azure pipelines save our connections in here and we use these connections during the pipelines. <br> — — Also Azure pipelines created Environment for us, and it will said that these connections related with this environment and we can track deployments on environment seciton. This is very good feauture that you can split your environemnt pipelines like Dev, Staging, production environment.</pre><pre># Container registry service connection established during pipeline creation<br> dockerRegistryServiceConnection: ‘b0f7d460–9890–4230–842e-4039178a0031’</pre><pre> — — These ares ACR required parameter in order to push docker images.<br> imageRepository: ‘shoppingapi’<br> containerRegistry: ‘swnzenacr.azurecr.io’</pre><pre> — — This is docker file path which using for building docker image <br> dockerfilePath: ‘$(Build.SourcesDirectory)/swnzen/Shopping.API/Dockerfile’</pre><pre> — — This is for unique tag name. Build is predefined class that we can BuildId or BuildNumber in our pipelines.<br> tag: ‘$(Build.BuildId)’<br> <br> — — Azure pipeline vm machine image, linux based. <br> # Agent VM image name<br> vmImageName: ‘ubuntu-latest’</pre><pre> — — Starting Stages, Stages mostly seperating Build and Deploy stages. Into Stages we have jobs, Into jobs we have tasks. It is a tree structure Stage -&gt; Job -&gt; Task<br>stages:<br>- stage: Build<br> displayName: Build and push stage<br> jobs: <br> — job: Build<br> displayName: Build<br> pool:<br> vmImage: $(vmImageName)<br> steps:</pre><pre> — — This is prebuild Azure pipeline task that provide to Build and push an image to container registry. We can examine task and input parameters from official documentation.</pre><pre>- task: Docker@2<br> displayName: Build and push an image to container registry<br> inputs:</pre><pre> — — As you can see that in input part, we used parameters which indicated to ACR and commadn is buildAndPush.</pre><pre>command: buildAndPush<br> repository: $(imageRepository)<br> dockerfile: $(dockerfilePath)<br> containerRegistry: $(dockerRegistryServiceConnection)<br> tags: |<br> $(tag)</pre><p>Seems good. <br>Save and Run. See portal watch build stages.</p><h4>Create Pipeline for Continues Delivery with Deploy to AKS Task</h4><p>Create new pipeline</p><ul><li>Give Repo of Github<br> swnzen</li><li>Configure your pipeline</li><li>Select Below template<br><strong> Deploy to Azure Kubernetes Service<br> Build and push image to Azure Container Registry; Deploy to Azure Kubernetes Service</strong></li></ul><p>This time we select to “Deploy to Azure Kubernetes Service” task, this covers both Continues Integration and Continues Deployment stages. <br>It means that pipeline has Build and deployment stage.</p><ul><li>Configure task<br> Login the Azure<br> Select Cluster, Namespace, Container Registry and so on..</li><li>Cluster — myAksCluster<br> Namespace — default<br> Container Registry — swnzenacr<br> ImageName — shoppingapi<br> ServicePort — 80 — see from shoppingapi.yaml kube file.<br>Ok</li></ul><p>After that it is generating yaml file.</p><h4>Review your pipeline YAML</h4><p>Examine yaml file;</p><pre># Deploy to Azure Kubernetes Service<br># Build and push image to Azure Container Registry; Deploy to Azure Kubernetes Service<br># <a href="https://docs.microsoft.com/azure/devops/pipelines/languages/docker">https://docs.microsoft.com/azure/devops/pipelines/languages/docker</a></pre><pre>trigger:<br>- main</pre><pre>resources:<br>- repo: self</pre><pre>variables:</pre><pre># Container registry service connection established during pipeline creation<br> dockerRegistryServiceConnection: ‘040b72f3–53e8–478f-89a2-b6c75bb009a7’<br> imageRepository: ‘shoppingapi’<br> containerRegistry: ‘swnzenacr.azurecr.io’<br> dockerfilePath: ‘**/Dockerfile’<br> tag: ‘$(Build.BuildId)’</pre><p>Above are already explained, imagePullSecret for pulling images from ACR, as your remember we have also created secret, it is mandatory for pulling images ACR from AKS.</p><pre># Agent VM image name<br> vmImageName: ‘ubuntu-latest’</pre><pre> — — Build Stage is same, but we need to add buildcontext again.</pre><pre>stages:<br>- stage: Build<br> displayName: Build stage<br> jobs: <br> — job: Build<br> displayName: Build<br> pool:<br> vmImage: $(vmImageName)<br> steps:<br> — task: Docker@2<br> displayName: Build and push an image to container registry<br> inputs:<br> command: buildAndPush<br> repository: $(imageRepository)<br> dockerfile: $(dockerfilePath)<br> containerRegistry: $(dockerRegistryServiceConnection)<br> tags: |<br> $(tag)<br> </pre><p>We seen newly, upload and artifacts. This is very important. Because this place is connection point of <strong>Build and Deploy stages.</strong> After build the docker images we upload some additional files in order to use from Deployment stage. <br>You can think about these task running different machines on cloud and once build finished it upload items into one place <br> that deploy task read and use these files when deploying application.<br>We will also can see and download these uploaded artifacts on azure pipelines, we will see details later.</p><pre>upload: manifests<br> artifact: manifests</pre><pre>- stage: Deploy<br> displayName: Deploy stage<br> dependsOn: Build</pre><p>This is deployment job, it is specific for deployments. <br>It includes deployment strategy is <strong>runOnce </strong>but we can modify later blue-green, canary and rollout deployments.<br>Basically it creates image pull secret for pulling images to ACR, after that running manifest files into AKS cluster.<br>As you can see that these manifests looking for manifest folder which uploaded in build stage and looking deployment and service yaml file for k8s deployment.</p><pre>jobs:<br> — deployment: Deploy<br> displayName: Deploy<br> pool:<br> vmImage: $(vmImageName)<br> environment: ‘mehmetozkayaswnzen.default’<br> strategy:<br> runOnce:<br> deploy:<br> steps:<br> — task: KubernetesManifest@0<br> displayName: Create imagePullSecret<br> inputs:<br> action: createSecret<br> secretName: $(imagePullSecret)<br> dockerRegistryEndpoint: $(dockerRegistryServiceConnection)<br> <br> — task: KubernetesManifest@0<br> displayName: Deploy to Kubernetes cluster<br> inputs:<br> action: deploy<br> manifests: |<br> $(Pipeline.Workspace)/manifests/deployment.yml<br> $(Pipeline.Workspace)/manifests/service.yml<br> imagePullSecrets: |<br> $(imagePullSecret)<br> containers: |<br> $(containerRegistry)/$(imageRepository):$(tag)</pre><p>BTW, we prepare this task for only 1 microservice which is Shopping.API, but we have more microservices, we will cover all of them in the next section.<br>We should separate pipelines as per microservices.</p><h4>Running Pipeline for Continues Delivery with Deploy to AKS</h4><p>If you click to “Save and Run” you saw that its trying to add new items.</p><p>Files to be added to your repository (3)</p><p>azure-pipelines<br>manifest/deployment.yaml files</p><p>Open pipeline see Runs.<br> <a href="https://dev.azure.com/ezozkme/swnzen/_build?definitionId=4">https://dev.azure.com/ezozkme/swnzen/_build?definitionId=4</a></p><p>See Build stages<br>See Artifacts<br>See Deployment logs<br> — You can see that Build and Deployment Succeed !!</p><p>Check kubectl pods:</p><pre>C:\Users\ezozkme&gt;kubectl get pod<br>NAME READY STATUS RESTARTS AGE<br>mongo-deployment-754d654ff7–8n559 1/1 Running 0 2d4h<br>shoppingapi-deployment-97f75c5f8-chvh5 1/1 Running 0 24s<br>shoppingapi-deployment-97f75c5f8-n6sbp 1/1 Running 0 34s<br>shoppingclient-deployment-6bbd995854–5c5tq 1/1 Running 0 45h<br>shoppingclient-deployment-6bbd995854–77jnm 1/1 Running 0 45h<br>shoppingclient-deployment-6bbd995854-gnttk 1/1 Running 0 45h</pre><p>Check details of image pulling log.</p><pre>kubectl describe pod shoppingapi-deployment-97f75c5f8-chvh5</pre><pre>Events:<br> Type Reason Age From Message<br> — — — — — — — — — — — — -<br> Normal Scheduled 38s default-scheduler Successfully assigned default/shoppingapi-deployment-97f75c5f8-chvh5 to aks-nodepool1–30369730-vmss000000<br> Normal Pulled 37s kubelet, aks-nodepool1–30369730-vmss000000 Container image “swnzenacr.azurecr.io/shoppingapi:24” already present on machine<br> Normal Created 37s kubelet, aks-nodepool1–30369730-vmss000000 Created container shoppingapi<br> Normal Started 37s kubelet, aks-nodepool1–30369730-vmss000000 Started container shoppingapi</pre><p>You saw that -&gt; Container image “swnzenacr.azurecr.io/shoppingapi:24” already present on machine.</p><p>As you can see that, we have successfully build and deploy our shopping.api microservices with full automation, with writing our pipeline yaml file.<br>But we developed multi-container microservices application, so we should have separate pipelines for microservices. We should create a new pipeline for Shopping.Client application. You can follow the same steps for Shopping.Client application in order to Manage Pipelines for <strong>Multi-Container Microservices with CI/CD flows in Azure Pipelines.</strong></p><h3>References</h3><p><a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops">https://docs.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops</a><br><a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/pipelines-get-started?view=azure-devops">https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/pipelines-get-started?view=azure-devops</a><br><a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/create-first-pipeline?view=azure-devops&amp;tabs=java%2Ctfs-2018-2%2Cbrowser">https://docs.microsoft.com/en-us/azure/devops/pipelines/create-first-pipeline?view=azure-devops&amp;tabs=java%2Ctfs-2018-2%2Cbrowser</a> <br><a href="https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/containers/aks-microservices/aks-microservices">https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/containers/aks-microservices/aks-microservices</a><br><a href="https://docs.microsoft.com/en-us/azure/architecture/solution-ideas/articles/secure-devops-for-kubernetes">https://docs.microsoft.com/en-us/azure/architecture/solution-ideas/articles/secure-devops-for-kubernetes</a><br><a href="https://docs.microsoft.com/en-us/azure/architecture/solution-ideas/articles/microservices-with-aks">https://docs.microsoft.com/en-us/azure/architecture/solution-ideas/articles/microservices-with-aks</a><br><a href="https://docs.microsoft.com/en-us/azure/architecture/solution-ideas/articles/microservices-with-aks">https://docs.microsoft.com/en-us/azure/architecture/solution-ideas/articles/microservices-with-aks</a> <br><a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/ecosystems/kubernetes/aks-template?view=azure-devops">https://docs.microsoft.com/en-us/azure/devops/pipelines/ecosystems/kubernetes/aks-template?view=azure-devops</a><br><a href="https://medium.com/faun/application-deployment-with-azure-kubernetes-service-and-azure-pipelines-a0bf43916746">https://medium.com/faun/application-deployment-with-azure-kubernetes-service-and-azure-pipelines-a0bf43916746</a><br><a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/kubernetes?view=azure-devops#imagepullsecret">https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/kubernetes?view=azure-devops#imagepullsecret</a><br><a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/create-first-pipeline?view=azure-devops&amp;tabs=java%2Ctfs-2018-2%2Cbrowser">https://docs.microsoft.com/en-us/azure/devops/pipelines/create-first-pipeline?view=azure-devops&amp;tabs=java%2Ctfs-2018-2%2Cbrowser</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13a83d3dd67a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/automate-deployments-with-ci-cd-pipelines-on-azure-devops-13a83d3dd67a">Automate Deployments with CI/CD pipelines on Azure Devops</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry…]]></title>
            <link>https://medium.com/aspnetrun/deploy-microservices-into-cloud-azure-kubernetes-service-aks-with-using-azure-container-registry-b661698610b1?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/b661698610b1</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[azure-container-registry]]></category>
            <category><![CDATA[azure-kubernetes-service]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Fri, 02 Apr 2021 06:16:30 GMT</pubDate>
            <atom:updated>2021-10-02T11:21:46.953Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deploy Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry (ACR)</h3><p>In this article, we are going to Deploy Shopping Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry (ACR).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*2WHIbwc-NX9uATWKmEaJGA.png" /></figure><p>See the overall picture. As you can see that, we have finished to development and deployments of our microservices on local environment also test it. <br>Both Docker and Kubernetes deployments were in our local environment.<br>So now its time to move Kubernetes to the cloud one which is <strong>Azure Kubernetes Service (AKS).</strong></p><p>But before that we will talk about what is <strong>Azure Container Registry (ACR)</strong> and <strong>Azure Kubernetes Service (AKS)</strong> and why we are using them. Let’s get some information about <strong>ACR </strong>and <strong>AKS</strong>.</p><h3>Background</h3><p>This is the third article of the series. You can follow the series with below links.</p><ul><li><a href="https://mehmetozkaya.medium.com/deploying-net-microservices-to-azure-kubernetes-services-aks-and-automating-with-azure-devops-c50bdd51b702"><strong>0- Deploying .Net Microservices</strong></a></li><li><a href="https://mehmetozkaya.medium.com/preparing-multi-container-microservices-applications-for-deployment-793d60f48d31"><strong>1- Preparing Multi-Container Microservices Applications for Deployment</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploying-microservices-on-kubernetes-35296d369fdb"><strong>2- Deploying Microservices on Kubernetes</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploy-microservices-into-cloud-azure-kubernetes-service-aks-with-using-azure-container-registry-b661698610b1"><strong>3- Deploy Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry (ACR)</strong></a></li><li><a href="https://mehmetozkaya.medium.com/automate-deployments-with-ci-cd-pipelines-on-azure-devops-13a83d3dd67a"><strong>4- Automate Deployments with CI/CD pipelines on Azure Devops</strong></a></li></ul><h3>Step by Step Development w/ Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/0*mFupcIszE0FDiYnD.png" /></figure><p><a href="https://www.udemy.com/course/deploying-net-microservices-with-k8s-aks-and-azure-devops/?couponCode=OCTOBER2021"><strong>I have just published a new course — Deploying .Net Microservices with K8s, AKS and Azure DevOps.</strong></a></p><p>In this course, we’re going to learn how to <strong>Deploying .Net Microservices</strong> into <strong>Kubernetes</strong>, and moving deployments to the cloud <strong>Azure kubernetes services (AKS)</strong> with using <strong>Azure Container Registry(ACR)</strong> and last section is we will learn how to Automating Deployments with <strong>CI/CD pipeline of Azure DevOps</strong> and GitHub.</p><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-devops"><strong>Get the Source Code from AspnetRun Microservices Github</strong></a> — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.</p><h3>Azure Container Registry (ACR)</h3><p>Azure Container Registry is a managed, private Docker registry service based on the open-source Docker Registry. Create and maintain Azure container registries to store and manage your private Docker container images and related artifacts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BtYb1TAK1Wz4Sr62mq_HtQ.png" /></figure><p>If you look at the big picture, we have already finished our local development and dockerize all images. <br>Also we push the docker hub, but this time we will start using ACR for pushing our docker images from dockerhub to ACR private registry.<br>And <strong>ACR </strong>has organic communication with <strong>AKS</strong>, so that’s why we move our images ACR and when we deploy to AKS, Our main target is <strong>deploy our microservices on Cloud kubernetes which is AKS. </strong>AKS will get the images from ACR effectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ggU9ElC2z946YOfQOcaoUw.png" /><figcaption>Example use of ACR Devops Pipeline</figcaption></figure><p>You can use Azure container registries with your existing container development and deployment pipelines, or use <strong>Azure Container Registry</strong> Tasks to build container images in Azure. Build on demand, or fully automate builds with triggers such as source code commits and base image updates.</p><p>In the image above, you can find an example usage of ACR in a Devops pipeline, we will see this topic in the next article.</p><h3>Azure Kubernetes Service (AKS)</h3><p>Azure Kubernetes Service (AKS) makes it simple to deploy a managed Kubernetes cluster in Azure. AKS reduces the complexity and operational overhead of managing Kubernetes by offloading much of that responsibility to Azure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/753/1*wxXW9g3tTFsO9R4pgH0v1w.png" /></figure><p>As a managed Kubernetes service, <strong>Azure </strong>handles critical tasks like health monitoring and maintenance for you. The Kubernetes masters are <strong>managed by Azure</strong>. You only manage and maintain the agent nodes. As a managed Kubernetes service, AKS is free plan also I am using free plan for all azure operations.</p><p>You can create an <strong>AKS cluster</strong> in the Azure portal, with the<strong> Azure CLI</strong>, or template driven deployment options such as Resource Manager templates and Terraform. We will use the Azure CLI.<br>When you deploy an AKS cluster, the Kubernetes master and all nodes are deployed and configured for you. Additional features such as advanced networking, Azure Active Directory integration, and monitoring can also be configured during the deployment process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*Xd931N2z-r3Is7tLe9BZHA.png" /><figcaption>Shifting Local to Cloud</figcaption></figure><p>If you look at the image above, In the left side, what we have done so far, we dockerize our images and deploy on local Kubernetes. In the right side is our target cloud deployment items which&#39;s are ACR and AKS.<br>We will push image to ACR, and deploy our current Kubernetes configurations into the AKS with pull images from ACR.</p><h3>Steps to the AKS Deployment</h3><p>These are the main steps what we are going to follow.</p><p>In the first steps we have already done for dockerize our images and deploy on local Kubernetes.<br>So now our target cloud deployment items whichs are ACR and AKS.<br>We will push image to ACR, and deploy our current kubernetes configurations into the AKS with pull images from ACR.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gN4Te3KEwmHqi7VGsOBE5A.png" /><figcaption>Direct Deployment to K8s cluster in ACR</figcaption></figure><p>If we expand the details of these steps, we can see this image.<br>As you can see that there is several steps to deploy AKS cluster,<br>1- Build your docker images <br>2- Tag these images according to ACR login name<br>3- Create your ACR in to Azure subscription and protect with admin user and get the loginname , username and password<br>4- Connect to ACR and push your images to ACR<br>5- Create your AKS service into Azure subscription, it will take some time.<br>6- Build communication with AKS and ACR<br>7- Update your k8s manifest files as per AKS deployment, for example change image names docker hub to ACR address.<br>8- Run your k8s config files into AKS cluster.<br>9- Create load balancer service and test your application from cloud AKS deployment.</p><p>So as you can see that we have lots of steps that we have to do, but no worries, we will do all steps with together and will verify that our application is running on AKS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q-MR7aGVscnijz_P4MrmYg.png" /><figcaption>Advance Senario</figcaption></figure><p>In this image above, you can see one of the advance real-world example.<br>In the advance scenarios, we can combine ci/cd azure pipelines for pushing images to ACR and deploying to AKS. <br>I am not going details of this picture but We will see details on Azure Devops section.</p><h4>Deploy and use Azure Container Registry</h4><p>Azure Container Registry (ACR) is a private registry for container images. <br>A private container registry lets you securely build and deploy your applications and custom code. <br>We are going to deploy an ACR instance and push a container image to it.</p><ul><li>Create an Azure Container Registry (ACR) instance</li><li>Tag a container image for ACR</li><li>Upload the image to ACR</li><li>View images in your registry</li></ul><p>After these operations, this ACR instance is integrated with a Kubernetes cluster in AKS, and an application is deployed from the image.</p><p>We should understand to how to create resource on Azure. You can use portal or you can use;</p><p>Install the Azure CLI<br> <a href="https://docs.microsoft.com/en-us/cli/azure/install-azure-cli">https://docs.microsoft.com/en-us/cli/azure/install-azure-cli</a><br> <a href="https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?tabs=azure-cli">https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?tabs=azure-cli</a></p><p>Download and Install Azure CLI</p><h4>Create a resource group</h4><p>To create an Azure Container Registry, first need a resource group. <br>An Azure resource group is a logical container into which Azure resources are deployed and managed.</p><p>Create a resource group with the az group create command. <br>Location could be “westeurope” for us but you can choose your nearest data center for example “eastus”.</p><pre>az group create — name myResourceGroup — location westeurope</pre><h4>Create an Azure Container Registry</h4><p>Create an Azure Container Registry instance with the az acr create command and provide your own registry name. <br>The container registry name must be unique within Azure. <br>Acr name should be global unique.</p><pre>az acr create — resource-group myResourceGroup — name swnzenacr — sku Basic</pre><h4>Tag a container image</h4><p>Now we had a container registry and we can push our images, but first we need tag them.</p><p>In order to use shopping containers image with ACR, the image needs to be tagged with the login server address of your registry. <br>This tag is used for routing when pushing container images to an image registry.</p><pre>docker tag shoppingapi:latest swnzenacr.azurecr.io/shoppingapi:v1</pre><pre>docker tag shoppingclient:latest swnzenacr.azurecr.io/shoppingclient:v1</pre><p>Check docker images</p><pre>REPOSITORY TAG IMAGE ID CREATED SIZE<br>&lt;none&gt; &lt;none&gt; b6a015a1206b 19 minutes ago 649MB<br>shoppingclient latest 384a53ce0484 19 minutes ago 210MB<br>swnzenacr.azurecr.io/shoppingclient v1 384a53ce0484 19 minutes ago 210MB<br>shoppingapi latest 61d5a390a952 19 minutes ago 215MB<br>swnzenacr.azurecr.io/shoppingapi v1</pre><p>Push images to registry</p><pre>docker push swnzenacr.azurecr.io/shoppingapi:v1</pre><pre>docker push swnzenacr.azurecr.io/shoppingclient:v1</pre><p>List images in registry</p><pre>az acr repository list — name swnzenacr — output table</pre><p>As you can see that, we have finished to create acr and push our shopping images to the acr. <br>Next step we will create aks and provide to get our images from acr.</p><h4>Deploy an Azure Kubernetes Service (AKS) cluster</h4><p>Kubernetes provides a distributed platform for containerized applications. <br>With AKS, you can quickly create a production ready Kubernetes cluster.</p><p>We are going to :</p><ul><li>Deploy a Kubernetes AKS cluster that can authenticate to an Azure container registry</li><li>Install the Kubernetes CLI (kubectl)</li><li>Configure kubectl to connect to your AKS cluster</li></ul><p>After these operations, our Shopping application is deployed to the cluster, scaled, and updated.</p><p>We are going to Create an AKS cluster using az aks create.</p><p>We will create a cluster named myAKSCluster in the resource group named myResourceGroup. This resource group was created in the previous section in the westeurope region. For now we don&#39;t specify a region so the AKS cluster is also created in the westeurope region.</p><p>To allow an AKS cluster to interact with other Azure resources, an Azure Active Directory service principal is automatically created, since you did not specify one. <br>This service principal is granted the right to pull images from the Azure Container Registry (ACR) instance you created in the previous section. <br>So by this way, we only need to specify acr name with — attach-acr command in order to give access to pull images from AKS.</p><pre>az aks create — resource-group myResourceGroup — name myAKSCluster — node-count 1 — generate-ssh-keys — attach-acr swnzenacr</pre><p>After a few minutes, the command completes and returns JSON-formatted information about the cluster.</p><h4>Install the Kubernetes CLI</h4><p>In order to connect to the Kubernetes cluster from your local computer, we use kubectl, the Kubernetes command-line client.<br>So we should install kube cli locally using the az aks install-cli command:</p><pre>az aks install-cli</pre><h4>Connect to cluster using kubectl</h4><p>In order to configure kubectl to connect to your Kubernetes cluster, use the az aks get-credentials command. <br>We are going to gets credentials for the AKS cluster named myAKSCluster in the myResourceGroup:</p><pre>az aks get-credentials — resource-group myResourceGroup — name myAKSCluster</pre><p>Now we can verify To verify the connection to your cluster, run the kubectl get nodes command to return a list of the cluster nodes:</p><p>Test kubectl</p><pre>kubectl get nodes</pre><pre>NAME STATUS ROLES AGE VERSION<br>aks-nodepool1–12345678–0 Ready agent 32m v1.14.8</pre><pre>kubectl get all</pre><pre>NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br>service/kubernetes ClusterIP 10.0.0.1 &lt;none&gt; 443/TCP 12m</pre><p>As you can see that we have created AKS and connect kubernetes from our local computer with using kubectl commands.</p><h3>Run applications in Azure Kubernetes Service (AKS)</h3><p>We have created AKS and now it is ready to run our Shopping microservices on cloud AKS. We are going to build and deploy Shopping applications and services into a AKS Kubernetes cluster.<br>But before that we need to update existing k8s manifest yaml files.</p><p>So let me recap, We are going to;</p><ul><li>Update a Kubernetes manifest file</li><li>Run an application in Kubernetes</li><li>Test the application on AKS</li></ul><p>After these tasks, we will scaled out and updated to Shopping microservices on AKS.</p><h4>Edit K8s Manifest Yaml Files For Deploying AKS</h4><p>We are going to Edit existing K8s Manifest Yaml Files for Deploying AKS.<br>We have created AKS now on Azure Cloud and have created pull secret for ACR.<br>So for now we modify our k8s manifest files according to cloud AKS requirements.<br>Let’s take an action.</p><p>First of all, we should replace exiting yaml file images to ACR ones.</p><pre>Go to <br>shoppingapi.yaml</pre><pre>spec:<br> containers:<br> — name: shoppingapi<br> image: mehmetozkaya/shoppingapi:latest — CHANGE — swnzenacr.azurecr.io/shoppingapi:v1<br> ..<br> imagePullSecrets: — ADDED<br> — name: acr-secret — ADDED</pre><pre>Go to <br>shoppingclient.yaml</pre><pre>spec:<br> containers:<br> — name: shoppingclient<br> image: swnzenacr.azurecr.io/shoppingclient:v1<br> ..<br> imagePullSecrets: — ADDED<br> — name: acr-secret — ADDED</pre><p>As you can see that, we have replaced images names as it ACR names.<br> Also add imagePullSecrets configuration into deployment yaml container confugurations in order to allow to pull images.</p><p>Now we should also update service definitions</p><pre>Go to <br>shoppingapi.yaml</pre><pre>apiVersion: v1<br>kind: Service<br>metadata:<br> name: shoppingapi-service<br>spec:<br> type: NodePort — — DELETED<br> selector:<br> app: shoppingapi<br> ports:<br> — protocol: TCP<br> port: 8000 — — CHANGED 80<br> targetPort: 80 — — DELETED<br> nodePort: 31000 — — DELETED</pre><p>We are going to deploy AKS on live cloud so we dont need to port-forwarding , every pod take new ip from Cloud AKS. <br>We did it before in order to see in our localhost.<br>Thats why removed targetPort and set port = 80</p><h4>Run K8s Manifest Yaml Files For Deploying AKS</h4><p>we are going to run configured K8s Manifest Yaml Files for Deploying AKS.<br>As you know that we have configured k8s manifest files, now its time to run k8s yaml files on Azure AKS and Deploy the application.</p><p>Run command</p><pre>kubectl apply -f .\aks</pre><p>Check all resources on aks</p><pre>kubectl get all<br> NAME READY STATUS RESTARTS AGE<br> pod/mongo-deployment-754d654ff7-s2mcf 1/1 Running 0 56s<br> pod/shoppingapi-deployment-64964c4f6c-dxvzj 1/1 Running 0 56s<br> pod/shoppingclient-deployment-66fdcffbcd-8gwhf 1/1 Running 0 55s</pre><pre>NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br> service/kubernetes ClusterIP 10.0.0.1 &lt;none&gt; 443/TCP 29m<br> service/mongo-service ClusterIP 10.0.102.16 &lt;none&gt; 27017/TCP 56s<br> service/shoppingapi-service ClusterIP 10.0.79.109 &lt;none&gt; 8000/TCP 56s<br> service/shoppingclient-service LoadBalancer 10.0.78.220 20.71.9.224 8001:31498/TCP 55s</pre><pre>NAME READY UP-TO-DATE AVAILABLE AGE<br> deployment.apps/mongo-deployment 1/1 1 1 56s<br> deployment.apps/shoppingapi-deployment 1/1 1 1 56s<br> deployment.apps/shoppingclient-deployment 1/1 1 1 56s</pre><pre>NAME DESIRED CURRENT READY AGE<br> replicaset.apps/mongo-deployment-754d654ff7 1 1 1 56s<br> replicaset.apps/shoppingapi-deployment-64964c4f6c 1 1 1 56s<br> replicaset.apps/shoppingclient-deployment-66fdcffbcd 1 1 1 55s<br> PS C:\Users\ezozkme\source\repos\swnzen&gt;</pre><p>As you can see that, all k8s resources created successfully on AKS.</p><p>Once external-ip created, we can test the application.</p><p>TEST</p><p>FAIL<br> <a href="http://20.71.9.224/">http://20.71.9.224:80/</a></p><p>SUCCESS !!</p><p>Our application on live production now. So now we can scale and deploy new changes with zero-downtime on production. !!</p><h4>Scale Shopping applications in Azure Kubernetes Service (AKS)</h4><p>We have a working Kubernetes cluster in AKS and we deployed the Shopping microservices. <br>So now we are going to scale out the pods in the shopping app and try pod autoscaling.</p><p>We are going to;</p><ul><li>Scale the Kubernetes nodes</li><li>Manually scale Kubernetes pods that run your application</li><li>Configure autoscaling pods that run the app front-end</li></ul><p>You can scale with running below command;</p><pre>kubectl scale — replicas=3 deployment.apps/shoppingclient-deployment</pre><p>Scale with Yaml</p><p>Go to aks/shoppingclient.yaml</p><pre>Change</pre><pre>spec:<br> replicas: 3</pre><p>Run kubectl apply -f ./aks</p><p>See pods;</p><pre>kubectl get pod</pre><pre>mongo-deployment-754d654ff7-bpdp4 1/1 Running 0 81m<br>shoppingapi-deployment-56599b6f9f-llm6m 1/1 Running 0 81m<br>shoppingclient-deployment-774bc94c4-jdztz 1/1 Running 0 4m4s<br>shoppingclient-deployment-774bc94c4-khtxz 1/1 Running 0 81m<br>shoppingclient-deployment-774bc94c4-r5c2l 1/1 Running 0 13s</pre><p>As you can see that we have scaled our application both manually and using yaml files. But we can go one more step which is auto-scaling with aks.</p><p>For the next articles -&gt;</p><ul><li><a href="https://mehmetozkaya.medium.com/automate-deployments-with-ci-cd-pipelines-on-azure-devops-13a83d3dd67a"><strong>4- Automate Deployments with CI/CD pipelines on Azure Devops</strong></a></li></ul><h3>References</h3><p><a href="https://docs.microsoft.com/en-us/azure/container-registry/container-registry-intro">https://docs.microsoft.com/en-us/azure/container-registry/container-registry-intro</a><br><a href="https://docs.microsoft.com/en-us/azure/container-registry/container-registry-tutorial-prepare-registry">https://docs.microsoft.com/en-us/azure/container-registry/container-registry-tutorial-prepare-registry</a><br><a href="https://docs.microsoft.com/en-us/azure/container-registry/container-registry-auth-kubernetes">https://docs.microsoft.com/en-us/azure/container-registry/container-registry-auth-kubernetes</a><br><a href="https://docs.microsoft.com/en-us/azure/aks/">https://docs.microsoft.com/en-us/azure/aks/</a><br><a href="https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough">https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough</a><br><a href="https://docs.microsoft.com/en-us/azure/aks/concepts-clusters-workloads?WT.mc_id=containers-11729-bbenz">https://docs.microsoft.com/en-us/azure/aks/concepts-clusters-workloads?WT.mc_id=containers-11729-bbenz</a><br><a href="https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-scale">https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-scale</a><br><a href="https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-app-update">https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-app-update</a><br><a href="https://www.digitalocean.com/community/questions/k8s-insufficient-cpu-even-though-there-is-enough-cpu">https://www.digitalocean.com/community/questions/k8s-insufficient-cpu-even-though-there-is-enough-cpu</a><br><a href="https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/">https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b661698610b1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/deploy-microservices-into-cloud-azure-kubernetes-service-aks-with-using-azure-container-registry-b661698610b1">Deploy Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry…</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying Microservices on Kubernetes]]></title>
            <link>https://medium.com/aspnetrun/deploying-microservices-on-kubernetes-35296d369fdb?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/35296d369fdb</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[azure-kubernetes-service]]></category>
            <category><![CDATA[docker-compose]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Fri, 02 Apr 2021 06:16:18 GMT</pubDate>
            <atom:updated>2023-04-09T12:23:06.139Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article, we are going to Deploy our Shopping Microservices on Kubernetes. Kubernetes retrieve microservices images from DockerHub.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*2WHIbwc-NX9uATWKmEaJGA.png" /></figure><p>See the overall picture. As you can see that, we have created docker images for our microservices. Also we compose docker containers and tested them.<br>So now, we are going to deploy these docker container images on <strong>Kubernetes </strong>clusters. <br>We are going to deploy local Kubernetes so for that reason we will use <strong>Docker Kubernetes cluster</strong> in order to use Kubernetes features in docker. <br>It is the easiest way to run Kubernetes on local environment.</p><p>We will develop Kubenernetes manifest yamls files for Shopping Client — API and mongoDb. <br>We will create deployment and service yamls and also configMap and secret definitions for storing database related parameters.</p><h3>Background</h3><p>This is the second article of the series. You can follow the series with below links.</p><ul><li><a href="https://mehmetozkaya.medium.com/deploying-net-microservices-to-azure-kubernetes-services-aks-and-automating-with-azure-devops-c50bdd51b702"><strong>0- Deploying .Net Microservices</strong></a></li><li><a href="https://mehmetozkaya.medium.com/preparing-multi-container-microservices-applications-for-deployment-793d60f48d31"><strong>1- Preparing Multi-Container Microservices Applications for Deployment</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploying-microservices-on-kubernetes-35296d369fdb"><strong>2- Deploying Microservices on Kubernetes</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploy-microservices-into-cloud-azure-kubernetes-service-aks-with-using-azure-container-registry-b661698610b1"><strong>3- Deploy Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry (ACR)</strong></a></li><li><a href="https://mehmetozkaya.medium.com/automate-deployments-with-ci-cd-pipelines-on-azure-devops-13a83d3dd67a"><strong>4- Automate Deployments with CI/CD pipelines on Azure Devops</strong></a></li></ul><h3>Step by Step Development w/ Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/0*Sk4rNFAMPvknoFhz.png" /></figure><p><a href="https://www.udemy.com/course/deploying-net-microservices-with-k8s-aks-and-azure-devops/?couponCode=APRI23"><strong>I have just published a new course — Deploying .Net Microservices with K8s, AKS and Azure DevOps.</strong></a></p><p>In this course, we’re going to learn how to <strong>Deploying .Net Microservices</strong> into <strong>Kubernetes</strong>, and moving deployments to the cloud <strong>Azure kubernetes services (AKS)</strong> with using <strong>Azure Container Registry(ACR)</strong> and last section is we will learn how to Automating Deployments with <strong>CI/CD pipeline of Azure DevOps</strong> and GitHub.</p><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-devops"><strong>Get the Source Code from AspnetRun Microservices Github</strong></a> — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.</p><h3>Introduction to Kubernetes</h3><p>Before we start to deploy microservices on Kubernetes, we are going to give introduction about Kubernetes.<br>We will talk about What Kubernetes is and why we should use Kubernetes for deploying microservices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6vb9LwyU5PpbSCw2B_RD9A.png" /></figure><p>Also we will talk about kubernetes components and general architecture of operations. We will discuss minimum manifest definitions for deploying microservices, we will talk about <strong>deployments, replicasets, pods, services </strong>and so on. We will install kubernetes on our local machine. And see main kubectl commands that can create kubernetes resources. Give some information about K8s yaml configuration files.</p><h4>What is Kubernetes ?</h4><p>Kubernetes (also known as k8s or “kube”) is an open source container orchestration platform that automates many of the manual processes involved in deploying, managing, and scaling containerized applications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ACDQ0zG8Yo_bpw5zrS4W3Q.png" /></figure><p>Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.</p><p>You can cluster together groups of hosts running Linux containers, and Kubernetes helps you easily and efficiently manage those clusters.</p><h4>Kubernetes Components</h4><p>we are going to talk about Kubernetes Components like <strong>deployment, replicaset, pod, service, configmap</strong> and so on.<br>We will explain most basics of components that we can use and deploy our microservice to the k8s.</p><p><strong>Pods</strong><br>Pods are the smallest deployable units of computing that you can create and manage in Kubernetes.<br>A Pod is a group of one or more containers, with shared storage/network resources, and a specification for how to run the containers. <br>So we can say that pods stores and manage our docker containers.</p><p><strong>ReplicaSet</strong><br>A ReplicaSet’s purpose is to maintain a stable set of replica Pods running at any given time. For example, it is often used to guarantee the availability of a specified number of identical Pods.</p><p><strong>Deployments</strong><br>A Deployment provides declarative updates for Pods and ReplicaSets.</p><p>You describe a desired state in a Deployment, and the Deployment Controller changes the actual state to the desired state at a controlled rate. <br>You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*yp8HTZE99LM4p_7F-VzgHw.png" /></figure><p>So we can say that Deployments are an abstraction of ReplicaSets, and ReplicaSets are an abstaction of Pods. So Pods should not created directly, if needed, Deployment objects should be created and the rest operation will handle by k8s with creating replicaset and pods automaticly.</p><p><strong>Service</strong><br>An abstract way to expose an application running on a set of Pods as a network service. With Kubernetes you don’t need to modify your application to use an unfamiliar service discovery mechanism. <br>Kubernetes gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them.<br><strong>ConfigMaps</strong><br>A ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume.</p><p><strong>Secrets</strong><br>Kubernetes Secrets let you store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys. Storing confidential information in a Secret is safer and more flexible than putting it verbatim in a Pod definition or in a container image.</p><h4>Local Kubernetes Installment</h4><p>We are going to install and run Kubernetes on local environment. <br>There are few options to run kubernetes on local like minikube, docker kubernetes and so on. We will follow the docker kubernetes installation.</p><p>We should go to Docker Settings</p><ul><li>Docker support of Kubernetes<br> Go to Docker Settings<br> Kubernetes section<br> Enable Kubernetes</li></ul><h4>Declarative vs Imperative</h4><p>We are going to talk about Declarative vs Imperative.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hyx2Li8VgvHkEMwcpzo_1g.png" /></figure><p>There are two basic ways to deploy to Kubernetes: imperatively, which is working on CLI with kubectl commands, or declaratively, by writing manifests and using kubectl apply. <br>Kubernetes able to run both Declarative and Imperative way.</p><p><strong>Imperative Configuration</strong><br>The shortest way to deploy to Kubernetes is to use the kubectl run command. <br>It said that run myapp with some image with 2 replicas, now!</p><p><strong>kubectl run myapp — image myrepo:mytag — replicas 2</strong></p><p><strong>Declarative Configuration</strong><br>The power of Kubernetes is in its declarative API and controllers. <br>You can just tell Kubernetes what you want, and it will know what to do.</p><p><strong>kubectl apply –f app.yaml</strong></p><p>You’ll just use kubectl apply and YAML (or JSON) manifests of the state to be saved by Kubernetes in etcd.<br>Impritive — Declarative<br>Impritive Commands</p><p><strong>kubectl run [container_name] — image=[image_name]<br>kubectl port-forward [pod] [ports]</strong></p><p><strong>kubectl create [resource]<br>kubectl apply [resource] — create or modify resources</strong></p><p>So we will start with Imperative way of kubernetes commands. After that show how we can work with yaml files.</p><h3>Create Mongo Db Deployment yaml File</h3><p>We are going to Create Mongo Db Deployment yaml File.</p><p>Before we start, lets Check Mongo DockerHub Page</p><p>Check Mongo DockerHub Page :<br> <a href="https://hub.docker.com/_/mongo">https://hub.docker.com/_/mongo</a></p><ul><li>See Env Variables<br> mongo:<br> image: mongo<br> restart: always<br> environment:<br> MONGO_INITDB_ROOT_USERNAME: root<br> MONGO_INITDB_ROOT_PASSWORD: example</li></ul><p>We will use these environment variables for activate mongodb.<br><strong>Add New file -&gt; mongo.yaml</strong></p><pre>mongo.yaml</pre><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br> name: mongo-deployment<br> labels:<br> app: mongodb<br>spec:<br> replicas: 1<br> selector:<br> matchLabels:<br> app: mongodb<br> template:<br> metadata:<br> labels:<br> app: mongodb<br> spec:<br> containers:<br> — name: mongodb<br> image: mongo<br> ports:<br> — containerPort: 27017<br> env:<br> — name: MONGO_INITDB_ROOT_USERNAME<br> value:<br> — name: MONGO_INITDB_ROOT_PASSWORD<br> value: <br> — <br> — For username — password environment variables, its good to create secret definition on k8s.</pre><h4>Use K8s Secret Values in Mongo Deployment yaml file</h4><p>We are going to Use kubernetes Secret Values in Mongo Deployment yaml file.<br>As you know that we didnt finish the mongo deployment yaml. Now we can able to set mongo username and password.</p><p>— So now, our mongo deployment yaml referenced to secret;</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br> name: mongo-deployment<br> labels:<br> app: mongodb<br>spec:<br> replicas: 1<br> selector:<br> matchLabels:<br> app: mongodb<br> template:<br> metadata:<br> labels:<br> app: mongodb<br> spec:<br> containers:<br> — name: mongodb<br> image: mongo<br> ports:<br> — containerPort: 27017<br> resources:<br> requests:<br> memory: “64Mi”<br> cpu: “250m”<br> limits:<br> memory: “128Mi”<br> cpu: “500m”<br> env:<br> — name: MONGO_INITDB_ROOT_USERNAME<br> valueFrom:<br> secretKeyRef:<br> name: mongo-secret<br> key: mongo-root-username<br> — name: MONGO_INITDB_ROOT_PASSWORD<br> valueFrom:<br> secretKeyRef:<br> name: mongo-secret<br> key: mongo-root-password</pre><p>— As you can see that we have used valueFrom and secretKeyRef in order to access secret data. <br>K8s manage our data with these definitions.</p><h4>Run Kubernetes Manifest File</h4><p>We are ready to create</p><p><strong>kubectl apply -f .\mongo.yaml</strong></p><pre>PS C:\Users\ezozkme\source\repos\swnzen\k8s&gt; kubectl apply -f .\mongo.yaml<br> deployment.apps/mongo-deployment created<br> PS C:\Users\ezozkme\source\repos\swnzen\k8s&gt; kubectl get all</pre><p>Watch pods</p><p><strong>kubectl get pod — watch<br>kubectl describe pod mongo-deployment-9c5b4dddb-lps5h</strong></p><p>SUCCESS !!</p><p>We finally created mongo db pods with username-password secret protection.</p><h4>Build Shopping Docker Images , Tag and Push to Docker Hub</h4><p>We are going to Build Shopping Docker Images , Tag and Push to Docker Hub.<br>As you know that in the last section we have finished mongodb k8s yaml definitions. For mongodb, k8s pull the official mongo docker hub image.<br>But shopping images not exits on dockerhub yet.<br>For that reason, before writing k8s yaml file for Shopping projects, we should create docker images, tag them and push to docker hub.<br>By this way, k8s retrieves images from the docker hub.</p><p>run docker-compose</p><p><strong>run = docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d<br> stop = docker-compose -f docker-compose.yml -f docker-compose.override.yml down</strong></p><pre>docker ps <br> <br>PS C:\Users\ezozkme\source\repos\swnzen\swnzen&gt; docker ps<br>CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES<br>ed664082a27f shoppingclient “dotnet Shopping.Cli…” 7 seconds ago Up 5 seconds 443/tcp, 0.0.0.0:8001-&gt;80/tcp shoppingclient<br>f0a20a1e9467 shoppingapi “dotnet Shopping.API…” 7 seconds ago Up 6 seconds 443/tcp, 0.0.0.0:8000-&gt;80/tcp shoppingapi<br>507ece34b2f0 mongo “docker-entrypoint.s…” 8 seconds ago Up 6 seconds 0.0.0.0:27017-&gt;27017/tcp shoppingdb</pre><p>TEST<br><a href="http://localhost:8000/swagger/index.html">http://localhost:8000/swagger/index.html</a><br><a href="http://localhost:8001/">http://localhost:8001/</a></p><ul><li>see images<br> docker images</li></ul><pre>shoppingclient latest 3fa4c59328fd 9 minutes ago 210MB<br> shoppingapi latest 054137853823 10 minutes ago 215MB</pre><p>latest tag images created.</p><p>We will Tag images and push to dockerhub.</p><p>Tag the lastest one with dockerhub repo name</p><p><strong>docker tag 3fa mehmetozkaya/shoppingclient</strong></p><p><strong>docker tag 054 mehmetozkaya/shoppingapi</strong><br><br>docker images</p><pre> shoppingclient latest 3fa4c59328fd 12 minutes ago 210MB<br> mehmetozkaya/shoppingclient latest 3fa4c59328fd 12 minutes ago 210MB<br> shoppingapi latest 054137853823 12 minutes ago 215MB<br> mehmetozkaya/shoppingapi latest 054137853823 12 minutes ago 215MB</pre><p>see its tagged.</p><p>Push Docker Hub</p><pre>docker push mehmetozkaya/shoppingclient</pre><pre>docker push mehmetozkaya/shoppingapi</pre><pre>The push refers to repository [docker.io/mehmetozkaya/shoppingclient]<br> b57cdc9e8ec8: Pushed<br> d066a90a6a65: Pushed<br> 024230939f4e: Pushed<br> ea4124eb3c7e: Pushed<br> 8ed87ee178f4: Pushing [======================================&gt; ] 58.3MB/75.66MB<br> 0916aa79e133: Pushing [==================================&gt; ] 28.54MB/41.33MB<br> 87c8a1d8f54f: Pushing [====================&gt; ] 28.68MB/69.23MB</pre><p>See DockerHub</p><ul><li>mehmetozkaya/shoppingclient<br> mehmetozkaya/shoppingapi<br> Last pushed: 12 minutes ago</li></ul><h4>Create Shopping.API k8s Deployment and Service yaml File</h4><p>We are going to create Shopping.API k8s Deployment yaml File.</p><p>Create New File under k8s</p><pre><strong>shoppingapi.yaml</strong></pre><pre>Write Deployment and Service</pre><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br> name: shoppingapi-deployment<br> labels:<br> app: shoppingapi<br>spec:<br> replicas: 1<br> selector:<br> matchLabels:<br> app: shoppingapi<br> template:<br> metadata:<br> labels:<br> app: shoppingapi<br> spec:<br> containers:<br> — name: shoppingapi<br> image: mehmetozkaya/shoppingapi:latest<br> imagePullPolicy: IfNotPresent<br> ports:<br> — containerPort: 80<br> env:<br> — name: ASPNETCORE_ENVIRONMENT<br> value: Development<br> — name: DatabaseSettings__ConnectionString<br> value: mongo-service<br> resources:<br> requests:<br> memory: “64Mi”<br> cpu: “250m”<br> limits:<br> memory: “128Mi”<br> cpu: “500m”<br> — -<br>apiVersion: v1<br>kind: Service<br>metadata:<br> name: shoppingapi-service<br>spec:<br> type: NodePort — ADDED TESTING PURPOSE<br> selector:<br> app: shoppingapi<br> ports:<br> — protocol: TCP<br> port: 8000<br> targetPort: 80<br> nodePort: 31000 — ADDED TESTING PURPOSE</pre><p>In order to add test our API we set the service as a nodeport.<br>So now we can run our shoppingapi yaml file on k8s.</p><h4>Create Shopping.Client K8s Deployment and Service yaml File</h4><p>We are going to Create Shopping.Client K8s Deployment and Service yaml File.</p><p>Create New File under k8s</p><pre><strong>shoppingclient.yaml</strong></pre><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br> name: shoppingclient-deployment<br> labels:<br> app: shoppingclient<br>spec:<br> replicas: 1<br> selector:<br> matchLabels:<br> app: shoppingclient<br> template:<br> metadata:<br> labels:<br> app: shoppingclient<br> spec:<br> containers:<br> — name: shoppingclient<br> image: mehmetozkaya/shoppingclient:latest<br> imagePullPolicy: IfNotPresent<br> ports:<br> — containerPort: 80<br> env:<br> — name: ASPNETCORE_ENVIRONMENT<br> value: Development<br> — name: ShoppingAPIUrl<br> valueFrom:<br> configMapKeyRef:<br> name: shoppingapi-configmap<br> key: shoppingapi_url<br> resources:<br> requests:<br> memory: “64Mi”<br> cpu: “250m”<br> limits:<br> memory: “128Mi”<br> cpu: “500m”<br> — -<br>apiVersion: v1<br>kind: Service<br>metadata:<br> name: shoppingclient-service<br>spec:<br> type: LoadBalancer<br> selector:<br> app: shoppingclient<br> ports:<br> — protocol: TCP<br> port: 8001<br> targetPort: 80<br> nodePort: 30000</pre><p>As you can see we create LoadBalancer for external access.</p><p>Run apply k8s command;</p><p><strong>kubectl apply -f shoppingclient.yaml</strong></p><p>Check<br> <strong>kubectl get all<br> <br> kubectl get pod — watch</strong></p><p>created!</p><p><strong>kubectl get service</strong></p><pre>NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br>kubernetes ClusterIP 10.96.0.1 &lt;none&gt; 443/TCP 29d<br>mongo-service ClusterIP 10.110.82.16 &lt;none&gt; 27017/TCP 5h12m<br>shoppingapi-service NodePort 10.104.209.228 &lt;none&gt; 8000:31000/TCP 113m<br>shoppingclient-service LoadBalancer 10.107.145.225 localhost 8001:30000/TCP 85s</pre><p>TEST:</p><p><a href="http://localhost:30000/">http://localhost:30000</a></p><p>As you can see that, we have finally deploy our microservices into local Kubernetes environment. Next article we are going to deploy microservices into Azure Kubernetes Services (AKS) with using Azure Container Registry (ACR).</p><p>For the next articles -&gt;</p><ul><li><a href="https://mehmetozkaya.medium.com/deploy-microservices-into-cloud-azure-kubernetes-service-aks-with-using-azure-container-registry-b661698610b1"><strong>3- Deploy Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry (ACR)</strong></a></li></ul><h3>References</h3><p><a href="https://www.redhat.com/en/topics/containers/what-is-kubernetes">https://www.redhat.com/en/topics/containers/what-is-kubernetes</a><br><a href="https://blog.newrelic.com/engineering/what-is-kubernetes/">https://blog.newrelic.com/engineering/what-is-kubernetes/</a><br><a href="https://medium.com/payscale-tech/imperative-vs-declarative-a-kubernetes-tutorial-4be66c5d8914">https://medium.com/payscale-tech/imperative-vs-declarative-a-kubernetes-tutorial-4be66c5d8914</a><br><a href="https://medium.com/bitnami-perspectives/imperative-declarative-and-a-few-kubectl-tricks-9d6deabdde">https://medium.com/bitnami-perspectives/imperative-declarative-and-a-few-kubectl-tricks-9d6deabdde</a><br><a href="https://www.youtube.com/watch?v=X48VuDVv0do&amp;t=4576s">https://www.youtube.com/watch?v=X48VuDVv0do&amp;t=4576s</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35296d369fdb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/deploying-microservices-on-kubernetes-35296d369fdb">Deploying Microservices on Kubernetes</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Preparing Multi-Container Microservices Applications for Deployment]]></title>
            <link>https://medium.com/aspnetrun/preparing-multi-container-microservices-applications-for-deployment-793d60f48d31?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/793d60f48d31</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker-compose]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[azure-container-registry]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Fri, 02 Apr 2021 06:16:08 GMT</pubDate>
            <atom:updated>2021-10-02T11:20:49.812Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article, we’re going to prepare our <strong>multi-container microservices </strong>application. We will develop from scratch and create <strong>DockerFiles</strong>, Build images and orchestrate with <strong>docker-compose</strong> yaml files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*2WHIbwc-NX9uATWKmEaJGA.png" /></figure><p>See the overall picture. You can see that we will have 3 microservices which we are going to develop and deploy together.</p><h4>Shopping MVC Client Application</h4><p>First of all, we are going to develop Shopping MVC Client Application For Consuming Api Resource which will be the Shopping.Client Asp.Net MVC Web Project. But we will start with developing this project as a standalone Web application which includes own data inside it. And we will add container support with <strong>DockerFile</strong>, push docker images to Docker hub and see the deployment options like “Azure Web App for Container” resources for 1 web application.</p><h4>Shopping API Application</h4><p>After that we are going to develop Shopping.API Microservice with MongoDb and Compose All Docker Containers. <br>This API project will have Products data and performs CRUD operations with exposing api methods for consuming from Shopping Client project.<br>We will containerize API application with creating dockerfile and push images to Azure Container Registry.</p><h4>Mongo Db</h4><p>Our API project will manage product records stored in a no-sql mongodb database as described in the picture. <br>we will pull <strong>mongodb </strong>docker image from docker hub and create connection with our API project.</p><p>At the end of the section, we will have 3 microservices whichs are <strong>Shopping.Client — Shopping.API — MongoDb</strong> microservices.</p><p>As you can see that, we have</p><ul><li>Created docker images,</li><li>Compose docker containers and tested them,</li></ul><h3>Background</h3><p>This is the first article of the series. You can follow the series with below links.</p><ul><li><a href="https://mehmetozkaya.medium.com/deploying-net-microservices-to-azure-kubernetes-services-aks-and-automating-with-azure-devops-c50bdd51b702"><strong>0- Deploying .Net Microservices</strong></a></li><li><a href="https://mehmetozkaya.medium.com/preparing-multi-container-microservices-applications-for-deployment-793d60f48d31"><strong>1- Preparing Multi-Container Microservices Applications for Deployment</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploying-microservices-on-kubernetes-35296d369fdb"><strong>2- Deploying Microservices on Kubernetes</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploy-microservices-into-cloud-azure-kubernetes-service-aks-with-using-azure-container-registry-b661698610b1"><strong>3- Deploy Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry (ACR)</strong></a></li><li><a href="https://mehmetozkaya.medium.com/automate-deployments-with-ci-cd-pipelines-on-azure-devops-13a83d3dd67a"><strong>4- Automate Deployments with CI/CD pipelines on Azure Devops</strong></a></li></ul><h3>Step by Step Development w/ Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/0*6bjp3FCov5ZyrROm.png" /></figure><p><a href="https://www.udemy.com/course/deploying-net-microservices-with-k8s-aks-and-azure-devops/?couponCode=OCTOBER2021"><strong>I have just published a new course — Deploying .Net Microservices with K8s, AKS and Azure DevOps.</strong></a></p><p>In this course, we’re going to learn how to <strong>Deploying .Net Microservices</strong> into <strong>Kubernetes</strong>, and moving deployments to the cloud <strong>Azure kubernetes services (AKS)</strong> with using <strong>Azure Container Registry(ACR)</strong> and last section is we will learn how to Automating Deployments with <strong>CI/CD pipeline of Azure DevOps</strong> and GitHub.</p><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-devops"><strong>Get the Source Code from AspnetRun Microservices Github</strong></a> — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.</p><h3>Developing Your First Microservice</h3><p>In this section we are going to Building MVC Client Application For Consuming Api Resource which will be the Shopping.Client Asp.Net MVC Web Project. But we will start with developing simple MVC application before integrate with API project.</p><p>As you can see the overall picture, we are in here and start to developing Shopping MVC Web Application project. <br>First, We will develop this project as a standalone Web application which includes own data inside it. And we will add container support with DockerFile, push docker images to Docker hub and see the deployment options like “Azure Web App for Container” resources for 1 web application.</p><p>So in this section, we are going to develop Shopping.Client MVC application which will be standalone web application and includes own data. <br>Next sections we will consume the Shopping.API project.</p><p>— First, we are going to create;<br>Create New Project<br>New Blank Solution</p><p>and Give the Name of the solution is “Shopping”</p><p>— Create Asp.Net Core Web Project :<br> Right Click — Add new Web Project — Asp.Net Core Web Application — Web Application (Model-View-Controller) <br> — not select HTTPS <br> — n : Shopping.Client</p><p>— Run the application<br> Set a Startup Project<br> Change Run Profile to “Shopping.Client” and Run the application<br> See 5001 http port worked — <a href="http://localhost:5001/">http://localhost:5001</a></p><h4>Adding Model Class into MVC Application</h4><p>In Solution Explorer, select the project. Create Models folder.<br> Right-click the Models folder and select Add &gt; Class. Name the class Product and select Add.</p><p>Create Product Model into Models folder</p><pre>namespace Shopping.Client.Models<br> {<br> public class Product<br> {<br> public string Id { get; set; }<br> public string Name { get; set; }<br> public string Category { get; set; }<br> public string Description { get; set; }<br> public string ImageFile { get; set; }<br> public decimal Price { get; set; }<br> }<br> }</pre><h4>Developing Shopping.Client Microservices Data Model and Context Objects</h4><p>Every microservice should have its own database, so we should create data store for Shopping.Client Microservices.</p><p>We should create a context class which should store Discount entity objects. <br>So I am going to;</p><p>Create Data Folder<br>Add ProductContext.cs</p><pre>public static class ProductContext<br> { <br> public static readonly List&lt;Product&gt; Products = new List&lt;Product&gt;<br> {<br> new Product()<br> {<br> Name = “IPhone X”,<br> Description = “This phone is the company’s biggest change to its flagship smartphone in years. It includes a borderless.”,<br> ImageFile = “product-1.png”,<br> Price = 950.00M,<br> Category = “Smart Phone”<br> },<br> new Product()<br> {<br> Name = “Samsung 10”,<br> Description = “This phone is the company’s biggest change to its flagship smartphone in years. It includes a borderless.”,<br> ImageFile = “product-2.png”,<br> Price = 840.00M,<br> Category = “Smart Phone”<br> },<br> new Product()<br> {<br> Name = “Huawei Plus”,<br> Description = “This phone is the company’s biggest change to its flagship smartphone in years. It includes a borderless.”,<br> ImageFile = “product-3.png”,<br> Price = 650.00M,<br> Category = “White Appliances”<br> },<br> new Product()<br> {<br> Name = “Xiaomi Mi 9”,<br> Description = “This phone is the company’s biggest change to its flagship smartphone in years. It includes a borderless.”,<br> ImageFile = “product-4.png”,<br> Price = 470.00M,<br> Category = “White Appliances”<br> },<br> new Product()<br> {<br> Name = “HTC U11+ Plus”,<br> Description = “This phone is the company’s biggest change to its flagship smartphone in years. It includes a borderless.”,<br> ImageFile = “product-5.png”,<br> Price = 380.00M,<br> Category = “Smart Phone”<br> },<br> new Product()<br> {<br> Name = “LG G7 ThinQ New8”,<br> Description = “This phone is the company’s biggest change to its flagship smartphone in years. It includes a borderless.”,<br> ImageFile = “product-6.png”,<br> Price = 240.00M,<br> Category = “Home Kitchen”<br> }<br> }<br> }</pre><h4>Listing Products on Index Page of Shopping.Client Microservice</h4><p>We are going to Listing Products on Index Page of Shopping.Client Microservice.</p><p>First, we are going to update home page of MVC Application;</p><pre>HomeController.cs<br> public IActionResult Index()<br> {<br> return View(ProductContext.Products); — ADDED parameter<br> }</pre><p>So we have now Products on Index page, we should map this data into html list objects.<br> After that, let me modify Index.cshtml</p><p>Update Index.cshtml</p><pre><a href="http://twitter.com/model">@model</a> IEnumerable&lt;Shopping.Client.Models.Product&gt;</pre><pre>@{<br> ViewData[“Title”] = “Home Page”;<br> }</pre><pre>&lt;h1&gt;Products&lt;/h1&gt;<br> &lt;table class=”table”&gt;<br> &lt;thead&gt;<br> &lt;tr&gt;<br> &lt;th&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayNameFor(model =&gt; model.Name)<br> &lt;/th&gt;<br> &lt;th&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayNameFor(model =&gt; model.Category)<br> &lt;/th&gt; <br> &lt;th&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayNameFor(model =&gt; model.Description)<br> &lt;/th&gt;<br> &lt;th&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayNameFor(model =&gt; model.ImageFile)<br> &lt;/th&gt;<br> &lt;th&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayNameFor(model =&gt; model.Price)<br> &lt;/th&gt;<br> &lt;th&gt;&lt;/th&gt;<br> &lt;/tr&gt;<br> &lt;/thead&gt;<br> &lt;tbody&gt;<br> <a href="http://twitter.com/foreach">@foreach</a> (var item in Model)<br> {<br> &lt;tr&gt;<br> &lt;td&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayFor(modelItem =&gt; item.Name)<br> &lt;/td&gt;<br> &lt;td&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayFor(modelItem =&gt; item.Category)<br> &lt;/td&gt; <br> &lt;td&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayFor(modelItem =&gt; item.Description)<br> &lt;/td&gt;<br> &lt;td&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayFor(modelItem =&gt; item.ImageFile)<br> &lt;/td&gt;<br> &lt;td&gt;<br> <a href="http://twitter.com/Html">@Html</a>.DisplayFor(modelItem =&gt; item.Price)<br> &lt;/td&gt; <br> &lt;/tr&gt;<br> }<br> &lt;/tbody&gt;<br> &lt;/table&gt;</pre><p>Finally, we can Run the application.<br> Set a Startup Project<br> Change Run Profile to “Shopping.Client” and Run the application<br> See 5001 http port worked — <a href="http://localhost:5001/">http://localhost:5001</a></p><h4>Create Docker Container for Shopping.Client Microservice</h4><p>We are going to Create Docker Container for our Shopping.Client Microservice. For creation docker container, we need to create Dockerfile for our MVC Application. This is very easy to use vs tooling.</p><p>First, we are going to Add Docker Support on our vs solution;</p><p>Right click — Add Docker Support — Linux</p><p>So this operation, create a new Dockerfile for us.<br> Let’s see the Dockerfile<br>Examine DockerFile</p><p>The Dockerfile file, which we will make the necessary settings for Docker, automatically appears.<br>The purpose of creating the Dockerfile file; When we ask docker to extract the image of our project, it will search for a file named “Dockerfile” in the project. This will make our application work according to the settings in our file.</p><p>See file;</p><pre>FROM mcr.microsoft.com/dotnet/aspnet:5.0-buster-slim AS base<br> WORKDIR /app<br> EXPOSE 80<br> EXPOSE 443</pre><pre>FROM mcr.microsoft.com/dotnet/sdk:5.0-buster-slim AS build<br> WORKDIR /src<br> COPY [“Shopping.Client/Shopping.Client.csproj”, “Shopping.Client/”]<br> RUN dotnet restore “Shopping.Client/Shopping.Client.csproj”<br> COPY . .<br> WORKDIR “/src/Shopping.Client”<br> RUN dotnet build “Shopping.Client.csproj” -c Release -o /app/build</pre><pre>FROM build AS publish<br> RUN dotnet publish “Shopping.Client.csproj” -c Release -o /app/publish</pre><pre>FROM base AS final<br> WORKDIR /app<br> COPY — from=publish /app/publish .<br> ENTRYPOINT [“dotnet”, “Shopping.Client.dll”]</pre><p>Dockerfile consists of 2 main parts. The first is to build the application and the second is to run the application. If we look at what the commands are in the file;</p><p><strong>FROM </strong>part where the base image is specified in whichever library the FROM project is used in. We will use dotnet 5 SDK image in this project.</p><p><strong>WORKDIR </strong>It is the part where we specify the folder under which the docker container will copy the files of our project.</p><p><strong>COPY </strong>is the command used to copy project files from local file system to image. In our project, we will first copy and restore the csproj file, then copy all these files again and create our application by running the dotnet publish command.</p><p><strong>RUN</strong>: It is used for commands that need to run while Docker containers are being prepared. First, it is ensured that the build is taken and then it is published.</p><p><strong>ENTRYPOINT </strong>is the command that we specify the first command and parameters that will run when the container is up. While the container is running, DockerExample.dll will be executed with the dotnet command.</p><p><strong>Check Output Logs</strong><br>Once we added Docker Support, in output its automatilcy build docker images</p><p><strong>Check docker images output logs</strong></p><pre><br> Starting up container(s)…<br> docker build -f “C:\Users\ezozkme\source\repos\swnzen\swnzen\Shopping.Client\Dockerfile” — force-rm -t shoppingclient:dev — target base — label “com.microsoft.created-by=visual-studio” — label “com.microsoft.visual-studio.project-name=Shopping.Client” “C:\Users\ezozkme\source\repos\swnzen\swnzen” <br> Sending build context to Docker daemon 4.386MB</pre><pre>Step 1/6 : FROM mcr.microsoft.com/dotnet/aspnet:5.0-buster-slim AS base<br> — -&gt; 5f9a6a778eac<br> Step 2/6 : WORKDIR /app<br> — -&gt; Running in d7b126c32d92<br> Removing intermediate container d7b126c32d92<br> — -&gt; d23b0bf721d8<br> Step 3/6 : EXPOSE 80<br> — -&gt; Running in 39654ab99d8b<br> Removing intermediate container 39654ab99d8b<br> — -&gt; 4c66963a83fe<br> Step 4/6 : EXPOSE 443<br> — -&gt; Running in efbdcf6019d2<br> Removing intermediate container efbdcf6019d2<br> — -&gt; 6588782be6b6<br> Step 5/6 : LABEL com.microsoft.created-by=visual-studio<br> — -&gt; Running in 8394a8470229<br> Removing intermediate container 8394a8470229<br> — -&gt; 4a876be8a155<br> Step 6/6 : LABEL com.microsoft.visual-studio.project-name=Shopping.Client<br> — -&gt; Running in 2861da6e1c69<br> Removing intermediate container 2861da6e1c69<br> — -&gt; 377350c2c197</pre><p>— As you can see that run “docker build” command. we can also run manually. But vs has great tooling experience for docker support.</p><h4>Push Docker Hub Container Registry to Shopping.Client Microservice Docker Image</h4><p>We are going to Push Docker Hub Container Registry to Shopping.Client Microservice Docker Image.</p><p>We should login the docker over the command line. Before push the image we should login the system.</p><ul><li>Login Docker</li><li>before push<br> docker login<br> username<br> pass</li></ul><pre>docker login<br>Login with your Docker ID to push and pull images from Docker Hub. If you don’t have a Docker ID, head over to <a href="https://hub.docker.com/">https://hub.docker.com</a> to create one.<br>Username: mehmetozkaya<br>Password:<br>Login Succeeded</pre><h4>Tag Docker Image For Docker Hub</h4><p>Before push the image, the image should have tag, so we should tag the image. It is mandatory step to apply tag for image that will be push to Docker Hub or any container registry, the image should have tag.</p><p>We are going to Tag from “lastest” one not dev one. That means ready for production deployment tag.</p><p>Lets check existing image;</p><pre>docker images</pre><pre>REPOSITORY TAG IMAGE ID CREATED SIZE<br>shoppingclient latest cc2a573482cf 45 minutes ago 210MB</pre><p>If there is no latest tag shoppingclient, you can create with Running VS release mod on Docker run profile. Otherwise you can also make it manually with docker build commands.</p><p>So once we see the latest shoppingclient docker image, we should tag the lastest one with dockerhub repo name;</p><pre>docker tag cc2 mehmetozkaya/swnzen</pre><p>As you can see that we give the name exactly same as with docker hub repository profile, otherwise it can’t match when push docker hub.</p><p>Let check now;</p><pre>docker images</pre><pre>REPOSITORY TAG IMAGE ID CREATED SIZE<br>shoppingclient latest cc2a573482cf 46 minutes ago 210MB<br>mehmetozkaya/swnzen latest cc2a573482cf 46 minutes ago 210MB</pre><p>see its tagged.<br><strong>Push Docker Hub</strong></p><p><strong>docker push mehmetozkaya/swnzen:latest</strong></p><pre>The push refers to repository [docker.io/mehmetozkaya/swnzen]<br> b57cdc9e8ec8: Pushed<br> d066a90a6a65: Pushed<br> 024230939f4e: Pushed<br> ea4124eb3c7e: Pushed<br> 8ed87ee178f4: Pushing [======================================&gt; ] 58.3MB/75.66MB<br> 0916aa79e133: Pushing [==================================&gt; ] 28.54MB/41.33MB<br> 87c8a1d8f54f: Pushing [====================&gt; ] 28.68MB/69.23MB</pre><p>It takes some time. When we push the image docker search for the tag name exist then push the image to the docker hub.</p><h4>See DockerHub</h4><p>As you can see that we can successfully push our image to docker hub.</p><p>mehmetozkaya/swnzen<br> Last pushed: 12 minutes ago</p><p>As you can see that we have pushed the our shopping.client image to the DockerHub successfully.</p><h3>Developing Shopping.API Microservice with MongoDb and Compose All Docker Containers</h3><p>In this section, we are going to develop Shopping.API Microservice with MongoDb and Compose All Docker Containers. <br>At the end of the section, we will have 3 microservices whichs are Shopping.Client — Shopping.API — MongoDb microservices.</p><p>We have developed Shopping.Client and now we are going to start with developing the Shopping.API project. <br>This API project will have Products data and performs CRUD operations with exposing api methods for consuming from clients.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*2WHIbwc-NX9uATWKmEaJGA.png" /></figure><p>Our API project will manage product records stored in a no-sql <strong>mongodb </strong>database as described in the picture. <br>First we will develop API project which using Mongo providers.<br>After that, we will pull mongodb docker image from docker hub and create connection with our API project.<br>And of course, we will refactor Shopping.Client MVC application, instead of using hard value Product list, Shopping Client will consume Products from API project.</p><h4>Create Asp.Net Core Web API Project For Shopping.API Microservice</h4><p>First, we are going to create;</p><p>Create Asp.Net Core Web API Project :<br> Right Click Solution — Add new Web API — NO HTTPS — name : Shopping.API</p><h4>Create Product Controller Class for Shopping.API Microservice</h4><p>We are going to create a new web api project For Shopping.API Microservice.</p><p>Go to Controller folder<br> Add ProductController class</p><pre>[ApiController]<br> [Route(“[controller]”)]<br> public class ProductController : ControllerBase<br> {<br> private readonly ILogger&lt;ProductController&gt; _logger;</pre><pre>public ProductController(ILogger&lt;ProductController&gt; logger)<br> {<br> _logger = logger;<br> }</pre><p>Run application<br> Change Run Profile to “Shopping.API” and Run the application<br> swagger and <br> <a href="https://localhost:5000/product">https://localhost:5000/product</a></p><h4>Consume Shopping.API from Shopping.Client Microservices with using HttpClientFactory</h4><p>We are going to Consume Shopping.API from Shopping.Client Microservices with using HttpClientFactory Object.</p><p>First, We should go to Shopping.Client Project.<br> Locate Startup.cs<br>We will use http client factory, Add register http client into Aspnet DI;</p><pre>public void ConfigureServices(IServiceCollection services)<br> {<br> services.AddHttpClient(“ShoppingAPIClient”, client =&gt;<br> {<br> client.BaseAddress = new Uri(“<a href="http://localhost:5000/">http://localhost:5000/</a>”); // Shopping.API url<br> });</pre><pre>services.AddControllersWithViews();<br> }</pre><p>After that we are going to Inject HttpClient into home controller in order to consume API project when retrieving products data.</p><pre>public class HomeController : Controller<br> { <br> private readonly HttpClient _httpClient;<br> private readonly ILogger&lt;HomeController&gt; _logger;</pre><pre>public HomeController(IHttpClientFactory httpClientFactory, ILogger&lt;HomeController&gt; logger)<br> {<br> _logger = logger ?? throw new ArgumentNullException(nameof(logger));</pre><pre>_httpClient = httpClientFactory.CreateClient(“ShoppingAPIClient”);<br> }</pre><p>After that, we are going to Implement Index Get method with calling /products api from shopping.api project.</p><pre>public async Task&lt;IActionResult&gt; Index()<br> {<br> var response = await _httpClient.GetAsync(“/product”);<br> var content = await response.Content.ReadAsStringAsync();<br> var productList = JsonConvert.DeserializeObject&lt;IEnumerable&lt;Product&gt;&gt;(content); — Install latest Newtonsoft software</pre><pre>return View(productList);<br> }</pre><h3>Setup Mongo Docker Database</h3><p>In this section we are going to Setup Mongo Docker Database. We are in here and finished to development of Shopping MVC and Shopping.API project. <br>Now its time to create a real database which is no-sql mongo db. After that we will create connection from API project.</p><h4>Setup Mongo Docker Database for Shopping.API Microservices</h4><p>First, We should go to DockerHub and find mongodb official image.</p><p>Setup mongodb docker<br>Go to docker hub mongo image</p><p>Examine documentations, ports and so on.<br> Pull the mongo docker image in your local docker.</p><p><strong>docker pull mongo</strong><br>Run and Start mongodb</p><p><strong>docker run -d -p 27017:27017 — name shopping-mongo mongo</strong></p><p>See that we are forwarding the same port — 27017.<br>For troubleshooting, we have same commands;</p><p><strong>docker logs -f shopping-mongo<br>docker exec -it shopping-mongo /bin/bash</strong></p><h3>Dockerize Microservices with Creating Multi-Container App using Docker Compose</h3><p>In this section we are going to Dockerize all Microservices with Creating Multi-Container App using Docker Compose.</p><p><strong>Docker Compose</strong> is a Docker tool that enables complex applications to be defined and run. With Docker Compose, you can make multiple container definitions in a single file, and run the application by raising all the requirements your application needs with a single command.<br>We have finished to developments of our microservices on local also test it. So before we deploy these microservices, we are going to create docker images and test on <strong>docker container environment</strong>.</p><p>You’ll learn how to manage more than one container and communicate between them when using <strong>Container Tools in Visual Studio</strong>. <br>Managing multiple containers requires container orchestration and requires an orchestrator such as <strong>Docker Compose, Kubernetes, or Service Fabric.</strong> <br>Here, we’ll use Docker Compose. Docker Compose is great for local debugging and testing in the project of the development cycle.</p><h4>Adding Docker-Compose File for Shopping Microservices Solution</h4><p>We are going to add Docker-Compose File for Shopping Microservices Solution. First, We should go Shopping.API project. As you know that we have already created DockerFile for Shopping.Client appliction before. <br>So now we need to add for Shopping.API. <br>But this time we also need orcestration with Client-API and mongodb, so thats why we need to create docker-compose with dockerfile.</p><p>In the Shopping.API project, <br> choose Add &gt; Container Orchestrator Support. <br> The Docker Support Options dialog appears.<br> Choose Docker Compose.<br> Choose Linux.</p><p>DockerFile and docker-compose created.</p><p>Visual Studio creates a docker-compose.yml and override file in the docker-compose node in the solution, <br> and that project shows in boldface font, which shows that it’s now the startup project.</p><p>See the file;</p><pre>docker-compose.yml</pre><pre>version: ‘3.4’</pre><pre>services:<br> shopping.api:<br> image: ${DOCKER_REGISTRY-}shoppingapi<br> build:<br> context: .<br> dockerfile: Shopping.API/Dockerfile</pre><pre>docker-compose.override.yml</pre><pre>version: ‘3.4’</pre><pre>services:<br> shopping.api:<br> environment:<br> — ASPNETCORE_ENVIRONMENT=Development<br> — ASPNETCORE_URLS=<a href="https://+:443;http://+:80">https://+:443;http://+:80</a><br> ports:<br> — “80”<br> — “443”<br> volumes:<br> — ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro</pre><p>Docker Compose file defines all the services to be deployed in docker environment. These services rely on either a DockerFile or an existing container image.<br> In our case we have also Shopping.Client application, we already created DockerFile but also we need to add into docker compose file.<br>For that purpose, you can add manually in these file like API projects but I would like to generate from Visual studio.</p><p>In the Shopping.Client project, <br> again right-click on the project node, <br> choose Add &gt; Container Orchestrator Support. <br> Choose Docker Compose, and then select the Linux.</p><p>Visual Studio makes changes to your docker compose YML file. Now both services are included.</p><p>As you can see that, we have created docker-compose and add 2 microservices configuration but 1 more left. we will add Mongodb database.</p><h4>Run multi-container application with Docker Compose</h4><p>We are going to Run multi-container application with Docker Compose.</p><p>We need to Run docker-compose</p><p>we have 2 options</p><p>1- Close all dockers and run with below command on that location;</p><p><strong>docker-compose -f docker-compose.yml -f docker-compose.override.yml up — build</strong></p><p>2- Run visual studio docker-compose run — actualy its also run the same command</p><p>choose option 2<br> Set a Startup Project for docker-compose file<br> Click Run button<br> See output window</p><pre>1&gt;docker-compose -f “C:\Users\ezozkme\source\repos\swnzen\swnzen\docker-compose.yml” -f “C:\Users\ezozkme\source\repos\swnzen\swnzen\docker-compose.override.yml” -p dockercompose12550600008089957795 — no-ansi config</pre><p>It Worked !!</p><p>Also right click — clean solution — it run docker-compose down command and stop docker ps</p><pre>docker-compose -f “C:\Users\ezozkme\source\repos\swnzen\swnzen\docker-compose.yml” -f “C:\Users\ezozkme\source\repos\swnzen\swnzen\docker-compose.override.yml” -f “C:\Users\ezozkme\source\repos\swnzen\swnzen\obj\Docker\docker-compose.vs.debug.partial.g.yml” -p dockercompose12550600008089957795 — no-ansi down — rmi local — remove-orphans</pre><p>See the generated images;</p><p><strong>docker ps</strong></p><pre>C:\Users\ezozkme&gt;docker ps<br>CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES<br>52b980ee1a0d shoppingclient:dev “tail -f /dev/null” 27 minutes ago Up 27 minutes 443/tcp, 0.0.0.0:8001-&gt;80/tcp shoppingclient<br>c784c0e0a7f5 shoppingapi:dev “tail -f /dev/null” 27 minutes ago Up 27 minutes 443/tcp, 0.0.0.0:8000-&gt;80/tcp shoppingapi<br>c86d72bf1ebb mongo “docker-entrypoint.s…” 27 minutes ago Up 27 minutes 0.0.0.0:27017-&gt;27017/tcp shoppingdb</pre><h4>Test application</h4><p><a href="http://localhost:8000/swagger/index.html">http://localhost:8000/swagger/index.html</a> <br><a href="http://localhost:8000/product">http://localhost:8000/product</a></p><p>As you can see that, we have finally dockerize all microservices and running on our local machines. Next article we are going to talk about Kubernetes.</p><p>For the next articles -&gt;</p><ul><li><a href="https://mehmetozkaya.medium.com/deploying-microservices-on-kubernetes-35296d369fdb"><strong>2- Deploying Microservices on Kubernetes</strong></a></li></ul><h3>References</h3><p><a href="https://docs.docker.com/get-started/overview/">https://docs.docker.com/get-started/overview/</a><br><a href="https://docs.docker.com/get-started/">https://docs.docker.com/get-started/</a><br><a href="https://medium.com/batech/docker-nedir-docker-kavramlar%C4%B1-avantajlar%C4%B1-901b37742ee0">https://medium.com/batech/docker-nedir-docker-kavramlar%C4%B1-avantajlar%C4%B1-901b37742ee0</a><br><a href="https://www.mediaclick.com.tr/tr/blog/docker-nedir-docker-ne-ise-yarar">https://www.mediaclick.com.tr/tr/blog/docker-nedir-docker-ne-ise-yarar</a><br><a href="https://www.docker.com/resources/what-container">https://www.docker.com/resources/what-containe</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=793d60f48d31" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/preparing-multi-container-microservices-applications-for-deployment-793d60f48d31">Preparing Multi-Container Microservices Applications for Deployment</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Microservices Using ASP.NET, PostgreSQL, Dapper Micro-ORM and Docker Container]]></title>
            <link>https://medium.com/aspnetrun/microservices-using-asp-net-postgresql-dapper-micro-orm-and-docker-container-e9d61a408d2a?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/e9d61a408d2a</guid>
            <category><![CDATA[aspnetcore]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[dapper]]></category>
            <category><![CDATA[pgadmin]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Thu, 01 Apr 2021 15:19:37 GMT</pubDate>
            <atom:updated>2021-04-01T15:14:24.457Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Building Discount Microservice on .Net platforms which used Asp.Net Web API, Docker, PostgreSQL, Dapper Micro-ORM and Swagger. Test microservice with using Postman.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v6FKERN4DtRpYfEHHUpP3g.png" /></figure><h3>Introduction</h3><p>In this article we will show <strong>how to perform Discount Microservices operations </strong>on ASP.NET Core Web application using <strong>PostgreSQL, Dapper micro-orm, Docker Container </strong>and <strong>Swagger</strong>.</p><p>By the end of the article, we will have a Web API which implemented CRUD operations over <strong>Coupon </strong>objects. These objects will be store in <strong>PostgreSQL database </strong>and retrieved data with using Dapper micro-orm tool.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kkUW7hzBPaQUtv-9.png" /></figure><p>Developing Discount microservice which includes;</p><ul><li><strong>ASP.NET Web API</strong> application</li><li><strong>REST API</strong> principles, <strong>CRUD </strong>operations</li><li><strong>PostgreSQL database </strong>connection and containerization</li><li><strong>Repository Pattern</strong> implementation</li><li>Using <strong>Dapper for micro-orm</strong> implementation to simplify data access and ensure high performance</li><li><strong>PostgreSQL </strong>database connection and <strong>containerization</strong></li></ul><p>We will Analysis and Architecting of Discount Microservices, applying N-Layer architecture. Containerize Discount Microservices with PostgreSQL database using<strong> Docker Compose.</strong></p><p>In the upcoming sections :</p><ul><li>ASP.NET Grpc Server application</li><li>Build a Highly Performant inter-service <strong>gRPC Communication </strong>with Discount Microservice</li><li>Exposing Grpc Services with creating Protobuf messages</li></ul><p>So in this section, we are going to Develop<strong> Discount.API Microservices </strong>with <strong>PostgreSQL</strong>.</p><h3>Background</h3><p>You can follow the previous article which explains overall microservice architecture of this example.</p><p><a href="https://medium.com/aspnetrun/microservices-architecture-on-net-3b4865eea03f"><strong>Check for the previous article which explained overall microservice architecture of this repository.</strong></a></p><p>We will focus on Discount microservice from that overall e-commerce microservice architecture.</p><h3>Step by Step Development w/ Udemy Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/0*0qCaRsT_7M-wOYGR.png" /></figure><p><a href="https://www.udemy.com/course/microservices-architecture-and-implementation-on-dotnet/?couponCode=FA24745CC57592AB612A"><strong>Get Udemy Course with discounted — Microservices Architecture and Implementation on .NET</strong></a><strong>.</strong></p><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-aspnetcore-microservices"><strong>Get the Source Code from AspnetRun Microservices Github</strong> </a>— Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.</p><h3>Prerequisites</h3><ul><li>Install the .NET Core 5 or above SDK</li><li>Install Visual Studio 2019 v16.x or above</li><li>Docker Desktop</li></ul><h3>PostgreSQL in Discount Microservices</h3><p><strong>PostgreSQL </strong>is an open source and completely free object relational database system with powerful features and advantages. Taking advantage of the security, storability and scalability features of the SQL language, PostgreSQL is also used as a database manager in many areas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*06fGtcmVuVZQqytJSaxkiw.png" /></figure><p><strong>PostgreSQL </strong>is one of the most accepted database management systems in the industry today. Because it offers users the advantages of successful data architecture,<strong> data accuracy</strong>, powerful feature set, and open source. PostgreSQL is supported by many major operating systems such as UNIX, Linux, MacOS and Windows.</p><p>In terms of performance, PostgreSQL has been found to be more successful compared to other commercial or open source databases. <br>In the face of some database systems, it is fast in some areas, but slow in others.</p><p>Compared to PostgreSQL, MySQL and databases in the same class, it is slower in <strong>INSERT / UPDATE transactions</strong> because it works transaction-based. <br>In some cases, PostgreSQL and MySQL have significant advantages in terms of features, reliability and flexibility. Aside from the highlights of PostgreSQL, this service is offered completely free of charge by open source developers.</p><p>For Discount microservices, we are going to store discount coupon data information into PostgreSQL database in a Coupon table.</p><h3>Analysis &amp; Design</h3><p>This project will be the REST APIs which basically perform CRUD operations on Basket databases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/0*-NLqW2w0LGiA2nBv.png" /></figure><p>We should define our Discount use case analysis. In this part we will create Coupon entities. Our main use case are Get Coupon by ProductName, able to add new item to Discount. Also performed CRUD operations on Coupom entity.</p><p>Our main use cases;</p><ul><li>Get Coupon with productname</li><li>Update Coupon</li><li>Delete Coupon</li><li>Checkout Coupon</li></ul><h3>Architecture of Discount microservices</h3><p>We are going to use traditional <strong>N-Layer architecture</strong>. Layered architecture basically consists of 3 layers. These 3 layers are generally the ones that should be in every project. You can create a layered structure with more than these 3 layers, which is called multi-layered architecture.</p><p><strong>Data Access Layer:</strong> Only database operations are performed on this layer.<br>The task of this layer is to add, delete, update and extract data from the database. There is no other operation in this layer other than these operations.</p><p><strong>Business Layer: </strong>We implement business logics on this layer. This layer is will process the data taken by Data Access into the project.<br>We do not use the Data Access layer directly in our applications.<br>The data coming from the user first goes to the Business layer, from there it is processed and transferred to the Data Access layer.</p><p><strong>Presentation Layer:</strong> This layer is the layer on which the user interacts.<br>It could be in Windows form, on the Web, or in a Console application.<br>The main purpose here is to show the data to the user and to transmit the data from the user to the Business Layer and Data Access.</p><h3>Simple Data Driven CRUD Microservice</h3><p>Discount.API microservices will be simple crud implementation on Coupom data on PostgreSQL databases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FsXfLguciWV1eW38.png" /></figure><p>You can apply any internal design pattern per microservices. Since we have 1 project, so we are going to separate this layers with using folders into the project.<br>But for the ordering microservices we also separate layers with projects with using clean arch and CQRS implementation.</p><p>So we don’t need to separate layers in different assemblies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7pl37oUbzkJi7Ogj.png" /></figure><p>If we look at the project structure, we are planning to create this layers,</p><ul><li>Domain Layer — Contains business rules and logic.</li><li>Application Layer — Expose endpoints and validations. API layer will be Controller classes.</li><li>Infrastructure Layer — responsible by persistence operations.</li></ul><h3>Project Folder Structure</h3><ul><li>Entities — PostgreSQL entity</li><li>Data — PostgreSQL data context</li><li>Repositories — PostgreSQL repos</li><li>Controllers — api classes</li></ul><h3>Setup PostgreSQL Docker Database for Discount.API Microservices</h3><p>We are going to Setup PostgreSQL Docker Database for Discount.API Microservices.</p><p>First, We should go to <strong>DockerHub </strong>and find Postgres official image.</p><p>Postgres DockerHub;<br> <a href="https://hub.docker.com/_/postgres">https://hub.docker.com/_/postgres</a></p><p>This time we are going to write directly on docker-compose file.</p><p><strong>docker-compose.yml</strong></p><pre>version: ‘3.4’</pre><pre>services:<br> catalogdb:<br> image: mongo</pre><pre>basketdb:<br> image: redis:alpine</pre><pre>discountdb: — — ADDED<br> image: postgres<br>…</pre><pre>volumes:<br> portainer_data:<br> postgres_data: — — ADDED</pre><p>— <br>Check environment variables - <a href="https://hub.docker.com/_/postgres">https://hub.docker.com/_/postgres</a><br>pgdata username etc..<br> —</p><p><strong>docker-compose.override.yml</strong></p><pre>discountdb:<br> container_name: discountdb<br> environment:<br> — POSTGRES_USER=admin<br> — POSTGRES_PASSWORD=admin1234<br> — POSTGRES_DB=DiscountDb<br> restart: always<br> ports:<br> — “5432:5432”<br> volumes:<br> — postgres_data:/var/lib/postgresql/data/</pre><p>We have added postgresql database in our dc file. We have not started yet, because we also need to manage portal of postresql.</p><h3>Setup pgAdmin Management Portal for PostgreSQL Database for Discount.API Microservices</h3><p>We are going to Setup pgAdmin Manage Portal for PostgreSQL Database for Discount.API Microservices.</p><p>pgAdmin is one of the popular and feature rich Open Source administration and development platform for PostgreSQL.<br>We will use pgAdmin for managing PostgreSQL Discount database creation and add same records into that table for Discount microservices.</p><p>We should go to DockerHub and find pgAdmin image. <br> — now its time to add manage portal of postresql whic is pgadmin.</p><p>Check env variables<br> <a href="https://www.pgadmin.org/docs/pgadmin4/latest/container_deployment.html">https://www.pgadmin.org/docs/pgadmin4/latest/container_deployment.html</a></p><p>Run a simple container over port 80, setting some configuration options:</p><pre>docker pull dpage/pgadmin4<br> docker run -p 80:80 \<br> -e ‘<a href="/service/mailto:PGADMIN_DEFAULT_EMAIL=user@domain.com">PGADMIN_DEFAULT_EMAIL=user@domain.com</a>’ \<br> -e ‘PGADMIN_DEFAULT_PASSWORD=SuperSecret’ \<br> -e ‘PGADMIN_CONFIG_ENHANCED_COOKIE_PROTECTION=True’ \<br> -e ‘PGADMIN_CONFIG_LOGIN_BANNER=”Authorised users only!”’ \<br> -e ‘PGADMIN_CONFIG_CONSOLE_LOG_LEVEL=10’ \<br> -d dpage/pgadmin4</pre><p>After getting these informations, its time to add this into our docker-compose file with postgreSQL<br>write dc</p><p><strong>docker-compose.yml</strong></p><pre>version: ‘3.4’<br>…</pre><pre>discountdb:<br> image: postgres</pre><pre>pgadmin: — — — — — — — — — — ADDED<br> image: dpage/pgadmin4</pre><pre>volumes:<br> portainer_data:<br> postgres_data:<br> pgadmin_data: — — — — — — — ADDED</pre><p><strong>docker-compose.override.yml</strong></p><pre>pgadmin:<br> container_name: pgadmin<br> environment:<br> — <a href="/service/mailto:PGADMIN_DEFAULT_EMAIL=admin@aspnetrun.com">PGADMIN_DEFAULT_EMAIL=admin@aspnetrun.com</a><br> — PGADMIN_DEFAULT_PASSWORD=admin1234<br> restart: always<br> ports:<br> — “5050:80”<br> volumes:<br> — pgadmin_data:/root/.pgadmin</pre><p>So now we had postgres database and also pgadmin management tool of postgres.</p><p>RUN with below command on that location;</p><pre>docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d<br> docker-compose -f docker-compose.yml -f docker-compose.override.yml down</pre><p>Check portainer containers<br> localhost:9000</p><p>Check pgAdmin</p><p>localhost:5050</p><p>Login system with <br> — <a href="/service/mailto:PGADMIN_DEFAULT_EMAIL=admin@aspnetrun.com">PGADMIN_DEFAULT_EMAIL=admin@aspnetrun.com</a><br> — PGADMIN_DEFAULT_PASSWORD=admin1234</p><p>After taht we can manage our postgresql database with adding new server and connect to our discountdb database.</p><pre>Add New Server<br> General<br> name — DiscountServer</pre><pre>Connection<br> hostname — discountdb — this should be the docker container name of postresql<br> username — admin<br> password — admin1234<br> <br> See dc-override of postresql<br> discountdb:<br> container_name: discountdb<br> environment:<br> — POSTGRES_USER=admin<br> — POSTGRES_PASSWORD=admin1234</pre><p>See “DiscountDb” database under servers.<br>You can create tables under schemas.</p><p>As you can see that, we have added <strong>pgAdmin management portal</strong> of postgresql database in our dc file. And run our docker-compose file and see that we can manage our postgresql database in pgAdmin portal like adding new server and see our Discount db.</p><h3>Create Coupon Table in the DiscountDb of PostgreSQL Database with pgAdmin Management Portal</h3><p>We are going to Create Coupon Table in the DiscountDb of PostgreSQL Database with pgAdmin Management.</p><p>See “DiscountDb” database under servers.<br>You can create tables under schemas.</p><p>Open in pgadmin<br>Tools — Query Tool</p><p>Create Table with below script</p><pre>Tools — Query Tool</pre><pre>CREATE TABLE Coupon(<br> ID SERIAL PRIMARY KEY NOT NULL,<br> ProductName VARCHAR(24) NOT NULL,<br> Description TEXT,<br> Amount INT<br> );</pre><p>Now our Postresql database and table is ready and loaded with predefined data.</p><h3>Developing Discount.API Microservices Creating Entities</h3><p>We should create entity objects for redis definitions.</p><p>Create “<strong>Entities</strong>” folder</p><p>Entities<br> Add Coupon class</p><pre>namespace Discount.API.Entities<br> {<br> public class Coupon<br> {<br> public int Id { get; set; }<br> public string ProductName { get; set; }<br> public string Description { get; set; }<br> public int Amount { get; set; }<br> }<br> }</pre><h3>Developing Repository Pattern Connect PostgreSQL use Dapper on Discount.API Microservice</h3><p>We are going to Developing Repository Pattern Connect PostgreSQL use Dapper on Discount.API Microservice.<br>We will encapsulate data related objects in the repository classes by this way there is an abstraction with the repository layer.</p><p>First of all, we should install required nuget packages</p><p>Install Nuget Package<br> Open Package Manager Console — PMC<br> Select Project — Discount.API</p><pre> Run Command:<br> Install-Package Npgsql<br> Install-Package Dapper</pre><p>If required Update Packages<br> Update-Package -ProjectName Discount.API</p><h4><strong>Dapper</strong></h4><p><strong>Dapper is a Micro-ORM</strong> (Object Relationship Mapper) tool developed by the Stack Overflow team as LightWeight, published open source on Github. It has support for most databases (SQL Server, MySQL, PosgreSQL so on).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*oqA6RK59ooDsjg5eN9FvVQ.png" /></figure><p>In Ado.Net, we perform our queries or procedures using SqlDataReader, SqlCommand etc. objects. Dapper takes the burden of writing these objects from us. <br>We can do our filtering by using generic and extension methods. By writing less code, we can execute our queries in a short time and convert them to the type we want. The biggest feature is it works very fast speed and very close to Ado.Net speed.</p><h3>Create Business Layer</h3><p>After that, We should create Create “<strong>Repositories</strong>” and <strong>IDiscountRepository </strong>interface.</p><p><strong>Repositories </strong>Folder</p><p>Add Interface — <strong>IDiscountRepository</strong></p><pre>namespace Discount.API.Repositories.Interfaces<br> { <br> public interface IDiscountRepository<br> {<br> Task&lt;Coupon&gt; GetDiscount(string productName);</pre><pre>Task&lt;bool&gt; CreateDiscount(Coupon coupon);<br> Task&lt;bool&gt; UpdateDiscount(Coupon coupon);<br> Task&lt;bool&gt; DeleteDiscount(string productName);<br> }<br> }</pre><p>After that, we can implement this interface with using NpgsqlConnection objects.</p><p>Create Repository — <strong>DiscountRepository</strong></p><pre>namespace Basket.API.Repositories<br>{<br> public class DiscountRepository : IDiscountRepository<br> {<br> private readonly IConfiguration _configuration;</pre><pre>public DiscountRepository(IConfiguration configuration)<br> {<br> _configuration = configuration ?? throw new ArgumentNullException(nameof(configuration));<br> }</pre><pre>public async Task&lt;Coupon&gt; GetDiscount(string productName)<br> {<br> using var connection = new NpgsqlConnection(_configuration.GetValue&lt;string&gt;(“DatabaseSettings:ConnectionString”));<br> <br> var coupon = await connection.QueryFirstOrDefaultAsync&lt;Coupon&gt;<br> (“SELECT * FROM Coupon WHERE ProductName = <a href="http://twitter.com/ProductName">@ProductName</a>”, new { ProductName = productName });<br> <br> if (coupon == null)<br> return new Coupon { ProductName = “No Discount”, Amount = 0, Description = “No Discount Desc” };<br> return coupon;<br> }</pre><pre>public async Task&lt;bool&gt; CreateDiscount(Coupon coupon)<br> {<br> using var connection = new NpgsqlConnection(_configuration.GetValue&lt;string&gt;(“DatabaseSettings:ConnectionString”));</pre><pre>var affected =<br> await connection.ExecuteAsync<br> (“INSERT INTO Coupon (ProductName, Description, Amount) VALUES (<a href="http://twitter.com/ProductName">@ProductName</a>, <a href="http://twitter.com/Description">@Description</a>, <a href="http://twitter.com/Amount">@Amount</a>)”,<br> new { ProductName = coupon.ProductName, Description = coupon.Description, Amount = coupon.Amount });</pre><pre>if (affected == 0)<br> return false;</pre><pre>return true;<br> }</pre><pre>public async Task&lt;bool&gt; UpdateDiscount(Coupon coupon)<br> {<br> using var connection = new NpgsqlConnection(_configuration.GetValue&lt;string&gt;(“DatabaseSettings:ConnectionString”));</pre><pre>var affected = await connection.ExecuteAsync<br> (“UPDATE Coupon SET ProductName=<a href="http://twitter.com/ProductName">@ProductName</a>, Description = <a href="http://twitter.com/Description">@Description</a>, Amount = <a href="http://twitter.com/Amount">@Amount</a> WHERE Id = <a href="http://twitter.com/Id">@Id</a>”,<br> new { ProductName = coupon.ProductName, Description = coupon.Description, Amount = coupon.Amount, Id = coupon.Id });</pre><pre>if (affected == 0)<br> return false;</pre><pre>return true;<br> }</pre><pre>public async Task&lt;bool&gt; DeleteDiscount(string productName)<br> {<br> using var connection = new NpgsqlConnection(_configuration.GetValue&lt;string&gt;(“DatabaseSettings:ConnectionString”));</pre><pre>var affected = await connection.ExecuteAsync(“DELETE FROM Coupon WHERE ProductName = <a href="http://twitter.com/ProductName">@ProductName</a>”,<br> new { ProductName = productName });</pre><pre>if (affected == 0)<br> return false;</pre><pre>return true;<br> }<br> }<br>}</pre><p>Basically we are using “<strong>NpgsqlConnection</strong>” object as a <strong>postgresql </strong>connection. This will create connection for us with using connection string. After that with using “connection” object and methods, we are using <strong>Dapper </strong>sql commands. This will be the fastest way to use<strong> micro-orm</strong> to <strong>retrieve data</strong> from databases.</p><h3>Create DiscountController Class for Discount.API Microservice</h3><p>First of all, We should create DiscountController into Controller class.</p><p>Controller + Startup.cs</p><pre>[ApiController]<br> [Route(“api/v1/[controller]”)]<br> public class DiscountController : ControllerBase<br> {<br> private readonly IDiscountRepository _repository;</pre><pre>public DiscountController(IDiscountRepository repository)<br> {<br> _repository = repository ?? throw new ArgumentNullException(nameof(repository));<br> }<br> <br> [HttpGet(“{productName}”, Name = “GetDiscount”)]<br> [ProducesResponseType(typeof(Coupon), (int)HttpStatusCode.OK)]<br> public async Task&lt;ActionResult&lt;Coupon&gt;&gt; GetDiscount(string productName)<br> {<br> var discount = await _repository.GetDiscount(productName);<br> return Ok(discount);<br> }</pre><pre>[HttpPost]<br> [ProducesResponseType(typeof(Coupon), (int)HttpStatusCode.OK)]<br> public async Task&lt;ActionResult&lt;Coupon&gt;&gt; CreateDiscount([FromBody] Coupon coupon)<br> {<br> await _repository.CreateDiscount(coupon);<br> return CreatedAtRoute(“GetDiscount”, new { productName = coupon.ProductName }, coupon);<br> }</pre><pre>[HttpPut]<br> [ProducesResponseType(typeof(Coupon), (int)HttpStatusCode.OK)]<br> public async Task&lt;ActionResult&lt;Coupon&gt;&gt; UpdateDiscount([FromBody] Coupon coupon)<br> {<br> return Ok(await _repository.UpdateDiscount(coupon));<br> }</pre><pre>[HttpDelete(“{productName}”, Name = “DeleteDiscount”)]<br> [ProducesResponseType(typeof(void), (int)HttpStatusCode.OK)]<br> public async Task&lt;ActionResult&lt;bool&gt;&gt; DeleteDiscount(string productName)<br> {<br> return Ok(await _repository.DeleteDiscount(productName));<br> }<br> }</pre><p>we have injected <strong>IDiscountRepository </strong>object and use this object when exposing apis.<br>You can see that we have exposed crud api methods over the DiscountController.</p><p>Also you can see the anotations. <br> <strong>ProducesResponseType </strong>— provide to restrictions on api methods. produce only including types of objects.<br> <strong>Route </strong>— provide to define custom routes. Mostly using if you have one more the same http method.<br><strong> httpget/put/post </strong>— send http methods.</p><h3>Test and Run Discount Microservice</h3><p>We are going to Test and Run Discount Microservice. Now the <strong>Discount microservice</strong> Web API application ready to run.</p><p>Before running the application, <strong>configure the debug profile</strong>;</p><p><em>Right Click the project File and Select to Debug section.</em></p><p>Change Launch browser to <strong>swagger</strong></p><p>Change the App URL to <a href="http://localhost:5000/"><strong>http://localhost:500</strong></a><strong>2</strong></p><p>Hit <strong>F5</strong> on <strong>Discount.API</strong> project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/0*h6xKkCCrORnfhlZs.png" /></figure><p>Exposed the Product APIs in our Catalog Microservices, you can <strong>test</strong> it over the <strong>Swagger GUI</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/0*HkffpXrFfcipwA0A.png" /></figure><p>You can also test it over the <strong>Postman</strong> as below way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/573/0*DjWqPXse1WFMsbty.png" /></figure><p><a href="http://localhost:5002/swagger/index.html">http://localhost:5002/swagger/index.html</a></p><p>See CRUD operations on Swagger<br> — -<br> Set productName</p><p>productName — IPhone X<br> productName — Samsung 10</p><pre>Get Success :<br> {<br> “id”: 1,<br> “productName”: “IPhone X”,<br> “description”: “IPhone Discount”,<br> “value”: 150<br> }<br> <br> Create<br> { <br> “productName”: “Huawei Plus”,<br> “description”: “test new product”,<br> “value”: 550<br> }</pre><pre>Update<br> { <br> “productName”: “Huawei Plus”,<br> “description”: “test update”,<br> “value”: 200<br> }</pre><pre>Delete<br> productName — IPhone X<br> productName — Samsung 10<br> productName — Huawei Plus</pre><p>As you can see that, we have test our Discount microservices is working fine.</p><p><a href="https://www.udemy.com/course/microservices-architecture-and-implementation-on-dotnet/?couponCode=FA24745CC57592AB612A"><strong>Get Udemy Course with discounted — Microservices Architecture and Implementation on .NET</strong></a><strong>.</strong></p><p><a href="https://github.com/aspnetrun/run-aspnetcore-microservices"><strong>Get the Source Code from AspnetRun Microservices Github</strong></a></p><h3>Follow Series of Microservices Articles</h3><p>This is the introduction of the series. This will be the series of articles. You can follow the series with below links.</p><ul><li><a href="https://medium.com/aspnetrun/microservices-architecture-on-net-3b4865eea03f"><strong>0- Microservices Architecture on .NET with applying CQRS, Clean Architecture and Event-Driven Communication</strong></a></li><li><a href="https://medium.com/aspnetrun/build-catalog-microservice-using-asp-net-core-mongodb-and-docker-container-88b8fd4d5040"><strong>1- Microservice Using ASP.NET Core, MongoDB and Docker Container</strong></a></li><li><a href="https://medium.com/aspnetrun/using-redis-with-asp-net-core-and-docker-container-for-basket-microservices-715ff739186e"><strong>2- Using Redis with ASP.NET Core, and Docker Container for Basket Microservices</strong></a></li><li><a href="https://mehmetozkaya.medium.com/microservices-using-asp-net-postgresql-dapper-micro-orm-and-docker-container-e9d61a408d2a"><strong>3- Using PostgreSQL and Dapper with ASP.NET and Docker Container for Discount Microservices</strong></a></li><li><a href="https://medium.com/aspnetrun/building-ocelot-api-gateway-microservices-with-asp-net-core-and-docker-container-13f96026e86c"><strong>4- Building Ocelot API Gateway Microservices with ASP.NET Core and Docker Container</strong></a></li><li><a href="https://medium.com/aspnetrun/microservices-event-driven-architecture-with-rabbitmq-and-docker-container-on-net-968d73052cbb"><strong>5- Microservices Event Driven Architecture with RabbitMQ and Docker Container on .NET</strong></a></li><li><a href="https://medium.com/aspnetrun/cqrs-and-event-sourcing-in-event-driven-architecture-of-ordering-microservices-fb67dc44da7a"><strong>6- CQRS and Event Sourcing in Event Driven Architecture of Ordering Microservices</strong></a></li><li><strong>7- Microservices Cross-Cutting Concerns with Distributed Logging and Microservices Resilience</strong></li><li><a href="https://medium.com/aspnetrun/securing-microservices-with-identityserver4-with-oauth2-and-openid-connect-fronted-by-ocelot-api-49ea44a0cf9e"><strong>8-</strong> <strong>Securing Microservices with IdentityServer4 with OAuth2 and OpenID Connect fronted by Ocelot API Gateway</strong></a></li><li><a href="https://medium.com/aspnetrun/using-grpc-in-microservices-for-building-a-high-performance-interservice-communication-with-net-5-11f3e5fa0e9d"><strong>9- Using gRPC in Microservices for Building a high-performance Interservice Communication with .Net 5</strong></a></li><li><a href="https://medium.com/aspnetrun/deploying-net-microservices-to-azure-kubernetes-services-aks-and-automating-with-azure-devops-c50bdd51b702"><strong>10-Deploying .Net Microservices to Azure Kubernetes Services(AKS) and Automating with Azure DevOps</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9d61a408d2a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/microservices-using-asp-net-postgresql-dapper-micro-orm-and-docker-container-e9d61a408d2a">Microservices Using ASP.NET, PostgreSQL, Dapper Micro-ORM and Docker Container</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying .Net Microservices to Azure Kubernetes Services(AKS) and Automating with Azure DevOps]]></title>
            <link>https://medium.com/aspnetrun/deploying-net-microservices-to-azure-kubernetes-services-aks-and-automating-with-azure-devops-c50bdd51b702?source=rss----7897df73dd1c---4</link>
            <guid isPermaLink="false">https://medium.com/p/c50bdd51b702</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[azure-container-registry]]></category>
            <category><![CDATA[azure-pipelines]]></category>
            <category><![CDATA[azure-devops]]></category>
            <category><![CDATA[azure-kubernetes-service]]></category>
            <dc:creator><![CDATA[Mehmet Ozkaya]]></dc:creator>
            <pubDate>Wed, 13 Jan 2021 13:16:52 GMT</pubDate>
            <atom:updated>2023-04-09T12:22:08.876Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deploying .Net Microservices to Azure Kubernetes Services(AKS) and Automating with Azure DevOps</h3><p>In this article, we’re going to learn how to Deploying .Net Microservices into <strong>Kubernetes</strong>, and moving deployments to the cloud <strong>Azure Kubernetes Services (AKS)</strong> with using <strong>Azure Container Registry (ACR)</strong> and last section is we will learn how to Automating Deployments with <strong>Azure DevOps</strong> and GitHub.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*Tmyb2CayTfVUy0GW0wF8lw.png" /></figure><p>The image above, you can find the steps of our article structure.<br>We’re going to containerize our microservices on docker environment, and push these images to the <strong>DockerHub </strong>and deploy microservices on <strong>Kubernetes</strong>. As the same setup, we are going to shifting to the cloud for deploying <strong>Azure Kubernetes Services (AKS)</strong> with pushing images to <strong>Azure Container Registry (ACR).</strong></p><p>Also we will cover additional topics that;</p><ul><li>Docker compose microservices</li><li>K8s components</li><li>Zero-downtime deployments</li><li>Using azure resources like ACR, AKS</li><li>Automate whole deployment process with writing custom pipelines with Azure DevOps and so on..</li></ul><h3>Step by Step Development w/ Course</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*4o4HWhA_zmoJxOkJH8ZeWQ.png" /></figure><p><a href="https://www.udemy.com/course/deploying-net-microservices-with-k8s-aks-and-azure-devops/?couponCode=APRI23"><strong>I have just published a new course — Deploying .Net Microservices with K8s, AKS and Azure DevOps.</strong></a></p><p>In this course, we’re going to learn how to <strong>Deploying .Net Microservices</strong> into <strong>Kubernetes</strong>, and moving deployments to the cloud <strong>Azure kubernetes services (AKS)</strong> with using <strong>Azure Container Registry(ACR)</strong> and last section is we will learn how to Automating Deployments with <strong>CI/CD pipeline of Azure DevOps</strong> and GitHub.</p><h3>Overall Picture</h3><p>See the overall picture. You can see that we will have 3 microservices which we are going to develop and deploy together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*2WHIbwc-NX9uATWKmEaJGA.png" /></figure><h4>Shopping MVC Client Application</h4><p>First of all, we are going to develop Shopping MVC Client Application For Consuming Api Resource which will be the Shopping.Client Asp.Net MVC Web Project. But we will start with developing this project as a standalone Web application which includes own data inside it. And we will add container support with <strong>DockerFile</strong>, push docker images to Docker hub and see the deployment options like “Azure Web App for Container” resources for 1 web application.</p><h4>Shopping API Application</h4><p>After that we are going to develop Shopping.API Microservice with MongoDb and Compose All Docker Containers. <br>This API project will have Products data and performs CRUD operations with exposing api methods for consuming from Shopping Client project.<br>We will containerize API application with creating dockerfile and push images to Azure Container Registry.</p><h4>Mongo Db</h4><p>Our API project will manage product records stored in a no-sql mongodb database as described in the picture. <br>we will pull <strong>mongodb </strong>docker image from docker hub and create connection with our API project.</p><p>At the end of the section, we will have 3 microservices whichs are <strong>Shopping.Client — Shopping.API — MongoDb</strong> microservices.</p><p>As you can see that, we have</p><ul><li>Created docker images,</li><li>Compose docker containers and tested them,</li><li>Deploy these docker container images on local Kubernetes clusters,</li><li>Push our image to ACR,</li><li>Shifting deployment to the cloud Azure Kubernetes Services (AKS),</li><li>Update microservices with <strong>zero-downtime</strong> deployments.</li></ul><h3>Deploy to Azure Kubernetes Services (AKS) through CI/CD Azure Pipelines</h3><p>And the last step, we are focusing on automation deployments with creating <strong>CI/CD pipelines</strong> on <strong>Azure Devops</strong> tool. We will develop separate microservices deployment pipeline yamls with using <strong>Azure Pipelines</strong>.<br>When we push code to <strong>Github</strong>, microservices pipeline triggers, build docker images and push the ACR, deploy to Azure Kubernetes services with zero-downtime deployments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RScqJLabL17D2DjSo3kwVQ.png" /></figure><blockquote>By the end of this articles, you’ll learn how to deploy your multi-container microservices applications with automating all deployment process seperately.</blockquote><h3>Background</h3><p>This is the introduction of the series. This will be the series of articles. You can follow the series with below links.</p><ul><li><a href="https://mehmetozkaya.medium.com/deploying-net-microservices-to-azure-kubernetes-services-aks-and-automating-with-azure-devops-c50bdd51b702"><strong>0- Deploying .Net Microservices</strong></a></li><li><a href="https://mehmetozkaya.medium.com/preparing-multi-container-microservices-applications-for-deployment-793d60f48d31"><strong>1- Preparing Multi-Container Microservices Applications for Deployment</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploying-microservices-on-kubernetes-35296d369fdb"><strong>2- Deploying Microservices on Kubernetes</strong></a></li><li><a href="https://mehmetozkaya.medium.com/deploy-microservices-into-cloud-azure-kubernetes-service-aks-with-using-azure-container-registry-b661698610b1"><strong>3- Deploy Microservices into Cloud Azure Kubernetes Service (AKS) with using Azure Container Registry (ACR)</strong></a></li><li><a href="https://mehmetozkaya.medium.com/automate-deployments-with-ci-cd-pipelines-on-azure-devops-13a83d3dd67a"><strong>4- Automate Deployments with CI/CD pipelines on Azure Devops</strong></a></li></ul><h3>Prerequisites</h3><ul><li>Visual Studio 2019 and VS Code</li><li>Docker Desktop and Docker Account for pushing images Docker Hub</li><li>Git on Local and Github Account for granting our devops pipelines triggering when push the code.</li><li>Azure Free Subscription for creating all azure resources like ACR, Web app for Containers, AKS and so on..</li><li>Azure Devops Account for ci/cd devops pipelines.</li></ul><h3>Source Code</h3><p><a href="https://github.com/aspnetrun/run-devops"><strong>Get the Source Code from AspnetRun Microservices Github</strong></a> — Clone or fork this repository, if you like don’t forget the star :) If you find or ask anything you can directly open issue on repository.</p><h3>Basics — What is Docker and Container ?</h3><p><strong>Docker </strong>is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*TkGq6r73nrgyN2wU5Z-ULA.png" /></figure><p>Advantages of Docker’s methodologies for <strong>shipping, testing, </strong>and <strong>deploying </strong>code quickly, you can significantly reduce the delay between writing code and running it in production. Docker provides for automating the deployment of applications as portable, self-sufficient containers that can run on the cloud or on-premises. Docker containers can run anywhere, in your local computer to the cloud. Docker image containers can run natively on Linux and Windows.</p><h4>Docker Container</h4><p>A container is a standard unit of software that packages up code and <strong>all its dependencies</strong> so the application runs quickly and reliably from one computing environment to another. A Docker container image is a lightweight, standalone, <strong>executable package</strong> of software that includes everything needed to run an application.</p><h4>Docker containers, images, and registries</h4><p>When using Docker, a developer develops an application and packages it with its dependencies into a container image. An image is a static representation of the application with its configuration and dependencies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zFXalFpmMbNc6lLJ4yGvBg.png" /></figure><p>In order to run the application, the application’s image is instantiated to create a container, which will be running on the Docker host. <br>Containers can be tested in a development local machines.</p><p>As you can see the images above, how docker components related each other.<br>Developer creates container in local and push the images the Docker Registry. <br>Or its possible that developer download existing image from registry and create container from image in local environment.</p><p>Developers should store images in a registry, which is a library of images and is needed when deploying to production orchestrators. Docker images are stores a public registry via Docker Hub; other vendors provide registries for different collections of images, including Azure Container Registry. <br>Alternatively, enterprises can have a private registry on-premises for their own Docker images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qpT6XXoBkPOSyqQ9d0QUFQ.png" /></figure><p>If we look at the more specific example of Application Containerization with Docker;</p><ul><li>First, we should write dockerfile for our application,</li><li>Build application with this docker file and creates the docker images.</li><li>And lastly run this images on any machine and creates running docker container from docker image.</li></ul><p>We will use all steps with orchestrating whole microservices application with docker and Kubernetes for the next articles -&gt;</p><ul><li><a href="https://mehmetozkaya.medium.com/preparing-multi-container-microservices-applications-for-deployment-793d60f48d31"><strong>1- Preparing Multi-Container Microservices Applications for Deployment</strong></a></li></ul><h3>References</h3><p><a href="https://docs.docker.com/get-started/overview/">https://docs.docker.com/get-started/overview/</a><br><a href="https://docs.docker.com/get-started/">https://docs.docker.com/get-started/</a><br><a href="https://medium.com/batech/docker-nedir-docker-kavramlar%C4%B1-avantajlar%C4%B1-901b37742ee0">https://medium.com/batech/docker-nedir-docker-kavramlar%C4%B1-avantajlar%C4%B1-901b37742ee0</a><br><a href="https://www.mediaclick.com.tr/tr/blog/docker-nedir-docker-ne-ise-yarar">https://www.mediaclick.com.tr/tr/blog/docker-nedir-docker-ne-ise-yarar</a><br><a href="https://www.docker.com/resources/what-container">https://www.docker.com/resources/what-containe</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c50bdd51b702" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aspnetrun/deploying-net-microservices-to-azure-kubernetes-services-aks-and-automating-with-azure-devops-c50bdd51b702">Deploying .Net Microservices to Azure Kubernetes Services(AKS) and Automating with Azure DevOps</a> was originally published in <a href="https://medium.com/aspnetrun">aspnetrun</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>