<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Marcos Gomes on Medium]]></title>
        <description><![CDATA[Stories by Marcos Gomes on Medium]]></description>
        <link>https://medium.com/@marcosangomes?source=rss-e2f222aae94e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*4BuDdpDQLFx4LwZ4znQ0dg.png</url>
            <title>Stories by Marcos Gomes on Medium</title>
            <link>https://medium.com/@marcosangomes?source=rss-e2f222aae94e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 15:40:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@marcosangomes/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Defining Inventa's Observability Strategy]]></title>
            <link>https://medium.com/building-inventa/defining-inventas-observability-strategy-4ab0988f3c96?source=rss-e2f222aae94e------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ab0988f3c96</guid>
            <dc:creator><![CDATA[Marcos Gomes]]></dc:creator>
            <pubDate>Thu, 25 May 2023 14:21:14 GMT</pubDate>
            <atom:updated>2023-05-25T14:21:14.564Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eH7bD3aeSm-5Bupf" /><figcaption>Photo by <a href="https://unsplash.com/ko/@freddymarschall?utm_source=medium&amp;utm_medium=referral">Frederick Marschall</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Upon arriving at Inventa, one of my primary responsibilities was monitoring our applications to better understand their behavior and our infrastructure.</p><p>At this point, Inventa was about a year old and we were transitioning from a serverless Lambda architecture to a microservices architecture with Kubernetes (EKS), where we didn&#39;t have a strategy for observability, alarms, or logs. More here <a href="https://medium.com/building-inventa/how-we-developed-our-deployment-model-in-inventa-part-1-ea3f83c07157">How we developed our deployment model.</a></p><p>So, we started investigating something that would allow us to scale, that was modern, and that the price wouldn’t be a pain for the business as we scaled.</p><p>Based on these requirements, an RFC was structured and the result came up with three possible options.</p><p><strong>Zabbix:</strong> Simple and easy to implement, with an agent on the instances, but the tool was created to monitor networks and evolved to keep up with the evolution of highly dynamic systems in a structure of containers and microservices. Since we were starting from scratch, we understood that something more robust and modern could meet our needs in the short, medium, and long term. We immediately ruled it out.</p><p>And then we went into a more in-depth comparison between <strong>Prometheus</strong> and <strong>Elastic Search</strong>, where we analyzed the following aspects:</p><h4><strong>Purpose:</strong></h4><ul><li>Prometheus: is a real-time monitoring tool for distributed systems, designed to collect, store, and query metrics in real-time.</li><li>Elasticsearch: is a search and data analysis engine that can be used to store, search, and analyze structured and unstructured data at scale.</li></ul><h4>Storage of Data:</h4><ul><li>Prometheus: stores data in a time series format.</li><li>Elasticsearch: uses an inverted index engine to store data.</li></ul><p>This means that Prometheus is optimized for storing and querying time series data, while Elasticsearch is more suitable for search and unstructured data.</p><h4>Scalability:</h4><ul><li>Prometheus: is highly scalable and designed to perform well in distributed environments.</li><li>Elasticsearch: is also scalable, but requires more configuration and tuning to perform well in distributed environments.</li></ul><h4>Queries and Analysis:</h4><ul><li>Prometheus: optimized for querying and analyzing real-time metrics.</li><li>Elasticsearch: more suitable for analyzing search data in unstructured data.</li></ul><p>With this, we understood that the ideal strategy for our scenario, in the medium and long term, would be to use both, where application and infrastructure metrics would be collected and stored using Prometheus, and for querying event logs and application logs, we would use Elastic Search.</p><p>However, we faced a significant challenge: we only had one engineer available to do the entire implementation. It was a large project where we wanted to cover the basic pillars of observability, which are logs, metrics, and tracing.</p><p>In the presentation of the RFC, one of our lead engineers blurts out in the virtual room:</p><blockquote>Egberto: Do you guys know about Grafana Cloud?</blockquote><p>Although no immediate decisions were made, an action point was added to explore the potential benefits of this solution.</p><p>In the study, we identified:</p><p>Roughly speaking, when we create an account on Grafana Cloud, which is a SaaS observability tool, the structure of the following services is automatically provisioned:</p><ul><li><strong>Grafana</strong>: Customizable data visualization platform that allows users to create, explore, and share dashboards and panels in real time. It supports multiple data sources, including databases, cloud services, and APIs, and offers a range of features for monitoring and analyzing systems like Kubernetes, Instances, Synthetic monitoring, Usage insights, Network usage, and more….</li><li><strong>Grafana Loki</strong>: A service used for log ingestion and querying. Configuration is done through an agent provided by Grafana Cloud upon account creation.</li><li><strong>Grafana Prometheus</strong>: A powerful open-source time series database, that can be a monitoring and alerting toolkit that enables users to collect and analyze metrics from various systems in real time, like our apps in Kubernetes.</li><li><strong>Grafana Tempo</strong>: On the <em>server side</em>, it is an easy-to-use, highly scalable open-source distributed tracing backend. It is deeply integrated with Grafana, Prometheus, and Loki. It can ingest common open-source tracing protocols, including Jaeger, Zipkin, and <strong>OpenTelemetry, which is our case</strong>. On the application side, an auto-instrumentation agent for Java based on OpenTelemetry was integrated to generate tracing spans.</li><li><strong>Grafana Alerts</strong>: It is a customized version of AlertManager highly integrated with the above-mentioned services.</li><li><strong>Grafana Incidents</strong>: also integrated with the services mentioned above, for incident management, in the same format as PagerDuty, but all these services are already available and configured when creating an account on Grafana Cloud.</li><li><strong>Synthetic Monitoring</strong>: We monitor how a specific endpoint is performing, with the possibility of configuring logins, headers, tokens, and more. For example, we use it to monitor our blog hosted outside our engineering stack.</li><li><strong>Grafana OnCall</strong>: Also integrated with the services above, when we have an alert with an incident, it is possible to escalate it to the responsible team to act on the incident, and at the same time we have a unified location where all engineers can track the status of the entire incident cycle.<br>In the free version we can register up to 5 incidents.</li><li><strong>Grafana Data sources</strong>: Grafana has plugins to integrate with virtually any possible data source, from Cloudwatch, Elastic Search, Kubernetes, Databases, Sentry, Windows, GitHub, and more.</li></ul><p>Amazing, isn’t it?</p><p>Based on this new information, we decided to adopt Grafana Cloud as our observability stack, and then we got down to work:</p><p>By leveraging Grafana Cloud’s pre-existing infrastructure as a SaaS model, we were able to streamline our configuration process and focus on creating processes and automation to effectively utilize their dashboard and visualization service, alerting and notifications service, and data source and integrations service.</p><p>But we know that not everything is perfect, right? Let’s not forget that only one engineer is working on this project. At the beginning of any new tool that we start using, we have a learning curve, which is normal and during this journey, we noticed an incredible team in the CS department, with amazing and delightful customer service always helping us!</p><p>If you’ve made it this far, you’re likely curious about the cost of using Grafana Cloud. The pricing for Grafana Cloud is determined by four different variants:</p><ul><li>Active metrics: the number of metrics being generated by the clusters and applications and consumed by Grafana Prometheus.</li><li>Logs: the number of logs in GB consumed by Grafana Loki.</li><li>Tracing: the amount of tracing in GB consumed by Grafana Tempo.</li><li>As it is a SAAS with a customized UI and authentication, we also pay per active user per month.</li></ul><p>After the applications were created and deployed in the new infrastructure, the costs grew as expected, but we identified an opportunity to improve our costs related to large-scale consumption of custom metrics, given each application sends thousands of metrics by default and this would significantly increase the costs of Grafana Cloud. Therefore, we adopted a strategy to consume these custom metrics from our Kotlin applications using a local Prometheus that we deploy and manage ourselves, and later, we extended this strategy to infrastructure metrics as well.</p><p>Today, an overview of our observability architecture would look like this:</p><p>As we were migrating from Lambda to Kubernetes, we still needed to see the logs from the Lambdas. In Grafana Cloud this is easy because we can integrate a bunch of Datasource providers as a plugin by Grafana.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*A9bnOV0s7PNNxuFyOxEj1A.png" /></figure><p>With Grafana Cloud working in our favor to abstract all the infrastructure parts (creating, configuring, and maintaining), we only had to configure integrations and create processes and automation. It took us a little over 2 months to create scalable and applicable ingestion processes for logs, metrics, tracing, incidents, and on-call in our organization.</p><p>This is how our days are here at Inventa. How about joining us!? We are looking for Engineers who enjoy the state-of-the-art platform and application development, and want to help build a very reliable, observable, and scalable product in Inventa! Hit <a href="https://www.linkedin.com/jobs/search/?currentJobId=3437853272&amp;f_C=71737018&amp;geoId=92000000&amp;originToLandingJobPostings=3437853267%2C3437852421%2C3437853272%2C3437846946%2C3437853264%2C3437847949%2C3457031342%2C3437850571%2C3437852426">here</a> to apply and, if you have any questions, do not hesitate to ping us on <a href="https://www.linkedin.com/in/marcosangomes/">LinkedIn</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*sNbD3eHefks8KwSd.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ab0988f3c96" width="1" height="1" alt=""><hr><p><a href="https://medium.com/building-inventa/defining-inventas-observability-strategy-4ab0988f3c96">Defining Inventa&#39;s Observability Strategy</a> was originally published in <a href="https://medium.com/building-inventa">Building Inventa</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mapping the costs of EC2 services]]></title>
            <link>https://medium.com/@marcosangomes/mapping-the-costs-of-ec2-services-aaa1bdf0c642?source=rss-e2f222aae94e------2</link>
            <guid isPermaLink="false">https://medium.com/p/aaa1bdf0c642</guid>
            <dc:creator><![CDATA[Marcos Gomes]]></dc:creator>
            <pubDate>Sun, 29 Mar 2020 20:01:19 GMT</pubDate>
            <atom:updated>2020-03-29T20:26:46.983Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZUeS17BSA-zbzVbawH33Qw.jpeg" /></figure><p>With all this boom that Coronavirus has been causing in the economy, I decided to share one of my projects for cost mapping in AWS EC2 services (Load Balancers, Instances and EBS).</p><p>The ‘Team’ tag is just an example, the code can be adapted to your infrastructure as you organize your costs.</p><p>Ex:</p><p>Tag -&gt; ‘Team’.<br> Value -&gt; ‘Financial’.</p><p>Tag -&gt; ‘Team’.<br>Value -&gt; ‘Administrative’</p><p>Tag -&gt; ‘Team’.<br>Value -&gt; ‘Node app xyz’.</p><p>Tag -&gt; ‘Team’.<br>Value -&gt; ‘PHP app xpto’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/1*VU2IJf1zfMTI-4R3VdVQOw.png" /></figure><p>In this first article I will start by mapping Load Balancers by team, checking what has the ‘Team’ tag and generating a report file and sending it to a Slack channel at the end.</p><p>When you have a small environment with few Load Balancers, you look at your infrastructure and quickly you can see what can be reduced and who is spending what, but when you have a large environment, this analysis is masking, time consuming and boring.</p><p>at Amazon we have two types of loadbalancers that boto3 sees, they are differentiated by boto3 by the class invoked:</p><p>1) Application Load Balancer and Network Load Balancer</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/477/1*zuRzNuZe__O7orRzACAkaw.png" /></figure><p>2) Classic Load Balancer</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/457/1*ZE679ZsKQKYbebV-cDQwZA.png" /></figure><p>Consequently I created two functions to map all lbs of the two types:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*4W8eum9NTpprqPVQbnv07Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/543/1*reR-46HUQjw7W5wxYNtpJw.png" /></figure><p>As we are working with two different classes of boto3, the part of checking the tags also works differently depending on the two functions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/522/1*lk6lOAidMV0r75T1ZDKZXg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*lAesBpKtHxUeqwwuMTQtaA.png" /></figure><p>And with these resources I created a routine to write the success and failure of the conditions in a file and generate a report at the end that is sent by Slack.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*yEWH47Vz_V62Qf6h2AHSWg.png" /></figure><p>and the function to report on Slack:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/1*h65sRZ_xZqLgLzVdfn4UCw.png" /></figure><p>It also includes a dockerfile to create an automated pipeline, which can be done by Github Actions, Bitbucket pipelines, Circle CI, Jenkins.<br>You schedule the task for the report to be done per day, or week or month and follow up on your decision making.</p><p>And the result is this on Slack:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*K5fdtSSM3DmVtwQRekFqZg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*RJL0_c7JHX3uRUkW0B29mA.png" /></figure><p>So that’s it folks, I’ll share some automations about cost mapping in EC2 and EBS later on.</p><p>Feel free to contact me on linkedin if you have questions, suggestions, improvements.</p><p>project github: <a href="https://github.com/marcossangomes/reports-elb-teams">https://github.com/marcossangomes/reports-elb-teams</a></p><p>See ya!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aaa1bdf0c642" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[About Big Data Foundations — Part 2]]></title>
            <link>https://medium.com/@marcosangomes/about-big-data-foundations-part-2-7035707a1bdd?source=rss-e2f222aae94e------2</link>
            <guid isPermaLink="false">https://medium.com/p/7035707a1bdd</guid>
            <category><![CDATA[hadoop]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[iot]]></category>
            <dc:creator><![CDATA[Marcos Gomes]]></dc:creator>
            <pubDate>Tue, 28 Aug 2018 15:19:30 GMT</pubDate>
            <atom:updated>2018-08-28T15:19:30.652Z</atom:updated>
            <content:encoded><![CDATA[<p>Continuing from where we left off, <strong>APACHE HADOOP</strong> is a framework with several modules, some of them:</p><p><strong>— Hadoop HDFS(Hadoop Distributed File System):</strong> Google’s engineers saw that with massive data growth they needed to store large amounts of data and process them in real-time, with low-cost hardware prepared to perform with failures (which would happen because it deals with low cost hardware ) and scalable (if you needed the infrastructure to grow to keep up with data and processing growth), and so the Google File System (GFS) was created. Google made this tool open-source and introduced the community, kicking off Hadoop HDFS.</p><p><strong>Hadoop HDFS</strong> although storing data is not an RDBMS (database) but is a file storage manager.</p><p>We say that HADOOP HDFS works with data storage in clusters with low cost hardware, so let’s clarify the concept of Cluster.</p><p><strong>Cluster</strong> are many machines (called nodes), installed in a rack that are communicated via the network. An infrastructure where they have multiple rack’s are called Data Center.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/566/1*3aWU1SWY7iUUh6ekj6LYtA.png" /></figure><p>In the <strong>Hadoop HDFS</strong> architecture we have:</p><p><strong>-Namenode (MasterNode):</strong> Manages the storage process.</p><p><strong>-Secondary NameNode:</strong> Works in parallel doing the same function as the MasterNode and / or functions as redundancy of the service as well.</p><p><strong>-DataNode (Slave):</strong> Stores the data distributed by the slave nodes, in case a host fails, the files are not lost, because the data blocks are parsed in all DataCenter (here we perceive the high redundancy of the fault that the tool offers).</p><p><strong>Hadoop</strong> services require a Java Virtual Machine (JVM) to run, and to configure a Hadoop ecosystem environment, we can apply 3 types of architectures</p><p><strong>- Standalone:</strong> Indicated for environment testing, where all ecosystem services are installed in the same JVM.</p><p><strong>- Pseudo Distributed:</strong> Indicated for environment testing too, but requires a higher level of processing (more powerful hardware). In the same server we have several JVMs, where the services are installed distributed by the JVM’s.</p><p><strong>- Totally Distributed:</strong> Indicated for production environment, where we have clusters where the services will be install distributed by the JVMs of the environment.</p><p>These environments can be deployed in Cloud architectures when On premise (Data Centers in house), as we know maintaining an infrastructure in the cloud is more cheap and easy than having its own Data Center, but depending on the company’s type of information, until where I know Hospitals and Banks work with confindental information, which by law, can not store certain kinds of information in the cloud.</p><p>We are living the beginning of Big Data, where the term appeared around 2001/2003 and the tools began to evolve in 2011, with the technology evolving and also worth highlighting the security part, in the near future these impasses tend to evolve and we will certainly be able to migrate this sensitive information to architectures in the cloud.</p><p><strong>- Hadoop MapRecude:</strong> Unlike HDFS (storage), MapReduce works with distributed parallel processing of information stored in clusters, transforming the data into small pieces for analysis.</p><p>The purpose of <strong>MapReduce</strong> is to take a large set of data and map and reduce(process) to a set of information useful for data scientists to use analysis techniques to generate valuable insights (value information for the business).</p><p>It is the job of the data scientist to apply MapReduce, which generates Dataframes with ordered lists (key-values) to carry out the business objective definition analysis (recommendation system, real-time analysis, model of analysis of time series, analysis of feelings and etc …)</p><p>In the Hadoop MapReduce architecture we have:</p><p><strong>-JobTracker:</strong> Coordinate how the processing will be distributed across the cluster.<br><strong>- TaskTracker:</strong> responsible for performing Map or Reduce task processing.</p><p>MapReduce services can be installed on the same machines as NameNodes + JobTrackers and DataNodes + TaskTracker.</p><p>It should be noted that a JVM can have one or more services installed on them.</p><p>Here’s a diagram for an overview of how it would work:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/536/1*5J4TxK7HFRGXVcHTXaXpYg.png" /></figure><p>See you next week with the third part, hugs!</p><p>Sources: <a href="http://www.ic.unicamp.br/~cortes/mo601/trabalho_mo601/tiago_cruz_map_reduce/relatorio.pdf%20/">http://www.ic.unicamp.br/~cortes/mo601/trabalho_mo601/tiago_cruz_map_reduce/relatorio.pdf /</a> <a href="https://www.datascienceacademy.com.br/">https://www.datascienceacademy.com.br/</a> / <a href="https://cognitiveclass.ai/">https://cognitiveclass.ai/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7035707a1bdd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[About Big Data Foundations]]></title>
            <link>https://medium.com/@marcosangomes/about-big-data-foundations-d8b314f129d3?source=rss-e2f222aae94e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8b314f129d3</guid>
            <category><![CDATA[hadoop]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[industry-4-0]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Marcos Gomes]]></dc:creator>
            <pubDate>Tue, 21 Aug 2018 15:47:29 GMT</pubDate>
            <atom:updated>2018-08-25T15:56:47.936Z</atom:updated>
            <content:encoded><![CDATA[<p>Today I bring you an overview of the Big Data Fundamentals course i completed in the weekend at Data Science Academy. The course aborte fundamentals, concepts and tools for Big Data.</p><p><strong>What is Big Data?</strong></p><p>Wikipedia: <em>“term used to name very large or complex data sets that traditional data processing applications still can not handle”</em></p><p>Oracle: “<em>big data is an management strategic of holistic informations that include and integrated many types of data and management data with tradicional datas.”</em></p><p>Data Science Academy: <em>“The ability of a society to obtain information on new ways, generating new ideas and goods services of significant value”</em></p><p>Big Data is a large, structured, unstructured or streaming data volume from various sources and formats, which together with analytical techniques generate insights for the business making in real-time decision.</p><p>When we talk about <strong>Big Data</strong> we have 4 pillars that help us to understand the reality of Big Data, <strong>the 4 V’s Big Data</strong>: Volum (data size), Variety (data format), Velocity ​​(which are data generated) and Veracity (of datas).</p><p>Ever wondered what the volume of Facebook and Google of datas?</p><p>The volume of data from these largest internet companies is from around of 1 Yottabyte.</p><p>It would be something like:</p><p>1000 Terabytes = 1 Petabyte</p><p>1000 Petabytes = 1 Exabyte</p><p>1000 Exabytes = 1 Zettabyte</p><p>1000 Zettabytes = 1 Yottabyte</p><p>Surreal do not agree? But how do you store, process, and manage these massive amounts of information? Now that’s the cool part, nice to meet you, <strong>Hadoop!</strong></p><p>At the beginning of this course I was intrigued, because it was <strong>Hadoop</strong> there, <strong>Hadoop</strong> here and I had no idea of ​​the size of this tool yet, but we will get there.</p><p>So what is Hadoop? Also known as Apache Hadoop?</p><p>It is an open-source framework created in 2008 <strong>for shared, distributed, and highly reliable storage and processing</strong> management for <strong>large datasets</strong> in <strong>low-cost</strong> hardware clusters.</p><pre>This tool is extremely powerful because it was created to be worked with Big Data, it was born to meet a reality that until then we did not know how to handle. There was no tool that did this effectively in the market. A framework that uses multiple machines in clusters as one, sharing the storage and processing of low cost hardware, with reliability and scalability, potentializing the use of these machines together to work with massive amounts of data.</pre><p><strong>Hadoop</strong> consists of 3 main:</p><ul><li><strong>Hadoop Distributed File System (HDFS)</strong> manages the HD’s of a cluster of machines, so that several HD’s are viewed as only 1 HD. Responsible for the recording and readings in the HD’s of the clusters</li><li><strong>Hadoop Yarn</strong> responsible for the management of clustered computing resources, as well as the scheduling of resources.</li><li><strong>Hadoop MapRecude</strong> responsible for the management of data processing in clusters.</li></ul><p>It is worth noting that <strong>APACHE HADOOP</strong> is for problems so large and complex that traditional systems are not able to handle, it was born to meet the present and future of Big Data.</p><p><strong>Hadoop</strong> is a free Java-based <strong>framework</strong> and is inspired by the <strong>Google File System (GFS)</strong>, a file system created by Google, that is, it uses a technique from one of the best technology companies in the world, so we begin to realize the real power of this tool.</p><p>I separated this post in 4 pieces to not have too much content, in the nexts weeks i will post the rest of the content.</p><p>If you want to leave your opnion, tips, corrections, feel free to contact me.</p><p>sources:</p><p><a href="https://pt.wikipedia.org/wiki/Big_data%20/">https://pt.wikipedia.org/wiki/Big_data /</a> <a href="https://www.oracle.com/br/big-data/index.html%20/">https://www.oracle.com/br/big-data/index.html /</a> <a href="https://pt.wikipedia.org/wiki/Hadoop%20/">https://pt.wikipedia.org/wiki/Hadoop /</a> <a href="https://www.datascienceacademy.com.br/">https://www.datascienceacademy.com.br/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8b314f129d3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>