<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sivanujan Sivakumar on Medium]]></title>
        <description><![CDATA[Stories by Sivanujan Sivakumar on Medium]]></description>
        <link>https://medium.com/@sivanujansivakumar?source=rss-17ea3263b73b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Dq5kXzrIeh8kZe8SnnYCaQ.jpeg</url>
            <title>Stories by Sivanujan Sivakumar on Medium</title>
            <link>https://medium.com/@sivanujansivakumar?source=rss-17ea3263b73b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 16:08:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sivanujansivakumar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What is data leakage in machine learning?]]></title>
            <link>https://sivanujansivakumar.medium.com/what-is-data-leakage-in-machine-learning-d0f24c812ae5?source=rss-17ea3263b73b------2</link>
            <guid isPermaLink="false">https://medium.com/p/d0f24c812ae5</guid>
            <category><![CDATA[data-leak]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-leakage-prevention]]></category>
            <category><![CDATA[data-leak-prevention]]></category>
            <dc:creator><![CDATA[Sivanujan Sivakumar]]></dc:creator>
            <pubDate>Wed, 18 Dec 2024 16:32:19 GMT</pubDate>
            <atom:updated>2024-12-18T16:32:19.374Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6XNl0JWdEz-YKVmo_B9Iyw.png" /></figure><p>What is data leakage?</p><p>Data leakage is the creation of unexpected additional information in the training data, allowing a machine learning algorithm to make unrealistically good predictions in a development environment (in the training and evaluation phase).</p><p>We might overestimate how well the model performs on real data in production.</p><p>But it performs worst in the production environment.</p><p>Why?</p><p>First, we should understand how this additional information is added to training data indirectly in the training phase. This additional information is added from test data, not training data. That is why there is good accuracy in the training and evaluation processes.</p><p>📌 Train-test contamination is one type of data leakage. How did this occur?</p><p>Datasets have columns in different scales, like age and salary. Here Both column values are in different scales, right? Age 1 to 100, while salary 10,000$ to 100,000$, here both are different scales.</p><p>So if we want to work this data in an ML model, we should do feature scaling to maintain the same scale of both features.</p><p>Then where does the data leakage problem arise? Well, when we do the feature scaling technique before splitting the data into a train and test set, there is data leakage that occurs. So-called information sharing within training and testing. How does this happen?</p><p>When we do feature scaling in the dataset before splitting into train and test sets, for example, we do standardization (one of the feature scaling techniques) in the age and salary column; it calculates the z-score using the mean and standard deviation of each column using all values in order to maintain both columns in the same scale. Once we have done feature scaling, then we do split the dataset into train and test. Here is the problem that arises.</p><p>When we split the dataset into train and test after feature scaling, the model sees test data indirectly. How did it happen? When we do feature scaling, we calculate the z-score using mean and standard deviation. It calculates the mean and standard deviation using the entire dataset in each column. This means that both training and testing datasets are used to calculate the z-score. Actually, it should be used only for training datasets. The test dataset should not be used. Here, data leakage indirectly happens if we do the split after feature scaling.</p><p>When it comes to model development, the test dataset should be unseen or isolated from the training set. So if we do feature scaling the entire dataset before the split, data leakage occurs indirectly in the model development.</p><p>How can we avoid these data leakage problems?</p><p>First, we should split our dataset into training and testing sets.</p><p>Then we can apply feature scaling.</p><p>Use the training dataset mean and standard deviation to rescale the training data. In the model-building phase, you might use the “.fit_transform()” method for the training set.</p><p>Use the same train dataset mean and standard deviation to rescale test data. In the model-building phase, you might use the “.transform()” method for the test dataset.</p><p>These are procedures we should follow to avoid the data leakage problem. We will discuss another type of data leakage problem in an upcoming article. Have a great day! :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d0f24c812ae5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Delve into RDBMS and SQL.]]></title>
            <link>https://sivanujansivakumar.medium.com/delve-into-rdbms-and-sql-42ed83622a4e?source=rss-17ea3263b73b------2</link>
            <guid isPermaLink="false">https://medium.com/p/42ed83622a4e</guid>
            <category><![CDATA[rdbms]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[sql-server]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Sivanujan Sivakumar]]></dc:creator>
            <pubDate>Fri, 02 Feb 2024 15:44:56 GMT</pubDate>
            <atom:updated>2024-02-02T15:44:56.473Z</atom:updated>
            <content:encoded><![CDATA[<p>It is good to see you here again. Thus, we covered a quick synopsis of data, databases, and database management systems last week. We will delve into <strong>RDBMS</strong> today. Data is kept in a relational database management system in the form of tables with rows and columns. Data in a database is stored according to a predetermined structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w2CMbXDQHMeAwVUweXlRRA.png" /></figure><p>Every column contains a different kind of data. We need to indicate the data type for every column in our table when we create it. An example of how this appears is provided below: The Order Number column needs to have an integer data type in order to store numeric values. The order date column needs to have a date data type in order to store date information. Sortly, each column needs to contain the data type that will be utilized to hold a particular value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/847/1*eKF_OnEZoBgdRKqrrBQFmA.png" /></figure><p>Let us discuss the meaning behind the name of the database management system: relational. Relational refers to the ability to establish a relationship between tables. Take a look at the tables below. A common column called <strong>customer ID</strong> is present in both the order table and the customer table. There is a common column called<strong> Product ID</strong>, just like in the order and product tables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*itB3LjpISDhcHdcvba-42Q.png" /></figure><p><strong>Structured Query Language: SQL</strong></p><p>We were familiar with data, databases, and database management systems. What, then, is SQL? What role does SQL play? SQL is the language used to communicate between the DBMS and its users. Database interaction is possible with SQL.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ss9U822c2ufcxo0p_DZFQA.png" /></figure><p>As mentioned above, lookup data was kept in tables within databases. With this SQL, we can build databases. We can create tables inside each database after they have been created. The table structure can be changed if desired. In the table, we can add new data. Data in our tables can be added, updated, and removed. If the table and database are no longer needed, we can remove them.</p><p>I have included a list of the most often used SQL commands below:</p><ul><li>ALTER</li><li>CREATE</li><li>DELETE</li><li>DROP</li><li>INSERT</li><li>SELECT</li><li>UPDATE</li></ul><p>With it, we can communicate using this SQL variant of DBMS. For example:</p><ul><li>Microsoft SQL Server</li><li>MySQL</li><li>ORACLE</li><li>PostgreSQL</li><li>SQLite</li></ul><p>The majority of the SQL commands apply to all of the databases mentioned above. For certain particular operations, each database has a special command, though. For example, T-SQL, or Transact SQL, is the query language specific to the Microsoft SQL Server. It can assist with tasks like inserting new rows, retrieving multiple rows, and retrieving data from a single row.</p><p><strong>In summary</strong></p><p>So we cover what RDBMS is, what SQL is, how to use SQL in databases, how to create tables, how data is stored in tables, and the various DBMS that use SQL nowadays. I think this story has given you some knowledge. I appreciate your taking the time to read this. I will soon have another captivating story to share. Enjoy your pleasant day. Ba Bye.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42ed83622a4e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting Started with Data]]></title>
            <link>https://sivanujansivakumar.medium.com/getting-started-with-data-a410756f2f45?source=rss-17ea3263b73b------2</link>
            <guid isPermaLink="false">https://medium.com/p/a410756f2f45</guid>
            <category><![CDATA[sqlite]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[sql-server]]></category>
            <dc:creator><![CDATA[Sivanujan Sivakumar]]></dc:creator>
            <pubDate>Tue, 23 Jan 2024 16:08:36 GMT</pubDate>
            <atom:updated>2024-01-23T16:39:14.356Z</atom:updated>
            <content:encoded><![CDATA[<p>Bonjour, You finally made it here. Cool. To talk about data, are you ready? All right, so just what is data? Well, that is a good question. First, let us get that understood. <strong>A collection of relevant facts is called data, and it must be transformed in order to provide relevant information for decision-making. </strong>There are several formats for these data, including unstructured, semi-structured, and structured formats.<strong> </strong>OK, where do these data come from? Well, now you have read this article on Medium. If you could use an email address and password that match your account, that would be ideal for logging in. Medium currently has your log information. Be aware that you generated the log-in information yourself. As a result, the data we create is based on our actions. We make transactions online, shop online, and so forth. Organizations must effectively manage the diverse data formats generated by these digital interactions.</p><h3><strong>Database</strong></h3><p>Cool, Now you have an overview of what data is. Ok, where does the database stand here?? And what is its role? A<strong> database is an organized collection of data, and it supports data storage and manipulation.</strong>Above, I mention various samples of data that need to be stored. Yes, so how can we store this data? For that purpose, we use databases. Not only for the storage, but we can manipulate that data. Manipulating means we can insert new data, and we can update existing data. Also, retrieve our data when we need it in any situation. Think Dev Context: Each front-end application retrieves data from the backend database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*irwy1bvkzNXssRsPNmNlrQ.png" /></figure><h3><strong>Database Management System</strong></h3><p>Hmm, now we have overlooked data and databases. But how do we create this database and store this data? Cool. For this purpose, we need a platform, right? <strong>That is the so-called DBMS, aka Database Management System. Like its name suggests, it is a system that manages databases.</strong> You can think of it as a platform, medium, tool, or set of software programs. DBMS acts as a middleman between end users and databases. This DBMS has the capability of allowing multiple users to access the same data at the same time. But all the users can&#39;t access all the data. There is a restriction on data access. In an organization,</p><ul><li>Some users can only read the data from the DBMS.</li><li>Some users can only update data from databases.</li><li>Some users can only delete data from databases</li></ul><p>It all comes from role-based access control (RBAC). I will discuss this in my later story. In the industry, various DBMS platforms are available. Have a look at some DBMS platforms, such as</p><ul><li>Microsoft Access</li><li>Microsoft SQL Server</li><li>MongoDB</li><li>MySQL</li><li>Oracle</li><li>Redis</li><li>SQLite</li></ul><p>There are various platforms available. Also, there are various types of DBMS.</p><ul><li>Flatfile DBMS</li><li>Hierarchical DBMS</li><li>Network DBMS</li><li>Non-Relational DBMS</li><li>Relational DBMS</li></ul><p>On the other hand, the most widely used DBMSs in this context are relational and non-relational DBMSs.</p><h3>Relational Database Management System</h3><p>First, we will talk about relational database management systems. So-called RDBMS. So, what is RDBMS? Here, data is stored in a table format, like rows and columns. The table has a predefined structure. For example, if we have a sales order table, the Order No column should have an integer format, and the Order Date column should have a date format. Some constraints will apply when we design our tables in databases. I will discuss this in my next story.</p><h3>Conclusion</h3><p>So, today we go through what data is, where it comes from, how we can store and manipulate this data, and how we can define our table in a DBMS. I believe you acquired some knowledge from this intro data story. Thank you so much for reading this. I will come up with another interesting data story soon. Have a nice day. Ba Bye.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a410756f2f45" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>