<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Marten Gartner on Medium]]></title>
        <description><![CDATA[Stories by Marten Gartner on Medium]]></description>
        <link>https://medium.com/@martengartner?source=rss-d3e102089d47------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*95LvKb8kV5fasNcms7lATw.jpeg</url>
            <title>Stories by Marten Gartner on Medium</title>
            <link>https://medium.com/@martengartner?source=rss-d3e102089d47------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 01:08:18 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@martengartner/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Egorm — A Simple and Powerful Wrapper of Gorm]]></title>
            <link>https://martengartner.medium.com/egorm-a-simple-and-powerful-wrapper-of-gorm-3d0e4a33a4e7?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d0e4a33a4e7</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[orm]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Thu, 27 Apr 2023 13:07:57 GMT</pubDate>
            <atom:updated>2023-06-05T09:55:54.363Z</atom:updated>
            <content:encoded><![CDATA[<h3>Egorm — A Simple and Powerful Wrapper of Gorm</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CNvKbVD0ubf5Zf2AUX1Hlw.jpeg" /><figcaption>Foto von <a href="https://unsplash.com/@casparrubin?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Caspar Camille Rubin</a> auf <a href="https://unsplash.com/de/fotos/fPkvU7RDmCo?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Go is a lightweight and performant language which makes it an ideal choice for creating small, efficient services that can be easily deployed and maintained. The language’s straightforward syntax and small footprint mean that Go programs can run with minimal overhead. This makes Go particularly well-suited for environments where resources are limited. The Go ecosystem makes building and deploying microservices a breeze. Go’s built-in concurrency model, based on goroutines and channels, allows developers to create highly concurrent applications without the complexities often associated with multithreading. This makes it easier to build scalable microservices that can handle a large number of requests simultaneously. Finally, Go offers excellent performance, rivaling that of C and C++ in many cases. With its focus on simplicity and efficiency, Go provides a solid foundation for creating high-performance microservices that can meet the demands of modern applications.</p><h3>Why We like Gorm</h3><p>For microservices, database operations are often at the heart of their functionality. The ability to store, retrieve, and manipulate data efficiently and effectively is crucial to the success of any microservice-based architecture. Enter <a href="https://gorm.io/">Gorm</a>, a powerful Object-Relational Mapping (ORM) library for Go.</p><p>Gorm provides several benefits that make it an attractive choice for managing database operations in Go-based microservices:</p><ul><li><strong>Easy to use:</strong> Gorm’s intuitive API simplifies the process of working with databases, allowing developers to focus on implementing their microservices’ core functionality.</li><li><strong>Feature-rich:</strong> Gorm offers a wide range of features such as associations, transactions, and query building, which makes it a versatile choice for various application requirements.</li><li><strong>Extensible:</strong> Gorm’s plugin system enables developers to extend its functionality and adapt it to their specific needs.</li><li><strong>Cross-database support:</strong> Gorm supports a variety of databases, including PostgreSQL, MySQL, SQLite, and SQL Server, making it suitable for projects with diverse database requirements.</li></ul><h3>Why EGorm?</h3><p>Despite its many benefits, Gorm can still require a significant amount of boilerplate code to set up and use effectively. This article aims to introduce <a href="https://github.com/martenwallewein/easy-going/tree/master/pkg/egorm">E(asy)Gorm</a>, an abstraction of Gorm that leverages Go generics to further simplify its usage. Egorm’s goal is to minimize the amount of boilerplate code required to work with Gorm, making it even easier to integrate Gorm into your microservices.</p><p>In the following sections, we will walk through the implementation of Egorm, covering setting up Gorm, using Egorm to interact with your database, and automating migrations. By the end of this article, you should have a solid understanding of how EGorm can enhance your Gorm experience and streamline the development of your Go-based microservices.</p><h3>Implementation and Usage</h3><p>In this section we will have a look at how EGorm is implemented and how it can be used. Egorm targets to give you as a developer just the core methods that you need to perform database operations, e.g. the common CRUD operations. However, there is a small fraction of code required to setup EGorm.</p><h4>Setting up EGorm</h4><p>To setup database connections EGorm provides a list of setup methods, one for each supported database type, e.g. SQLite or Postgres. Let’s have a look at the SQLite setup code. To configure EGorm to use an sqlite connection, all we need to do is to call SetSQLiteConnectOpts before the first database operation, e.g. when our service starts, or to configure the respectire EGORM_DB=sqlite and EGORM_DB_SQLITE_PATH=egorm.sqlite. To connect to e.g. postgres, the code looks similar, including a configuration via environment variables. When the database is configured via environment, this allows to immediately perform database operations without calling any setup method before.</p><pre><br>// SQLITE<br>// Via environment<br>// EGORM_DB_SQLITE=egorm_test.sqlite<br><br>// Via setup method<br>SetSQLiteConnectOpts(&amp;egorm.SQLiteConnectOpts{<br> Path: path.Join(os.TempDir(), &quot;egorm_test.sqlite&quot;),<br>})<br><br>// POSTGRES<br>// Via environment<br>// EGORM_DB_POSTGRES_HOST=postgres<br>// EGORM_DB_POSTGRES_PORT=5432<br>// EGORM_DB_POSTGRES_DATABASE=postgres<br>// EGORM_DB_POSTGRES_USER=user<br>// EGORM_DB_POSTGRES_PASSWORD=user123<br><br>// Via setup method<br>SetPostgresConnectOpts(&amp;egorm.PostgresConnectOpts{<br>  Host     &quot;postgres&quot;,<br> Port     5432,<br> Database &quot;postgres&quot;,<br> User     &quot;user&quot;,<br> Password &quot;user123&quot;,<br>})</pre><p>Thats all. We are now ready to perform any operation on the configured database.</p><h4>Insert into Database</h4><p>Let’s go through the common CRUD case and have a look how EGorm implements all these cases. We start with inserting entities into the database using EGorms DbCreatemethod:</p><pre>type SampleStruct struct {<br>  gorm.Model<br>  Name string<br>}<br><br>func insertSample() error {<br>  sample1 := SampleStruct{<br>   Name: &quot;Sample1&quot;,<br>  }<br>  err := egorm.DbCreate(&amp;sample1)<br>  if err != nil {<br>   return err<br>  }<br>  return nil<br>}</pre><p>It takes the reference to a struct as argument that it will insert into the database. After inserting, the struct will have a primary key in case the insert was successful, otherwise an error will be returned. Comparing this to gorm’s documentation may let you think that there is no real benefit of using egorm instead of gorm, both operations look quite similar. If you have done the proper database setup and migrations for gorm, then this is true. But egorm does all of this automatically. Let’s have a look at the internal implementation:</p><pre>func DbCreate[T any](input *T) error {<br>  if err := InitDB(); err != nil {<br>   return err<br>  }<br>  err := autoMigrate(input)<br>  if err != nil {<br>   return err<br>  }<br>  result := Db.Create(input)<br>  if result.Error != nil {<br>   return result.Error<br>  }<br>  return nil<br>}</pre><p>It performs 3 important steps:</p><ol><li>The InitDB() call ensures that the database setup is done correctly (it is ensured that it is done exactly once).</li><li>The automigrate(input) call checks if the passed type was already migrated, if not it will perform gorms automigration .</li><li>Now the actual insert operation using an obtained gorm.DB instance takes place.</li></ol><h4>Retrieving Entries from the Database</h4><p>In the next code fragment we show how to fetch all entries from the database using egorms DbGetAll method.</p><pre>type SampleStruct struct {<br> gorm.Model<br> Name string<br>}<br><br>func getAllSamples() ([]SampleStruct, error) {<br>  var samples []SampleStruct<br>  err := DbGetAll(&amp;samples)<br>  if err != nil {<br>    return nil, err<br>  }<br>  return samples, nil<br>}</pre><p>The DbGetAll method is also performing automigrations if required in the background to ensure consistency in the database.</p><p>Egorm also provides simple filter methods using a map, these can be applied for DbGet and DbFind to filter according to specific properties of the entries. The filter currently only supports to compare via equals, but we plan to extend it soon. In the following code sample we show how to filter entries using these two methods:</p><pre>func firstSample() (*SampleStruct, error) {<br>  var sample SampleStruct<br>  err := DbFirst(&amp;sample, Where{&quot;Name&quot;: &quot;Sample2&quot;})<br>  if err != nil {<br>   return nil, err<br>  }  <br>  return &amp;sample, nil<br>}<br><br><br>func getSamples() ([]SampleStruct, error) {<br>  var samples []SampleStruct<br>  err := DbGet(&amp;samples, Where{&quot;Name&quot;: &quot;Sample2&quot;})<br>  if err != nil {<br>   return nil, err<br>  }<br>  return samples, nil<br>}</pre><p>You can find examples for all provided methods in the <a href="https://github.com/martenwallewein/easy-going/blob/master/pkg/egorm/layer_test.go">tests we implemented for egorm</a>.</p><h3>Conclusions</h3><p>In this article, we show a simple but powerful wrapper of gorm, called egorm. It reduces boilerplate code by allowing to call database operations anywhere, while only a setup call or configuration via environment variables is required. Although the available features of egorm are quite sparse at the moment, we observed great developer experience so far when using egorm, and we plan to extend the featureset soon. In any case, you can access the gorm.DB struct that is used under the hood via egorm.Db.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d0e4a33a4e7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Cryptography in Go: Signing vs Encryption]]></title>
            <link>https://medium.com/better-programming/exploring-cryptography-in-go-signing-vs-encryption-f19534334ad?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/f19534334ad</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[cryptography]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[go]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Tue, 25 Apr 2023 14:04:07 GMT</pubDate>
            <atom:updated>2023-06-05T09:57:27.997Z</atom:updated>
            <content:encoded><![CDATA[<h4>A theoretical guide with practical examples</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rIB19v7nj0vr00QgD_bVrg.jpeg" /><figcaption>Photo from <a href="https://unsplash.com/@markusspiske?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Markus Spiske</a> on <a href="https://unsplash.com/de/fotos/iar-afB0QQw?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>This article aims to delve into the world of cryptography using the Go programming language, focusing on signing and encryption. We will explore the differences between these two cryptographic processes and discuss how they contribute to data security and integrity.</p><p>The article will cover both asymmetric cryptographies, using the RSA algorithm, and symmetric cryptography, illustrating how these techniques can be effectively employed in Go. We provide insights into the trade-offs between asymmetric and symmetric cryptography, enabling you to make informed decisions when designing secure systems.</p><p>Please note that despite the code examples, this article is quite theoretical. If you’re more of a practical learner, feel free to scroll down to the last section, in which we provide a <a href="https://github.com/martenwallewein/easy-going/tree/master/examples/cryptomessages">real-world example</a> of sending encrypted and verified messages over the network. No worries, we don’t blame anyone who skips the theory to dive directly into practice.</p><p>However, if you’re interested in the theoretical background, let’s begin our journey into the fascinating world of cryptography and learn how to harness its power using Go.</p><h3>Understanding Signing and Encryption</h3><p>Before diving into the implementation details, let’s first establish a clear understanding of signing and encryption and the differences between these two cryptographic processes.</p><h4>Signing</h4><p>Signing, also known as digital signing, is a technique used to ensure data integrity and non-repudiation. In other words, it guarantees that the data has not been tampered with during transmission and that the sender cannot deny having sent the message. Signing involves generating a digital signature using a private key, which is then appended to the data.</p><p>The message&#39;s recipient can use the corresponding public key to verify the digital signature. If the verification is successful, it confirms that the data has not been altered and originated from the sender. In this way, signing helps establish trust between the sender and the recipient.</p><h4>Encryption</h4><p>Encryption transforms data into an unreadable format, ensuring data confidentiality. The primary goal of encryption is to protect sensitive information from unauthorized access during storage or transmission. To achieve this, encryption uses a specific algorithm and a key to convert plaintext data into ciphertext.</p><p>Only the intended recipient, who possesses the appropriate key, can decrypt the ciphertext and access the original data. Depending on the type of cryptography used, the same key (symmetric cryptography) or a different key (asymmetric cryptography) may be used for decryption.</p><h3>Differences Between Signing and Encryption</h3><p>Although both signing and encryption are essential components of secure communication, they serve different purposes:</p><ol><li>Purpose: Signing ensures data integrity and non-repudiation, while encryption guarantees data confidentiality.</li><li>Process: Signing generates a digital signature using a private key, while encryption transforms data using an encryption algorithm and a key.</li><li>Key usage: In asymmetric cryptography, signing involves the sender’s private and recipient’s keys for verification. Encryption uses the sender’s public key for encryption and the recipient’s private key for decryption.</li></ol><p>In summary, while signing focuses on verifying the authenticity and integrity of data, encryption aims to protect the confidentiality of data by rendering it unreadable to unauthorized parties.</p><h3>Asymmetric Cryptography (RSA)</h3><p>Asymmetric cryptography, also known as public-key cryptography, uses a pair of keys — public and private — for secure communication. The public key is freely shared, while the private key remains confidential to its owner. This section will explore how asymmetric cryptography can be used for digital signing and encryption in Go, focusing on the widely-used RSA algorithm.</p><h4>Introduction to asymmetric cryptography</h4><ol><li>Public and private key pairs: In asymmetric cryptography, each participant has a unique key pair consisting of public and private keys. The public key can be openly distributed, while the private key must be kept secret.</li><li>Key generation, distribution, and management: Asymmetric cryptography relies on secure key generation, distribution, and management processes to maintain security. Public keys must be easily accessible, while private keys need to be securely stored and protected.</li></ol><h3>Digital Signing With RSA in Go</h3><ol><li>Generating an RSA key pair: The first step in using RSA for signing is to generate a key pair. Go’s crypto/rsa package provides the necessary functionality to create RSA keys.</li><li>Creating a digital signature with the private key: Once the key pair is generated, the sender can sign the data using their private key. The crypto/rsa package, along with the crypto/rand and crypto/sha256 packages can be used to create a digital signature.</li><li>Verifying the signature with the public key: The recipient can then verify the signature using the sender’s public key. Successful verification confirms the authenticity and integrity of the data.</li><li>Example code and explanation: In this section, we will walk through a practical example of implementing digital signing with RSA in Go, explaining each step in detail.</li></ol><h3>Encryption With RSA in Go</h3><ol><li>Encrypting data with the public key: The sender uses the recipient’s public key to encrypt data using RSA. The crypto/rsa package provides functions for RSA encryption in Go.</li><li>Decrypting data with the private key: The recipient can then decrypt the encrypted data using their private key, ensuring that only the intended recipient can access the original data.</li></ol><h4>Example: Asymmetric cryptography in action</h4><p>Here we have an example demonstrating signing and encryption using RSA in Go, including key generation:</p><pre>package main<br><br>import (<br> &quot;crypto/rand&quot;<br> &quot;crypto/rsa&quot;<br> &quot;crypto/sha256&quot;<br> &quot;crypto/x509&quot;<br> &quot;encoding/pem&quot;<br> &quot;fmt&quot;<br> &quot;io/ioutil&quot;<br> &quot;log&quot;<br> &quot;os&quot;<br>)<br><br>func main() {<br> // Generate RSA key pair<br> privateKey, err := rsa.GenerateKey(rand.Reader, 2048)<br> if err != nil {<br>  log.Fatalf(&quot;Error generating key pair: %v&quot;, err)<br> }<br> publicKey := &amp;privateKey.PublicKey<br><br> // Save private key as PEM<br> privateKeyBytes := x509.MarshalPKCS1PrivateKey(privateKey)<br> privateKeyPEM := pem.EncodeToMemory(&amp;pem.Block{<br>  Type:  &quot;RSA PRIVATE KEY&quot;,<br>  Bytes: privateKeyBytes,<br> })<br> err = ioutil.WriteFile(&quot;private_key.pem&quot;, privateKeyPEM, 0600)<br> if err != nil {<br>  log.Fatalf(&quot;Error saving private key: %v&quot;, err)<br> }<br><br> // Save public key as PEM<br> publicKeyBytes, err := x509.MarshalPKIXPublicKey(publicKey)<br> if err != nil {<br>  log.Fatalf(&quot;Error marshaling public key: %v&quot;, err)<br> }<br> publicKeyPEM := pem.EncodeToMemory(&amp;pem.Block{<br>  Type:  &quot;RSA PUBLIC KEY&quot;,<br>  Bytes: publicKeyBytes,<br> })<br> err = ioutil.WriteFile(&quot;public_key.pem&quot;, publicKeyPEM, 0644)<br> if err != nil {<br>  log.Fatalf(&quot;Error saving public key: %v&quot;, err)<br> }<br><br> // Sign data<br> message := []byte(&quot;Hello, world!&quot;)<br> hash := sha256.New()<br> _, err = hash.Write(message)<br> if err != nil {<br>  log.Fatalf(&quot;Error hashing message: %v&quot;, err)<br> }<br> hashedMessage := hash.Sum(nil)<br> signature, err := rsa.SignPKCS1v15(rand.Reader, privateKey, crypto.SHA256, hashedMessage)<br> if err != nil {<br>  log.Fatalf(&quot;Error signing message: %v&quot;, err)<br> }<br><br> // Verify signature<br> err = rsa.VerifyPKCS1v15(publicKey, crypto.SHA256, hashedMessage, signature)<br> if err != nil {<br>  log.Fatalf(&quot;Error verifying signature: %v&quot;, err)<br> } else {<br>  fmt.Println(&quot;Signature verified.&quot;)<br> }<br><br> // Encrypt data<br> ciphertext, err := rsa.EncryptPKCS1v15(rand.Reader, publicKey, message)<br> if err != nil {<br>  log.Fatalf(&quot;Error encrypting message: %v&quot;, err)<br> }<br><br> // Decrypt data<br> plaintext, err := rsa.DecryptPKCS1v15(rand.Reader, privateKey, ciphertext)<br> if err != nil {<br>  log.Fatalf(&quot;Error decrypting message: %v&quot;, err)<br> }<br><br> if string(plaintext) == string(message) {<br>  fmt.Println(&quot;Decrypted message matches original.&quot;)<br> }<br>}</pre><p>This example demonstrates the following steps:</p><ol><li>Generate an RSA key pair with a 2048-bit key size.</li><li>Save the private and public keys as PEM files.</li><li>Sign a message using the private key and SHA-256 as the hash function.</li><li>Verify the signature using the public key.</li><li>Encrypt the message using the public key.</li><li>Decrypt the encrypted message using the private key.</li></ol><h3>Advantages and Disadvantages of RSA</h3><p>Asymmetric cryptography, particularly RSA, offers several advantages, such as enhanced security due to using separate keys for signing/encryption and decryption/verification. It also enables secure key distribution, as public keys can be freely shared without compromising the private keys.</p><p>However, RSA also has its drawbacks, including slower performance than symmetric cryptography, especially for large data sets. Additionally, key management can become complex in large systems, as each participant requires a unique key pair.</p><p>In the next section, we will explore symmetric cryptography and its applications in signing and encryption, as well as its advantages and disadvantages in comparison to RSA.</p><h3>Symmetric Cryptography</h3><p>Symmetric cryptography, also known as secret-key cryptography, relies on a single key for encryption and decryption. This shared key must be securely distributed to all parties involved in the communication. In this section, we will discuss symmetric cryptography, its applications in signing and encryption, and its advantages and disadvantages compared to asymmetric cryptography.</p><h3>Introduction to Symmetric Cryptography</h3><ol><li>Single key for encryption and decryption: In symmetric cryptography, the same key is used to encrypt and decrypt data. This key must be securely exchanged between the sender and the recipient before initiating communication.</li><li>Key distribution and management: One of the main challenges in symmetric cryptography is securely distributing the shared key to all parties involved. Effective key management processes are crucial to maintaining the security of symmetric cryptography-based systems.</li></ol><h4>Symmetric encryption in Go</h4><ol><li>Popular algorithms: Symmetric encryption algorithms, such as Advanced Encryption Standard (AES), are widely used due to their efficiency and security. AES is a block cipher that supports key sizes of 128, 192, and 256 bits.</li><li>Generating a symmetric key: To use symmetric encryption, a shared key must be generated and securely distributed to all parties. In Go, the crypto/rand package can be used to generate cryptographically secure random keys.</li><li>Encrypting and decrypting data with the key: With the shared key in place, the sender can encrypt data using the chosen symmetric encryption algorithm. The recipient can then decrypt the data using the same key. Go’s crypto/cipher package provides functions to perform encryption and decryption using symmetric algorithms like AES.</li></ol><h3>Digital Signing with Symmetric Cryptography</h3><ol><li>Message authentication codes (MACs): In symmetric cryptography, data integrity, and authentication are achieved using message authentication codes (MACs). A MAC is generated using a shared secret key and the data and is then sent along with the message.</li><li>Generating a MAC with a symmetric key: Go’s crypto/hmac package can be used to generate MACs using symmetric keys and hash functions like SHA-256.</li><li>Verifying the MAC: Upon receiving the message and the MAC, the recipient can use the shared key to generate their own MAC for the received data. If the recipient’s MAC matches the received MAC, it confirms the data’s integrity and authenticity.</li></ol><h4>Example: Symmetric cryptography in action</h4><p>Here we see an example demonstrating signing with HMAC and encryption using AES in Go:</p><pre>package main<br><br>import (<br> &quot;crypto/aes&quot;<br> &quot;crypto/cipher&quot;<br> &quot;crypto/hmac&quot;<br> &quot;crypto/rand&quot;<br> &quot;crypto/sha256&quot;<br> &quot;encoding/base64&quot;<br> &quot;fmt&quot;<br> &quot;io&quot;<br> &quot;log&quot;<br>)<br><br>func main() {<br> // Generate a random symmetric key for HMAC and AES<br> key := make([]byte, 32)<br> if _, err := io.ReadFull(rand.Reader, key); err != nil {<br>  log.Fatalf(&quot;Error generating symmetric key: %v&quot;, err)<br> }<br><br> // Sign data with HMAC-SHA256<br> message := []byte(&quot;Hello, world!&quot;)<br> mac := hmac.New(sha256.New, key)<br> _, err := mac.Write(message)<br> if err != nil {<br>  log.Fatalf(&quot;Error signing message: %v&quot;, err)<br> }<br> signature := mac.Sum(nil)<br><br> // Verify HMAC signature<br> mac2 := hmac.New(sha256.New, key)<br> _, err = mac2.Write(message)<br> if err != nil {<br>  log.Fatalf(&quot;Error signing message for verification: %v&quot;, err)<br> }<br> expectedSignature := mac2.Sum(nil)<br> if hmac.Equal(signature, expectedSignature) {<br>  fmt.Println(&quot;HMAC signature verified.&quot;)<br> } else {<br>  log.Fatal(&quot;HMAC signature verification failed.&quot;)<br> }<br><br> // Encrypt data with AES-GCM<br> block, err := aes.NewCipher(key)<br> if err != nil {<br>  log.Fatalf(&quot;Error creating AES cipher: %v&quot;, err)<br> }<br> nonce := make([]byte, 12)<br> if _, err := io.ReadFull(rand.Reader, nonce); err != nil {<br>  log.Fatalf(&quot;Error generating nonce: %v&quot;, err)<br> }<br> aesgcm, err := cipher.NewGCM(block)<br> if err != nil {<br>  log.Fatalf(&quot;Error creating GCM: %v&quot;, err)<br> }<br> ciphertext := aesgcm.Seal(nil, nonce, message, nil)<br> ciphertext = append(nonce, ciphertext...)<br><br> // Decrypt data with AES-GCM<br> nonce, ciphertext = ciphertext[:12], ciphertext[12:]<br> block, err = aes.NewCipher(key)<br> if err != nil {<br>  log.Fatalf(&quot;Error creating AES cipher for decryption: %v&quot;, err)<br> }<br> aesgcm, err = cipher.NewGCM(block)<br> if err != nil {<br>  log.Fatalf(&quot;Error creating GCM for decryption: %v&quot;, err)<br> }<br> plaintext, err := aesgcm.Open(nil, nonce, ciphertext, nil)<br> if err != nil {<br>  log.Fatalf(&quot;Error decrypting data: %v&quot;, err)<br> }<br><br> if string(plaintext) == string(message) {<br>  fmt.Println(&quot;Decrypted message matches original.&quot;)<br> }<br>}</pre><p>This example demonstrates the following steps:</p><ol><li>Generate a random symmetric key for HMAC and AES.</li><li>Sign a message using HMAC-SHA256 and the symmetric key.</li><li>Verify the HMAC signature using the symmetric key.</li><li>Encrypt the message using AES-GCM and the symmetric key.</li><li>Decrypt the encrypted message using AES-GCM and the symmetric key.</li></ol><h3><strong>Advantages and Disadvantages of Symmetric Cryptography</strong></h3><p>Symmetric cryptography offers several benefits, including faster performance and less computational overhead than asymmetric cryptography, making it suitable for encrypting large amounts of data.</p><p>However, symmetric cryptography also challenges key distribution and management, as securely exchanging the shared key can be difficult. Furthermore, symmetric cryptography does not inherently provide non-repudiation, as the same key is used by all parties involved.</p><p>In the next section, we will discuss factors to consider when choosing the appropriate cryptographic approach for your use case and explore how asymmetric and symmetric cryptography can be combined for optimal performance and security.</p><h3>Choosing the Right Cryptographic Approach</h3><p>Selecting the appropriate cryptographic approach for your use case is essential to balancing security, performance, and complexity. In this section, we will discuss factors to consider when choosing between asymmetric and symmetric cryptography and how to combine the two for optimal results.</p><h3>Factors to Consider</h3><ol><li>Data size and performance: Symmetric cryptography offers faster performance and is better suited for encrypting large amounts of data, while asymmetric cryptography is generally slower due to the complex mathematical operations involved.</li><li>Key management: Asymmetric cryptography simplifies key management in large systems, as each participant requires a unique key pair. In contrast, symmetric cryptography involves securely distributing and managing a shared key among all parties, which can be challenging.</li><li>Non-repudiation: Asymmetric cryptography provides non-repudiation, meaning the sender cannot deny sending a message after signing it. Symmetric cryptography does not inherently offer this feature since all parties use the same key.</li><li>Secure key distribution: Asymmetric cryptography enables secure key distribution, as public keys can be openly shared without compromising the private keys. Symmetric cryptography requires a secure method for exchanging the shared key.</li></ol><h3>Hybrid Cryptography</h3><p>In many practical scenarios, a hybrid approach combining asymmetric and symmetric cryptography is used to leverage both strengths. This approach typically involves:</p><ol><li>Key exchange: Asymmetric cryptography, such as RSA or Elliptic Curve Cryptography (ECC), is used to exchange symmetric keys between parties securely.</li><li>Data encryption: Symmetric cryptography, such as AES, is used to encrypt and decrypt data due to its faster performance and efficiency.</li><li>Digital signatures: Asymmetric cryptography is also used for signing and verifying messages, providing non-repudiation, and ensuring data integrity.</li></ol><h3>Assessing Your Needs</h3><p>When choosing a cryptographic approach, it is essential to assess the specific requirements of your use case. Consider the following questions:</p><ol><li>What are the performance and latency requirements for your application?</li><li>How large is the data you need to encrypt or sign?</li><li>Is non-repudiation a critical requirement?</li><li>How complex is your system, and how many parties are involved in communication?</li></ol><p>By evaluating these factors, you can decide whether asymmetric cryptography, symmetric cryptography, or a hybrid approach is best suited for your needs.</p><h3>A More Interesting Example</h3><p>While the previous examples look quite constructed, let’s look at a more realistic example: Sending a message over the network securely: combining encryption and signing. In the following image, we show the process of this example using RSA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*ljZkinNzYISrfrNqvdHVLw.jpeg" /><figcaption>Image by author</figcaption></figure><p>The sender and receiver have a keypair of private and public keys that both sides know and trust. At first, the sender encrypts the message using the receiver&#39;s public key. Afterward, the sender signs the message using its own private key and appends the signature to the encrypted message. Next, the sender sends the message, including the signature, over the network to the receiver. The receiver now splits the received data into the encrypted message and the signature.</p><p>With its own private key, it can now decrypt the encrypted message, and using the sender&#39;s public key, it can verify the signature. Please note that the same goal could be done by using symmetric cryptography. By combining these two approaches, we achieve the following:</p><ol><li>Confidentiality: The sender can ensure that only the receiver can decrypt the message.</li><li>Authentication: The receiver can ensure that only the sender could have sent this message.</li></ol><p>The respective code of this example, supporting HMAC and RSA modes, can be found in <a href="https://github.com/martenwallewein/easy-going/tree/master/examples/cryptomessages">this GitHub repository</a>. Feel free to try it out and play with encryption and signing in Go.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f19534334ad" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/exploring-cryptography-in-go-signing-vs-encryption-f19534334ad">Exploring Cryptography in Go: Signing vs Encryption</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Simple Cross-Platform SSH Client in 100 Lines of Go]]></title>
            <link>https://medium.com/better-programming/a-simple-cross-platform-ssh-client-in-100-lines-of-go-280644d8beea?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/280644d8beea</guid>
            <category><![CDATA[go]]></category>
            <category><![CDATA[crypto]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[ssh]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Fri, 21 Apr 2023 15:53:52 GMT</pubDate>
            <atom:updated>2023-06-05T09:59:18.591Z</atom:updated>
            <content:encoded><![CDATA[<h4>SSH in Go: Simple and powerful</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qJkwEYce4VZbDkhMocsnjA.jpeg" /><figcaption>Foto von <a href="https://unsplash.com/pt-br/@adigold1?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Adi Goldstein</a> auf <a href="https://unsplash.com/de/fotos/EUsVwEOsblE?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Go is a programming language designed for building fast, efficient, and reliable software. Google created it in 2007, and it has become a popular choice for a wide range of applications, including system administration tasks.</p><p>One of the benefits of Go is its simplicity and ease of use. Its clean and concise syntax makes it easy to learn and use, even for beginners. Additionally, Go is a compiled language, producing binary executables that can be run on any platform without needing an interpreter. What makes Go really powerful is its ability to compile statically linked binaries by just snapping your fingers (e.g., by setting CGO_ENABLED=0)</p><h3>The Importance of Executing SSH Commands</h3><p>Executing commands on remote servers via SSH is a common task in system administration. SSH is a secure and reliable protocol for accessing remote machines and performing administrative tasks. By executing commands remotely, system administrators can manage multiple machines from a central location, reducing the need for manual intervention.</p><h3>The Need for Cross-Platform SSH Support</h3><p>While SSH is a powerful tool, it can be challenging to integrate it into an application, particularly when working with different platforms. For example, some platforms may have different SSH client libraries or configurations, making it difficult to execute SSH commands reliably across all platforms.</p><p>This is where implementing an SSH client in Go can be useful. Go is a cross-platform language, meaning an application written in Go can be compiled to run on different operating systems and architectures. Implementing an SSH client in Go ensures that your application works consistently across all platforms. So definitely a great choice if you plan to integrate ssh into a cross-platform application. Now, let’s get started!</p><h3>Implementing an SSH Client in Go</h3><p>In this article, I’ll show you how to implement an SSH client in Go and execute commands on a remote server. My implementation will use the golang.org/x/crypto/ssh package, which provides a complete implementation of the SSH protocol in Go.</p><p>First, let’s start with the SSH client configuration. We start our implementation by loading an ssh config file (usually in ~/.ssh/config) and search for a specific host to connect to. This config file is a great place to configure users, ports, IPs, and jumphosts to connect to particular servers. To load the config and obtain values from it, we use the github.com/kevinburke/ssh_configmodule. It provides a method called Decode that takes an io.Reader as input and returns an ssh_config.Config object, which we can filter through, e.g., the Get method that accepts a host alias and a key to which property should be accessed.</p><pre>// Load user&#39;s SSH config file<br>f, err := os.Open(os.ExpandEnv(&quot;$HOME/.ssh/config&quot;))<br>sshConfig, err := ssh_config.Decode(f)<br>if err != nil {<br> log.Fatalf(&quot;Failed to load SSH config: %s&quot;, err)<br>}<br><br>// Get the host configuration<br>user, err := sshConfig.Get(host, &quot;User&quot;)<br>if err != nil {<br> log.Fatalf(&quot;Failed to get ssh User from config: %s&quot;, err)<br>}<br><br>hostName, err := sshConfig.Get(host, &quot;HostName&quot;)<br>if err != nil {<br> log.Fatalf(&quot;Failed to get ssh HostName from config: %s&quot;, err)<br>}<br><br>port, err := sshConfig.Get(host, &quot;Port&quot;)<br>if err != nil {<br> log.Fatalf(&quot;Failed to get ssh Port from config: %s&quot;, err)<br>}</pre><p>Next, we create an ssh.ClientConfig struct that contains the authentication information for the SSH client, such as the user, password, and private key. Here&#39;s an example of how to create an ssh.ClientConfig:</p><pre>// Load private key for authentication<br>keyPath := os.ExpandEnv(&quot;$HOME/.ssh/id_rsa&quot;)<br>keyBytes, err := ioutil.ReadFile(keyPath)<br>if err != nil {<br> log.Fatalf(&quot;Failed to read private key: %s&quot;, err)<br>}<br><br>key, err := ssh.ParsePrivateKeyWithPassphrase(keyBytes, []byte(&quot;password&quot;))<br>if err != nil {<br> log.Fatalf(&quot;Failed to parse private key: %s&quot;, err)<br>}<br><br>// Or without password<br>//key, err := ssh.ParsePrivateKey(keyBytes)<br>//if err != nil {<br>// log.Fatalf(&quot;Failed to parse private key: %s&quot;, err)<br>//}<br><br>// Configure SSH client<br>sshClientConfig := &amp;ssh.ClientConfig{<br> User: user,<br> Auth: []ssh.AuthMethod{<br>  ssh.PublicKeys(key),<br> },<br> HostKeyCallback: ssh.InsecureIgnoreHostKey(),<br> Timeout:         5 * time.Second,<br>}</pre><p>In this sample, we read the private key file (either password protected or not) and use public-key-auth as the Auth method for our ssh client.</p><p>Next, we’ll create an SSH client connection using the ssh.Dial function. This function takes the network type (tcp), the host address, and the ssh.ClientConfig as arguments. Here&#39;s an example:</p><pre>// Connect to the host<br>address := net.JoinHostPort(hostName, port)<br>client, err := ssh.Dial(&quot;tcp&quot;, address, sshClientConfig)<br>if err != nil {<br> log.Fatalf(&quot;Failed to connect to %s: %s&quot;, host, err)<br>}<br><br>fmt.Printf(&quot;Connected to %s\n&quot;, host)<br>defer client.Close()</pre><p>In this example, we’re connecting to the hostName and port we choose. We&#39;re also using the defer keyword to ensure the connection is closed when the function returns.</p><p>Once we connect, we can create an SSH session using the conn.NewSession function. This function returns an ssh.Session object, which we can use to execute commands on the remote server. Here&#39;s an example:</p><pre>// Create an SSH session<br>session, err := client.NewSession()<br>if err != nil {<br> log.Fatalf(&quot;failed to create session: %s&quot;, err)<br>}<br>defer session.Close()<br><br>// Run the command on the remote machine<br>stdout, err := session.StdoutPipe()<br>if err != nil {<br> log.Fatalf(&quot;failed to get stdout: %s&quot;, err)<br>}<br>if err := session.Start(command); err != nil {<br> log.Fatalf(&quot;failed to start command: %s&quot;, err)<br>}</pre><p>Finally, we’re reading the output of the command from the SSH session’s stdout stream using the ioutil.ReadAll function, and printing it on the console, as shown below:</p><pre>output, err := ioutil.ReadAll(stdout)<br>if err != nil {<br> log.Fatalf(&quot;failed to read stdout: %s&quot;, err)<br>}<br>if err := session.Wait(); err != nil {<br> log.Fatalf(&quot;command failed: %s&quot;, err)<br>}<br>fmt.Println(string(output))</pre><h3>Conclusion</h3><p>This article shows you how to implement an SSH client in Go and execute commands on a remote server. By using the golang.org/x/crypto/ssh package and Go&#39;s cross-platform capabilities, we can ensure our SSH client works reliably on different platforms.</p><p>We hope this article has helped demonstrate the power and flexibility of Go for system administration tasks. With its simplicity, ease of use, and cross-platform support, Go is an excellent choice for building robust and efficient software for managing remote systems.</p><p>In the following, the complete code takes two cmd arguments: a host alias (of the ssh config file) and a command to execute on the remote machine. The whole example, in combination with its instructions, can be found <a href="https://github.com/martenwallewein/easy-going/tree/master/examples/ssh">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=280644d8beea" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/a-simple-cross-platform-ssh-client-in-100-lines-of-go-280644d8beea">A Simple Cross-Platform SSH Client in 100 Lines of Go</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Favourite Setup for REST Microservices in Go]]></title>
            <link>https://medium.com/better-programming/my-favourite-setup-for-rest-microservices-in-go-770ca18615ba?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/770ca18615ba</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[go]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Thu, 27 Oct 2022 03:17:32 GMT</pubDate>
            <atom:updated>2023-06-05T09:58:47.207Z</atom:updated>
            <content:encoded><![CDATA[<h4>A template for microservices in Go</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2zaVQgYpZO5yUjkR_uZflQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@ffstop?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Fotis Fotopoulos</a> on <a href="https://unsplash.com/s/photos/programming?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>In this story, I want to share my personal template for microservices that serve REST APIs in Go. Since I’ve started to write a few services from scratch, observing that particular code parts always repeat, I decided to export them to a template and share it.</p><p>While some parts may be adapted in case other frameworks are preferred, I hope the overall structure can help in implementing your next project in an easier and faster way.</p><p>In this story, I present an example Project-Service, which offers API endpoints to manage projects, milestones, and tasks.</p><h3>Folder Structure</h3><p>First, I want to share my preferred folder structure, where you can find all the files to run the example project-service.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5e9051502606c16de1911ea845e40df6/href">https://medium.com/media/5e9051502606c16de1911ea845e40df6/href</a></iframe><ul><li>api/ contains one or more versions of the REST API</li><li>dbs/ contains the code to initialize database connections and perform migrations</li><li>models/ contains all model definitions and potential model-related logic</li><li>pkg/**/* contains the service logic for the models</li><li>seeds/ contains the code to initially seed the database with some data</li></ul><h3>Models</h3><p>To simplify database operations, ORMs are a pretty neat choice. I personally prefer <a href="https://gorm.io/">gorm</a>. Gorm works through tag reflection, meaning you add tags to your model’s fields and gorm will process them. A simple model that can be processed by gorm looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5a5842413ce363474e5f9b662b687df5/href">https://medium.com/media/5a5842413ce363474e5f9b662b687df5/href</a></iframe><p>Despite being super flexible, gorm provides simple usage with strong defaults. By extending the gorm.Model struct, your model will get an ID field that is used as an identifier (by default it will be an integer with auto increment).</p><p>Untagged fields are translated to snake case column names. Via field tags, we can configure gorm’s behavior, in this case, we just add an index to the name column.</p><p>Other models can be referenced by using their struct as field, or even as slices. Our service has three models: Project, Milestone, Task . One project has one or more milestones, each milestone may have one or more tasks.</p><p>I would propose creating a separate .go file for each model, which makes it easier to maintain if the models implement also other interfaces or have for example specific (de)serialization logic.</p><p>Finally, I always add one migrate.go file in the models folder, which contains the automigration code. With gorm, this is a no-brainer if you follow the guides, you can simply put references to all your models into gorm.AutoMigrate():</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/97ad182dd4b104551d9e637450d303e9/href">https://medium.com/media/97ad182dd4b104551d9e637450d303e9/href</a></iframe><h3>Database Initialization</h3><p>The next part of my template is the logic to setup database connections and to perform initial operations like migrations. In the dbs folder, I have one file called setup.go that runs the code to initialize the preferred database connection. The code looks like the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/af9d3ea6e84d2d6750893e93448c1214/href">https://medium.com/media/af9d3ea6e84d2d6750893e93448c1214/href</a></iframe><p>The code is straightforward, there is one InitializeDatabaseLayer function that passes the DB environment variable and checks, which database driver should be used. In this sample, SQLite for development and Postgres for production setups are implemented, but it can be easily extended to all other supported gorm drivers.</p><p>For each of the two drivers, I’ve added a method that loads the required information from the environment and connects to the database. Afterward, the presented automigrate method is called to ensure, that the database is in sync with the models. Finally, an instance of gorm.DB will be returned to the application. I like to save this instance in a variable and export a function to obtain it, in this case, the GetDB method.</p><h3>Service instances to perform database logic</h3><p>Now that the database connection is set up, the next part to consider is the actual database-related logic. In my showcase, I have CRUD operations for all three models that need to be implemented. In my template, I prefer to create a folder for each model and locate all related implementations there, beginning with a service.go file. The following code shows this file for the projects:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b413b3315765707aa4fe106fd2889894/href">https://medium.com/media/b413b3315765707aa4fe106fd2889894/href</a></iframe><p>The service implementation starts with a ProjectService struct and a singleton instance of it, that will be returned from calling projects.GetService(). The service itself contains the logic to perform database operations for the Project model. The basic CRUD operations are implemented as dedicated methods using gorm’s DB instance that was initialized in the setup.go file. Since gorm gives us a nice featureset, implementing these CRUD operations is straightforward. TODO: may reuse in different APIs</p><p>For the remaining models, the service implementations look similar. While this looks quite verbose for our three models and their simple logic, I think it will provide a huge benefit when your codebase grows, having things separated and implemented that way.</p><h3>REST Endpoints using Gin Gonic</h3><p>After implementing the core database-related logic, we need to expose it via REST API. Therefore, I mostly create an api folder that contains versioned APIs, e.g. in this simple case v1.go, in more complex APIs it might also be clever to create dedicated folders for each version.</p><p>The API with its specific version number has always its own URL context, e.g. /v1/* for version1. The API code is comparably simple since our core logic is implemented in services:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5272fae680a112dbf9abcc3e01cc54cd/href">https://medium.com/media/5272fae680a112dbf9abcc3e01cc54cd/href</a></iframe><p>I implement each API version by using a struct, e.g. RESTAPIV1 for the REST API version 1. As an HTTP framework, I personally like to use <a href="https://gin-gonic.com/">Gin Gonic</a>, since it is straightforward and powerful.</p><p>I put the gin router as a field in the API struct, and add Serve method, which can be called from the main program.</p><p>In the constructor NewRestApiV1, I instantiate this router and register all routers, via e.g. router.POST or router.GET etc. Parameters can be simply configured via :paramName in routes. As we can see in the EditProject method (which gets a reference to the gin.Context as the only parameter), the param can be fetched via c.Param(&quot;paramName&quot;). Gin Gonic provides a great developer experience for working with JSON, through the ShouldBindJSON method.</p><p>Finally, return codes and also JSON responses are easily possible via c.JSON and the shorthand gin.H which can be used as a generic map.</p><p>While REST via Gin Gonic is only one example, many other combinations are possible, e.g. GRPC or Websockets, or simply using another HTTP library if preferred.</p><h3>Seeds</h3><p>While we can sync our model representation to the database via migration, it’s often very helpful to add initial data to the database, for example, an admin user. This avoids connecting with an external management tool to the database to add this data manually.</p><p>I prefer to pass the path to a seed file as a command line argument to the service, and when this file exists and the database is empty, the service will load the seed file and insert all the contained data.</p><p>To make editing easy, I prefer to use JSON for the seeds file. The code to run the seeds looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f28b4b8aec96f1668618107f556be5bd/href">https://medium.com/media/f28b4b8aec96f1668618107f556be5bd/href</a></iframe><p>At first, all service instances are obtained. Afterward, the code checks if there are any tasks in the database, and returns the method if this is the case, since we want the seed data to be inserted only a single time.</p><p>If there are no tasks in the database, the code opens the seed, creates a JSON encoder and parses the file, in case it’s valid JSON. The JSON is expected to be a Seeds object, which has projects, milestones and tasks slices as fields. Finally, overall these slices will be iterated to insert the data.</p><p>One important note here is, that the seeded projects could also have milestones and these milestones could also have tasks.</p><h3>The Main Program</h3><p>After presenting all parts in detail, the last missing part is the main program, which is stored in main.go :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6dd1b90c1ee77e3ad4b4a612a6cfa8b0/href">https://medium.com/media/6dd1b90c1ee77e3ad4b4a612a6cfa8b0/href</a></iframe><p>The main program is quite short since this template has dedicated folders for all core parts of the microservice. It starts with setting some command line flags via Go’s internal flag package. The flags are the following: a local address laddr on which the HTTP endpoints listen, the logleveland a path to the initial seed file initialSeedFile. I like to use <a href="https://github.com/sirupsen/logrus">logrus</a> for logging, which accepts a list of different loglevels. The configureLogging method parses the loglevel from the flag and sets it accordingly, in addition to some preferred formatting settings. The main function calls configureLogging, initializes the database layer (including migrations), then runs seeds if initialSeedFile is set, and then creates the REST API and serves it under the configured laddr.</p><h3>Dockerfile</h3><p>Finally, I want to shortly present the Dockerfile that I always add to my Go services, to deploy them as containers. It’s a multi-stage dockerfile based on alpine, which creates comparably small containers.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a6f0e52071f3fcc0e54d3af94b0f64a4/href">https://medium.com/media/a6f0e52071f3fcc0e54d3af94b0f64a4/href</a></iframe><p>It starts using the 1.18 Golang alpine container, copies the current folder into it, and builds it via CGO_ENABLED=0 go build.</p><p>While the CGO_ENABLED=0part may not be necessary here, if all stages are based on similar alpine versions, it helps to avoid issues when changing the first or second-stage base containers.</p><p>After building, I use the default alpine image as the second stage, installing ca-certificates (in case you want to communicate to external endpoints that are secured via TLS, this is mandatory), and then copying the built binary from the first stage. The last step is simply setting the docker ENTRYPOINT .</p><p>In this story, I’ve shared my preferred template for microservices in Go. I think it provides a good starting point, which also scales for larger services. I’m happy to get any feedback or recommendations to improve the story.</p><p>The complete code can be found on <a href="https://github.com/martenwallewein/go-sample-microservice">GitHub</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=770ca18615ba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/my-favourite-setup-for-rest-microservices-in-go-770ca18615ba">My Favourite Setup for REST Microservices in Go</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Time-Tracking via the Telegram Bot API and Webhooks]]></title>
            <link>https://medium.com/better-programming/time-tracking-via-the-telegram-bot-api-and-webhooks-8371614f0b13?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/8371614f0b13</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Fri, 19 Aug 2022 14:03:12 GMT</pubDate>
            <atom:updated>2022-08-19T14:03:12.602Z</atom:updated>
            <content:encoded><![CDATA[<h4>How my search for the ideal time tracking tool ended with a surprising result</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YJx0csdADrKUKgkznAkM3w.jpeg" /><figcaption>Image from <a href="https://unsplash.com/photos/nPy0X4xew60">Unsplash</a> by <a href="https://unsplash.com/@bradneathery">Bred Neathery</a></figcaption></figure><p>In this article, I want to present a way to track your time using Telegram’s Bot API and Webhooks. I will guide you through the setup, configuration, and deployment. But first, let’s look at how this idea came to my mind. If you’re only interested in the technical part, skip to the next section.</p><h3>Intro</h3><p>In one of our evening talks, my wife said she needed to write the hourly report for her job for the last three months and tried to find out the exact hours she worked. She didn’t have any tooling to help her track her time, so it was quite a pain to find all these things in old notes and chats. Then she told me she agreed with her boss to make a WhatsApp group where she writes a message when she starts working and when she stops, and based on these messages, it’s easy to fill the respective reports.</p><p>I thought: Quite a nice idea. You have your phone mostly always with you, and it’s not a big deal just to write “start” or “end” or whatever. I was quite impressed and thought about this approach. I spent a lot of hours the last few months to find good tooling to track my time and always had the issue: it needs to be easy to enter that I now work on a task or finish a task, and it must be doable from all devices that I have.</p><p>So I tried some apps for time tracking, which can run on my Mac, Linux machine, and phone, but nothing really knocked my socks off. Back to the idea of how my wife does her time tracking now, I thought that writing down when I start what task and when I finish it in a messenger is a great idea. After a bit of research about Bots and Webhooks, I built some tooling based on telegram.</p><p>My idea was to create a telegram bot and to implement Webhooks for specifically formatted messages, which save timestamps and task descriptions in CSV format. Let’s have a look at how it works.</p><h3>Setting Up the Project</h3><p>At first, I started searching for existing libraries or examples that use Telegram Webhooks and found this one on GitHub: <a href="https://github.com/yagop/node-telegram-bot-api">node-telegram-bot-api</a>. It looked easy to use, and I am a fan of Node, so I decided to try it. Since it provided a library, I needed to set up a node application. I prefer TypeScript over JavaScript, so I set up my project accordingly.</p><p>There is a cool tool called typescript-starter that you can run as <a href="https://github.com/bitjson/typescript-starter">npm executable</a> via npx typescript-starter . It asks you some questions, for example, if you want to build a library or an application, which was the latter for me. After setting up the project by keeping the default values in the setup dialog of typescript-starter, I created a launch configuration for debugging in VS Code. After some experimenting, my final launch configuration looked like the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae3db9e34aa1e7f8f56f198b2886abae/href">https://medium.com/media/ae3db9e34aa1e7f8f56f198b2886abae/href</a></iframe><p>The main part of the code goes into src/main/index.ts. To configure Webhooks for messages sent to a telegram Bot, the node-telegram-bot-api provides a great template, which I adapted slightly. At first, I installed the required packages: <br>- npm i --save node-telegram-bot-api<br>- npm i --save @types/node-telegram-bot-api</p><h4>Telegram bot code</h4><p>Then I changed the imports from the example code from require to import (to get type safety for the imported code) and added my token. The code looked like this and was ready to be tested:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d99338ebcc5eaa3ae6d4f9d7223cadc/href">https://medium.com/media/6d99338ebcc5eaa3ae6d4f9d7223cadc/href</a></iframe><p>Using the on and onText hooks, functions for each incoming message (on) or for messages that match a particular (onText) regex are called. In these functions, the example code answers by sending messages back to the respective chat. The level of abstraction this library provides is great, in my opinion. So I was ready to test the example.</p><p>At first, I created a bot by going to telegram (or web telegram), opening a chat to <a href="https://core.telegram.org/bots#6-botfather">Botfather</a> (follow the t.me link in the doc), writing /newbot, and giving it a name and a username following the requirements. The BotFather printed the token, which needed to be set in the token constant at the beginning of the code. Afterward, I ran the telegram API bot via the launch command in VS Code I added before.</p><p>After opening a new chat to the bot by clicking the link I received from the Botfather and clicking start at the top right, I was able to test the Webhooks by sending /echo Hello to the bot. It should reply with hello (through the onText function) and send another text, “Received your message.”</p><p>Great! Now let’s add some more functionality to the example code.</p><h3>Adding Handlers and Code for Time-Tracking</h3><p>As mentioned before, my goal was to store my time-tracking in the CSV format to allow editing later in Excel. Working with CSV files in Node is quite easy. Despite many libraries doing this task, I decided to write it from scratch, just for fun. For my prototype, I wanted to have the following commands for my bot:<br>- /work $message -&gt; Starts the work on the task $message<br>- /done -&gt; Finishes the last started task<br>- /state -&gt; Shows the state of the last task (started or finished)<br>- /print-&gt; Writes the whole CSV into the chat</p><p>To read and write CSV entries, I implemented the following two functions:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/be18fddfe26c955935e8d45ad3999777/href">https://medium.com/media/be18fddfe26c955935e8d45ad3999777/href</a></iframe><p>The function getEntries returns the CSV content as JS objects, and the writeFile function stores the JS objects into the CSV file. So, there is no real need to use a library. With these two functions, I implemented the four handlers mentioned above:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9c05af1e81be6a87299dd15aa0b2ce56/href">https://medium.com/media/9c05af1e81be6a87299dd15aa0b2ce56/href</a></iframe><p>The work command adds an entry with a startDate and the respective text added to /work . The /done command checks if there is at least one entry in the file and sets the endDate and the duration. The /state and /print handlers work similarly, doing what I presented above. That’s it!</p><p>Not much code, but a pretty cool feeling to use this tool. I ran it in debug mode to test if everything worked as expected. Great. Now let’s head to the final task: deployment.</p><h3>Deployment of the Telegram Time-Tracker</h3><p>I added a Dockerfile with a multi-stage build to run it on any host. You could also pass the token as an env variable or similar for deployment. The easiest way to run it on a server (with docker setup) is to push your repo to the server and just build and run the container (after changing the token to your bot token):</p><ul><li>docker build -t telegram-timetracker:latest</li><li>docker run -ti --rm telegram-timetracker:latest</li></ul><p>One may set the token via an env variable to avoid editing the code, but that’s just a minor optimization. The whole repo can be found <a href="https://github.com/martenwallewein/telegram-timetracker">here</a>. I really like this way of tracking my tasks.</p><p>Please feel free to share any feedback. I’d be glad to add it to this article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8371614f0b13" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/time-tracking-via-the-telegram-bot-api-and-webhooks-8371614f0b13">Time-Tracking via the Telegram Bot API and Webhooks</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Awesome Website Monitoring With Uptime Kuma]]></title>
            <link>https://medium.com/better-programming/awesome-website-monitoring-with-uptime-kuma-736dfebc1dd3?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/736dfebc1dd3</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[docker-compose]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[telegram]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Fri, 05 Aug 2022 14:23:21 GMT</pubDate>
            <atom:updated>2022-08-05T14:23:21.322Z</atom:updated>
            <content:encoded><![CDATA[<h4>Check your site with this intuitive, lightweight program</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y2gEtjCCR2jkM0T_YeQM9A.png" /></figure><p>In this article, I want to share my experience setting up monitoring for my websites. While other monitoring stacks like using prometheus, node-exporter, and graphana in combination with alerting provide a ton of cool features, I want to present a lightweight and easy stack. The core component of this stack is a project called <a href="https://github.com/louislam/uptime-kuma">uptime-kuma</a>. It’s written in node.js and vue and comes with a really beautiful UI. The main arguments that convinced me to run uptime-kuma are:</p><ul><li>Super easy setup and configuration</li><li>Awesome UI</li><li>Simple integration of telegram notifications</li><li>A neat set of configurations for checking websites</li></ul><p>So let’s get started with deploying uptime-kuma. I love to use <a href="https://doc.traefik.io/traefik/">traefik</a> as a reverse proxy deployed using <a href="https://docs.docker.com/compose/">docker-compose</a>. This easy one-host setup allows you to issue LetsEncrypt certificates for domains automatically.</p><p>So, here is all that you need for the setup: A host with a public IP address, a domain under which your monitoring should be accessible, and an A record that points the domain to the respective host. On the host, you need to install docker and docker-compose. There are several guides on installing both tools for different OSes, for example, <a href="https://docs.docker.com/engine/install/ubuntu/">docker</a> and <a href="https://docs.docker.com/compose/install/">docker-compose</a>. That’s it.</p><h3>Traefik Docker-Compose Deployment</h3><p>For traefik, there are tons of different compose files only, so you can choose whatever you prefer. If you just want to follow this guide, let me show you mine. I like to put the docker-compose deployments under /usr/docker. So for traefik, we have the following files under /usr/docker/traefik. First, the following content needs to be put into docker-compose.yml:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e8084cd5f1d814caac1a70ee7ae6e0d8/href">https://medium.com/media/e8084cd5f1d814caac1a70ee7ae6e0d8/href</a></iframe><p>The compose file contains one service and one network. The service is the traefik container with some command line flags. Mainly, you just need to change your email. If you want to avoid running into rate limits of LetsEncrypt while testing with your setup, you can change the acme resolver to staging (see the comment in the gist). You can request as many certs as you like without rate limiting but will get certificates that are not automatically trusted by browsers.</p><p>So it’s not for production use. The traefik service forwards the ports 80 and 443 for HTTPS traffic and 127.0.0.1:8080 for the dashboard, which is only accessible from localhost. Furthermore, there is one external docker network called reverseproxy. We will attach all containers for which traefik will serve as reverse proxy to this network. The second file we need to put in this folder is traefik.toml:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/afcdc30842e667fa6c98ffed6fb8ab72/href">https://medium.com/media/afcdc30842e667fa6c98ffed6fb8ab72/href</a></iframe><p>This config sets the entry points for HTTP and HTTPS traffic and the logging. That’s it for traefik.</p><p>You can now run the following commands in the /usr/docker/traefik folder:<br>- docker network create -d bridge reverseproxy <br>- touch access.log <br>- docker-compose up -d</p><h3>Uptime-Kuma Docker-Compose Deployment</h3><p>For uptime-kuma, the authors provide a <a href="https://github.com/louislam/uptime-kuma/blob/master/docker/docker-compose.yml">docker-compose</a> file that works out of the box. To run it behind traefik with LetsEncrypt certificates, we just need to change it to the following /usr/docker/uptime-kuma/docker-compose.yml file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a2582c5a262b0bd984425dd3d22c8ea6/href">https://medium.com/media/a2582c5a262b0bd984425dd3d22c8ea6/href</a></iframe><p>We need to configure some traefik labels to let traefik obtain the correct certificates for the domain. Furthermore, the uptime-kuma container needs to be added to the reverse-proxy network so that traefik can forward incoming traffic to it. Finally, just run the following command in your uptime-kuma folder to start the container: docker-compose up -d</p><h3>Configuring Uptime-Kuma</h3><p>The first time you access your uptime-kuma instance, you must configure a secure admin password. Once you’re done, you can start adding endpoints. The UI looks like this. You can define particular behaviour on how and how often uptime-kuma should check your website, for example, HTTP return codes, certificate validation, etc. It’s great.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GjQVLR7iUew5qjoD5yCrAw.png" /></figure><p>Once you’ve added your first endpoints, I recommend configuring telegram notifications. With this, you will be notified whenever one or more endpoints are not working as expected. It’s quite easy to set up telegram notifications. You need to create a group, a bot, and configure the Chat ID of the group and the bot token in uptime-kuma.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0MIEooKyKqVvYv79G70fJw.png" /></figure><p>So, let’s do it step by step. To create a bot, you need to write the <a href="https://core.telegram.org/bots#6-botfather">Botfather</a>. Go to telegram (or web telegram), open a chat to Botfather (follow the link in the doc), then write /newbot as a command to the bot. You need to give it a name and a username following the requirements. You can see your token, which is the first part of the configuration you need for uptime-kuma.</p><p>Now, create a new telegram group and add the bot to it. You can now get the chat id by following the instructions <a href="https://stackoverflow.com/questions/32423837/telegram-bot-how-to-get-a-group-chat-id">here</a>. Basically, you need to write a message in the group chat, make a get request with whatever tool you like to https://api.telegram.org/bot&lt;YourBOTToken&gt;/getUpdates, and then look for the chat field. Sometimes this number is negative. Don’t worry, that’s fine.</p><p>With the chatId, you can complete the configuration. You may test the telegram notifications with the button in the settings dialog.</p><h3>Summary</h3><p>This article shows you how to install uptime-kuma in combination with the traefik reverse proxy and how to configure telegram notifications. In my opinion, it’s a great tool for monitoring websites with many useful settings. I can definitely recommend using it if you only need to ensure that your websites are running properly.</p><p>Please feel free to share any feedback. I’d be glad to add it to this article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=736dfebc1dd3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/awesome-website-monitoring-with-uptime-kuma-736dfebc1dd3">Awesome Website Monitoring With Uptime Kuma</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to setup and configure mptcp on Ubuntu]]></title>
            <link>https://medium.com/high-performance-network-programming/how-to-setup-and-configure-mptcp-on-ubuntu-c423dbbf76cc?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/c423dbbf76cc</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[network]]></category>
            <category><![CDATA[ubuntu]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Fri, 29 Jul 2022 18:18:52 GMT</pubDate>
            <atom:updated>2022-07-29T18:18:52.213Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rcd4jtucCbGQuw0Im9cP_w.jpeg" /><figcaption>Image from <a href="https://unsplash.com/photos/EUsVwEOsblE">unsplash</a> by <a href="https://unsplash.com/@adigold1">Adi Goldstein</a></figcaption></figure><p>In this story I want to share my experience and the steps I did to setup mptcp on two of my Ubuntu servers. My goal was to evaluate the performance of iperf3 using mptcp over two 10Gbit/s links. It was a stony road to get it to work, so please read this story as a mix of a short introduction to mptcp and a guide on how to configure it.</p><p>As always, I’m focusing more on the practical side. I found a lot of helpful resources for mptcp in the Internet, but a short guide for Ubuntu was missing. So please use this story as inspiration for your own machines to play around with mptcp. Just as a short spoiler: In terms of performance, it was definitely worth it to go through the trouble. But enough for prologue, let’s get started.</p><h4>1. Observation: There is not “that mptcp” for Linux</h4><p>At the beginning of my journey, I thought that there is one mptcp implementation which is <a href="https://multipath-tcp.org/pmwiki.php">this one</a>. It’s basically available for any kind of distribution, can be build from source and contains a ton of cool features. However, I read that it supports only specific Linux kernels (4.x.y), which sounded quite suspicious for me. Furthermore, I didn’t want to downgrade my kernel just to enable this tooling. So with my Ubuntu, I did some research on how to install it (apart from the things mentioned on the mptcp website) and found out that there is a package for Ubuntu 22.04. I thought: Okay nice, let’s just do a release upgrade and use the package. The cool thing is: Ubuntu 22.04 comes with Linux kernel ≥ 5.15.x which has a built-in mptcp implementation available. But (what took me a bit of time to figure out, so let me state it right at the beginning): These two implementations are completely separated. <a href="https://multipath-tcp.org/pmwiki.php">multipath-tcp</a>.org is the initial implementation done by researches who worked on mptcp and the implementation in the newer Linux kernels is the <a href="https://github.com/multipath-tcp/mptcp_net-next">mptcp upstream project</a>. The funny thing is, that on multipath-tcp.org the have a note that for all Linux Kernels ≥ 5.6 please use the mptcp upstream project, which I completely overlooked the whole time… So for the rest of this story please keep in mind that these two are configured differently.</p><h4>Installation</h4><p>As mentioned before, I decided to go with the upstream mptcp implementation in the kernel. If you’re running Ubuntu 21.10 or 22.04, you’re basically ready to use mptcp. It can be configured with the mptcp path manager in the <a href="https://manpages.ubuntu.com/manpages/jammy/man8/ip-mptcp.8.html">ip route package</a>. You can verify that mptcp is available on your system via sudo dmesg | grep mptcp. This command should log any reasonable output. To see if it’s enabled, you can use sude systcl net.mptcp.enabled. This value should be set to 1 if you plan to use mptcp.</p><p><strong>Side note:</strong> You can of course also use the multipath-tcp.org implementation. Once you install the package on your system it will add a dedicated mptcp kernel to your boot menu and after configure your boot manager to boot it (in my tests it was chosen automatically by grub) you can boot in your mptcp kernel. But please be aware that the configuration is in some nuances different than the one I present here.</p><h4>How applications can use mptcp</h4><p>For applications, mptcp aims to create a similar interface as using normal tcp. The applications creates a socket and binds it. Now there are basically three ways to use mptcp. <br>1. Use setsockopt call with the protocol number 26 to inform the kernel that your socket should use mptcp. This is great if you’re writing your own applications, but changing this option in external tools sounds not really reasonable, because these tools need to be all built from source.<br>2. That’s why we come to way number two: Enable mptcp automatically for all applications that open normal tcp sockets. This can be done via a systemtap script as shown on the <a href="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/getting-started-with-multipath-tcp_configuring-and-managing-networking">redhat site</a>. On my servers this didn’t work (I got some weird errors), but it could be worth a try.<br>3. And finally the way that I personally found best: Use mptcpize to make a specific application use mptcp instead of normal tcp. It works by hacking into library calls of the target application and set the mptcp protocol number. For me it looked like heavy black magic but it didn’t show any errors, so it seemed fine so far.</p><p>So using mptcpize I thought I was ready to run a sample application, in my case it was iperf3 since I was interested in doing performance measurements over mptcp. In the guide from <a href="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/getting-started-with-multipath-tcp_configuring-and-managing-networking">redhat</a> they show a command which should output that iperf3 is using mptcp (<strong>ss -nti ‘( dport :5201 )’</strong>), which I tried. Without success. It was just showing cubic/cwnd and nothing mptcp related. I thought it didn’t work, but I was wrong.</p><h4>2. Observation: Use reliable ways to test if mptcp is used</h4><p>The commands to check if a socket is using mptcp or not should be used carefully. I spend two hours trying to figure out why mptcp is not working, while it was working, the commands simply did not output what I expected. And finally, it was just not configured to multiple paths. What I can definitely recommend is using tcpdump sudo tcpdump -i lo any . There you should see mptcp as protocol in the packets targeted to the applications that use mptcp.</p><h4>Configure Routing for mptcp</h4><p>My first tries to run iperf3 over mptcp didn’t show any difference compared to iperf3 over normal tcp (around 9.3Gbit/s throughput over one 10Gbit/s link). The reason for this is that I didn’t configure any additional paths. There are some cool examples on how to do this (<a href="http://multipath-tcp.org/pmwiki.php">multipath-tcp.org</a>, <a href="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/getting-started-with-multipath-tcp_configuring-and-managing-networking">redhat guide</a>). Let me present how I configured my servers to run mptcp. My setup looks like this:</p><ul><li>Server1: eno1: 10.0.0.2, eno2: 192.168.0.2, 10Gbit/s direct</li><li>Server2: eno1: 10.0.0.1, eno2: 192.168.0.1, 10Gbit/s direct</li></ul><p>So let’s have a look at the configuration on Server1:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bce4bde3a2b9ea065198e4a0bb2998ad/href">https://medium.com/media/bce4bde3a2b9ea065198e4a0bb2998ad/href</a></iframe><p>And on Server2:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1707ca45558ea060753d04d33fdfaf77/href">https://medium.com/media/1707ca45558ea060753d04d33fdfaf77/href</a></iframe><p>The iperf commands on client and server with respect to the configuration above are the following:<br>client (Server2): mptcpize run iperf3 -c 10.0.0.2<br>server: (Server1): mptcpize run iperf3 -s</p><p>What we can see here it that iperf connect with the initial flow over the 10.0.0.x addresses and uses the configured endpoint 192.168.0.x as additional subflow. After configuring the routing, I ran iperf3 again and I saw an interesting change: iperf3 now achieves around 13 Gbit/s, evenly distributed between both links. That’s cool, I finally got mptcp to work properly. To further tune performance, I increased the read and write buffers of TCP and it went up to 17 Gbit/s. That’s it, I’m happy with the results. I think achieving the whole 20 Gbit/s is only an issue of fine tuning.</p><p>As always, please mail me any feedback you have, I really appreciate any kind of comments or additional information, and I will update this article with any helpful input I get. The original article was posted <a href="https://hpnpl.net/posts/mptcp-ubuntu/">here</a>.</p><p>Cheers,<br>Marten</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c423dbbf76cc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/high-performance-network-programming/how-to-setup-and-configure-mptcp-on-ubuntu-c423dbbf76cc">How to setup and configure mptcp on Ubuntu</a> was originally published in <a href="https://medium.com/high-performance-network-programming">High Performance Network Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Favourite Go Project Setup]]></title>
            <link>https://martengartner.medium.com/my-favourite-go-project-setup-479563662834?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/479563662834</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[project-structure]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Fri, 29 Jul 2022 17:55:21 GMT</pubDate>
            <atom:updated>2022-07-29T17:55:21.937Z</atom:updated>
            <content:encoded><![CDATA[<h3>My Favourite Go Project Structure</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hixeP9WGf5IZ1k1rjHxzQQ.jpeg" /><figcaption>Image by <a href="https://unsplash.com/@next_academy">Next_Academy</a> from <a href="https://unsplash.com/photos/G6k_uEjXygE">unsplash</a></figcaption></figure><p>In this story I want to present my project setup for applications written in Go. It’s suits well for larger applications that serve cmd commands, UIs, APIs, or all of them. I spent a lot of time thinking on where to locate things in my Go projects, that’s why I want to share the result with you to provide a reasonable foundation for your next Go project. Despite recommending specific libraries for particular tasks, the actual folder structure is not bound to these libraries, it can be used with any existing alternatives. So let’s not waste any time and start with the actual folder structure:</p><h4>cmd</h4><p>This folder contains the cmd commands generated and handled by <a href="https://github.com/spf13/cobra">Cobra</a>. Cobra has a very good documentation and a nice cmd tooling to create (sub)commands. And it allows easy configuration for nested commands, where each of the (sub)commands can be configured independently. So definitely a great choice!</p><h4>internal</h4><p>This folder is for internal logic that should not be imported from any other package. It’s a convention in go that prohibits other packages to import any functions or structs located in the internal folder, only the current module can import things from there.</p><blockquote>An import of a path containing the element “internal” is disallowed if the importing code is outside the tree rooted at the parent of the “internal” directory.</blockquote><p>I don’t often put code in the internal folder, because in the past I sometimes ran in the issue that I wanted to use internal code in my own packages but due to this rule, I could not do so. So I would propose to use this folder only for code that is wrapped in other files outside of the internal folder and which you don’t want anyone to import directly.</p><h4>pkg</h4><p>The pkgfolder is one of the most important ones, it contains core parts of the application logic which is intended to be imported by other packages. In my projects, I create subfolders for any specific group of logic I want to make public available. Examples I wrote are pkg/ssh for an abstraction that performs actions over ssh, or pkg/cmd to wrap the usage of executing cmd commands from my application.</p><p><strong>Side note</strong>: There is an interesting blog post about <a href="https://travisjeffery.com/b/2019/11/i-ll-take-pkg-over-internal/">pkg vs internal folder</a>.</p><h4>models</h4><p>If you’re working with any kind of database and a respective ORM to connect to it (e.g. <a href="https://gorm.io/">gorm</a>), a dedicated folder for these models makes sense. You can import the models from any other location. Since go allows configuring different kind of usages for fields via tag reflection, e.g. json or gorm specific configuration, there is mostly no need to write multiple structs for a model to be handled by different packages.</p><p><strong>Side note</strong>: You may also move the models in subfolders of the pkg directory if they rely to specific logic, e.g. an SshConfig model can also be located in pkg/ssh instead of models . It depends on your own favour.</p><h4>api</h4><p>If your application provides any API to connect to (e.g. gRPC, HTTP, …), the api folder is the correct location where to put any schemata or specification. You may also add the routes of the API endpoints here. It’s easy for other developers to get an overview of what endpoints your application provides.</p><h4>ui</h4><p>If you’re aiming to create a GUI application (instead of or even in combination with a cli application), moving things that are UI-related in a specific folder is a good way to separate it from the rest. I personally prefer <a href="https://wails.io/">wails</a> for GUI application development in Go. It’s a lightweight library using native WebView instead of Electron, provides a list of available starters (I prefer React with Typescript) and it’s easy to use.</p><h4>lib</h4><p>At the first view, you may think that lib sounds like a conflict with pkg , both are folders to store code that should be reused in the current or any other module importing it. But I think the lib folder can serve a different purpose: Bundling the core logic of the application to make it usable from the cli and the UI part. Let’s consider a build command, that should be exported in the UI and in cmd. To achieve this, we wrap all available cmd args for the build command into a struct called BuildArgs in lib/build.go . Additionally, a function called func Build(args *BuildArgs) is exported from lib/build.go . The cmd part can now simply wrap the cmd args into the BuildArgs struct and call the Build function. The UI part can collect the required input from the user and proceed the same way. So in my projects, the lib folder contains core function of the application that is planned to be reused with different kind of user input.</p><h4>scripts</h4><p>In case your project requires some external scripts (e.g. bash, lua, …) it’s a good idea to put these scripts into a dedicated folder. Each developer can easily recognise that this folder probably does not contain any go code but some other scripting languages.</p><h4>tools</h4><p>If your project needs some external tools to do particular tasks (e.g. code generation) that are probably not intended to be installed globally (in contrast to e.g. Cobra).</p><h4>utils</h4><p>This folder is probably the most dangerous of the ones presented here. In utils you can basically put everything that is not related to any particular part of logic but which is more a general helper that can be used in different places. However, you should be careful before adding any code to the utils folder: In general, even the helper functions should be located either in the internal or pkg directory to which they logically fit. And it’s easy to create a trash bin of functions and structs in the util folder to avoid thinking about how they can be integrated in the actual project structure. So please be careful putting code in this folder.</p><h3>Summary</h3><p>This story covers my structure for larger go applications. It’s a mix of recommendations from external sources (see below) and my personal experiences and choices. Please feel free to comment any feedback, I’m glad to improve the setup!</p><p>Inspired by:<br>— <a href="https://github.com/golang-standards/project-layout">https://github.com/golang-standards/project-layout</a><br> — <a href="https://dev.to/jinxankit/go-project-structure-and-guidelines-4ccm">https://dev.to/jinxankit/go-project-structure-and-guidelines-4ccm</a><br> — <a href="https://tutorialedge.net/golang/go-project-structure-best-practices/">https://tutorialedge.net/golang/go-project-structure-best-practices/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=479563662834" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lightweight and Type-Safe Localisation in React]]></title>
            <link>https://medium.com/better-programming/lightweight-and-type-safe-localisation-in-react-493b2609c282?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/493b2609c282</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[reactjs]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Mon, 04 Apr 2022 17:04:28 GMT</pubDate>
            <atom:updated>2022-04-04T17:04:28.313Z</atom:updated>
            <content:encoded><![CDATA[<h4>Using React Context API and Hooks</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-lDIay_OlMEFSKYGkxnXdQ.jpeg" /><figcaption>Image by <a href="https://unsplash.com/@ratushny">Dmitry Ratushny</a> from <a href="https://unsplash.com/photos/wpi3sDUrSEk">unsplash</a></figcaption></figure><p>Localisation (or multi-language support) is a core part of nearly all production-ready apps. Mostly, it’s a time-consuming task to keep all translations up to date and it may even stop your coding flow if you need to keep translations up to date while writing code.</p><p>In my experience, I often ran into typos in language files and untranslated strings while referring to a missing language key. I observed that it’s too easy to mix keys or to add duplicates.</p><p>Therefore, I implemented a lightweight, type-safe localisation for React.<br>It uses the React context API and hooks to make the usage inside functional components easy and forces translations and key lookups to be type-safe using Typescript. If you’re interested, let’s dive into the implementation!</p><h3>Implement Localisation in a Context</h3><p>First, let’s look at the actual store of the current locale selected by the user. I decided to save this information in a React context because it should be accessible in the whole app, and there are fewer changes in the current locale expected, so no fear of performance issues.</p><p>The selected locale is saved in the Localstorage, and loaded when the context first mounts, to keep the user’s decisions. If no locale could be loaded from the Localstorage, the default one is used (in my case mostly English). In the Localstorage, only the localeId (e.g., en_US) is stored instead of the whole locale.</p><p>The context file exports a Provider which we put around our whole example app. In this context, I provide the available locales, the current locale, and a function to set the current locale to be accessed in the components.</p><p>To inject the LocaleContextProvider into any position in the React component tree, props.children are simply set as React children and are therefore passed through. The whole context is shown in the following gist:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d9755102db57ed2a2301af83c397a7dc/href">https://medium.com/media/d9755102db57ed2a2301af83c397a7dc/href</a></iframe><p>That’s it for the context side, now let’s switch to the language resources.</p><h3>The Language Files</h3><p>As language files, we use simple.ts files which export a JS object containing lookup keys and their translations. For each locale, one file is added that exports the particular translation object. For typing performance, I experienced that autocompletion improves significantly if all your lang keys start with a prefix, e.g., lang. That worked quite well for me and makes it easy to differentiate translations from other variables.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4be6f624dde98f61a75b2a65567b075e/href">https://medium.com/media/4be6f624dde98f61a75b2a65567b075e/href</a></iframe><h4>Add type-safety</h4><p>Since I’m using JS objects for our translation maps, the implementation can benefit from enabling type checking via Typescript. To avoid creating a dedicated type that contains all keys required for a translation to be complete (trust me, I did this, it’s annoying…), I use the typeof operator to get the type of our default, English translation, and force all other translations to have the same type.</p><p>In this case, this means that all keys of the default translation need to be available in the other translation. Consequently, this leads to proper syntax highlighting and build errors for missing or wrong translation keys.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4012e79ed7667a9c96f9fff0c7e9fab8/href">https://medium.com/media/4012e79ed7667a9c96f9fff0c7e9fab8/href</a></iframe><h4>Usage in apps</h4><p>After implementing the translation and localisation, the last step is to include the translations in our app. Therefore, I have defined some useful hooks that can be used in every component (inside the component tree of the LocaleContext). The following hooks are available in my default implementation and can be extended easily:</p><ul><li>useLanguage: Returns the translation object of the current locale</li><li>useLocales: Returns the available locales</li><li>useSetLocale: Returns a function that sets a new locale</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1659ae5f3e1918dc1d023c26a0db81f8/href">https://medium.com/media/1659ae5f3e1918dc1d023c26a0db81f8/href</a></iframe><p>In the following example, the usage of all these three hooks is depicted. Via a button click, the app iterates through all available locales and changes the current locale accordingly. The translations can be used as normal variables of type string in any jsx or other function logic.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e85894817c753c36d894bc3e49a01837/href">https://medium.com/media/e85894817c753c36d894bc3e49a01837/href</a></iframe><p>Here comes the main clue of using Typescript: you can’t do any typos in the language keys, since the translation is typed and only allows to access existing keys, and all translations need to be complete, so there are no half-baked translations available.</p><h3>Summary</h3><p>In this story, I want to share my implementation of a lightweight, type-safe localisation with React contexts and hooks. The implementation allows easy use of translated strings, forces translations to be complete (otherwise typescript will blame you), and removes the possibility of typos while using translations. Even if the setup for translation seems to be comparatively complex, the benefits worked for me quite well.</p><p>The whole code is available on <a href="https://stackblitz.com/edit/react-starter-typescript-b7m7ct?file=App.tsx">stackblitz</a>.</p><p>I hope you enjoyed this small dive into my approach to localisation in <br>React.</p><p>Please feel free to share any feedback, I would appreciate it and update the story accordingly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=493b2609c282" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/lightweight-and-type-safe-localisation-in-react-493b2609c282">Lightweight and Type-Safe Localisation in React</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Easy (De)Serialisation of yaml files in Go]]></title>
            <link>https://medium.com/thedevproject/easy-de-serialisation-of-yaml-files-in-go-4557456b0a98?source=rss-d3e102089d47------2</link>
            <guid isPermaLink="false">https://medium.com/p/4557456b0a98</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[code]]></category>
            <category><![CDATA[coding]]></category>
            <dc:creator><![CDATA[Marten Gartner]]></dc:creator>
            <pubDate>Thu, 31 Mar 2022 18:48:28 GMT</pubDate>
            <atom:updated>2022-03-31T18:48:28.373Z</atom:updated>
            <content:encoded><![CDATA[<p>By using: <a href="http://gopkg.in/yaml.v3">gopkg.in/yaml.v3</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7UT1tvK_lZN5TugFRhsBlQ.jpeg" /><figcaption>Image from <a href="https://unsplash.com/@hishahadat"><strong>Shahadat Rahman</strong></a><strong> via </strong><a href="https://unsplash.com/photos/BfrQnKBulYQ"><strong>unsplash</strong></a></figcaption></figure><h4>Intro</h4><p>Yaml, as a comparatively new format to save data, comes with wide adoption and easy syntax. It’s more human readable than e.g. JSON. Consequently, it is used as a configuration format in many modern open-source projects, like kubernetes or docker-compose. To work with yaml files, gopkg provides an awesome library called <a href="https://gopkg.in/yaml.v3">yaml.v3</a>. Despite some other competing libraries, I really like the API yaml.v3 provides. Since I found the <a href="https://pkg.go.dev/gopkg.in/yaml.v3?utm_source=godoc">API description</a> quite low-level, I decided to write about the general usage and some caveats to consider. So if you’re interested in diving into this, let’s get started!</p><h4>Get Your Struct Ready</h4><p>yaml.v3 works, as many other libraries, using reflection to inspect fields of a struct. It provides on tags, that are processed via tag reflection, to make configuration easy and in line with other libraries, so a struct can be transformed to json, yaml and others without changing it. A sample struct looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f6b09bcef5a4747f4ffe81a674974d29/href">https://medium.com/media/f6b09bcef5a4747f4ffe81a674974d29/href</a></iframe><p>Compared to Name and Version, which are simply string fields without yaml tags, the Description field looks a bit different. Here I added two yaml tags. The first one is the representation within the yaml file, which is required if you want to add other tags. Per default, this is a lowercase version of the field name, therefor I named it accordingly. The second tag is omitempty, which is really useful to keep you yaml clean: Without it, empty strings (to which string fields are initialized per default in Go structs), will be serialized to “” in the yaml files. For larger structs, this looks really awkward. By adding omitempty, the field will not be serialized to yaml output and the final file looks cleaner.</p><p>So please keep the following two things in mind:</p><ul><li>Use omitempty to avoid bloating your files with empty strings</li><li>You need to add the yaml field name as first tag if you want to use e.g. omitempty multiple times.</li></ul><h4>(De)Serialisation</h4><p>After preparing the target struct to be ready to be transformed to yaml, the actual serialization and deserialization is the next step to look at. This can be done similarly to working with JSON by using <em>marshal</em> and <em>unmarshal</em>. The following gist provides an example for both ways.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/db358c2df582052dab3f3129880a2c2d/href">https://medium.com/media/db358c2df582052dab3f3129880a2c2d/href</a></iframe><p>That’s it. Using yaml.v3 is as easy as powerful. However, I want to add one caveat to consider.</p><h4>Using Maps in yaml.v3</h4><p>(De)serialising simple fields like strings or ints is straight forwarding. However, using complex types is sometimes not obvious. In my case, I wanted to add a slice of structs to my target struct and process it via yaml.v3. Adding them as slices didn’t work properly, therefor I tried using maps, which solved the issue. Looking at how my yaml file looks like, adding maps instead of slices makes sense. I have an example of maps of nested structs in the next gist:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c3e0826c53db9b63325ccc9311b60a3c/href">https://medium.com/media/c3e0826c53db9b63325ccc9311b60a3c/href</a></iframe><p>Looking at the yaml output, it gets clear why to use maps instead of slices: We need a key for each item (in this case <em>stack1</em> and <em>stack2</em>) to identify and further process it.</p><h4>Summary</h4><p>yaml.v3 did a very good job in my projects to save and load data from files in a human-readable format. It’s as easy to use as e.g. JSON, but much easier to read for humans. Only the usage of maps of nested structs got me a small headache. However, after understanding to use maps with string keys instead of slices, no further issues came along my way.</p><p>So I would definitely recommend the yaml.v3 library to everyone who wants to work with yaml in Go.</p><p>Cheers,<br>Marten!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4557456b0a98" width="1" height="1" alt=""><hr><p><a href="https://medium.com/thedevproject/easy-de-serialisation-of-yaml-files-in-go-4557456b0a98">Easy (De)Serialisation of yaml files in Go</a> was originally published in <a href="https://medium.com/thedevproject">The Dev Project</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>