Hey, how are you? ldmarz here, and you’re about to read about SRE (Site Reliability Engineering), a role I’ve been in for not too long and that I still find a bit hard to understand what it’s all about. Spoiler: I’m still trying to figure it out.

The Essential Role of Site Reliability Engineering

Think about the last time you were ready to buy the latest video game online. Excitement is high, you're just a click away, and then... the website crashes. It's more than a little frustrating, right? This is a common scenario when online platforms lack a crucial element: Site Reliability Engineering, or SRE.

It's not that developers and IT staff don't care about keeping things running. In fact, many of them do a bit of SRE work without even realizing it, ensuring their part of the service is reliable. But as a system grows—more features, more users, more data—the complexity increases. This is where the SRE team shines, specializing in keeping the entire ecosystem stable and efficient.

SRE: Beyond Just "Keeping the Lights On"

SRE is more than just making sure a website is up and running. They are the guardians of the user experience, ensuring that every interaction with the site is smooth, even during high traffic times like game launches or big sales.

Comparing with other roles

• Developers focus on building features and fixing bugs in the software. While they care about reliability, their main goal is to add new capabilities to the product.
• IT and Operations teams ensure that the hardware and networks that the software runs on are functioning correctly. They’re the ones who make sure the servers are up and the networks are fast.
• SREs, on the other hand, sit at the intersection of development and operations. They use their coding skills to automate responses to common problems, like a sudden spike in traffic or a service going down. They also make data-driven decisions, analyzing errors, measuring the cost and efficiency of solutions, and constantly looking for ways to improve reliability.

The SRE Toolbox

SREs have a variety of tools and practices at their disposal:

• Automation: This is key to handling repetitive tasks efficiently, like setting up new servers or responding to common errors.
• Monitoring and Alerting: SREs set up systems to constantly check the health of services. If something goes wrong, they know about it immediately and can start fixing it before most users even notice a problem.
• Error Budgets and SLOs (Service Level Objectives): These are agreements on how reliable a service needs to be. If a service is too reliable, it might mean the team is being too cautious and not releasing new features quickly enough. If it’s not reliable enough, users get frustrated. SREs use error budgets to find the right balance, allowing for innovation without sacrificing user experience
• Efficiency and Cost Measurement: They constantly evaluate the cost-effectiveness of their infrastructure and operations, ensuring the company gets the most bang for its buck.

The Impact of SRE

By focusing on reliability, automation, and efficiency, SRE teams free up developers to work on new features, knowing that the infrastructure will support them. They help companies move fast without breaking things, allowing for rapid growth and improvement.

In Conclusion

Returning to our game store example, imagine it's equipped with a robust SRE team. When you go to buy that game, the process is seamless. The page loads quickly, the transaction goes through without a hitch, and you're ready to play. That's the goal of SRE: to make sure that as systems grow and evolve, they remain stable, efficient, and user-friendly. It's about enabling progress without compromising on quality, ensuring that everyone, from the developers to the end-users, has a great experience.

The Challenge of Natural Language Queries

In the world of programming, we often structure our data in a way that makes it easy for machines to understand. We categorize, label, and normalize data to fit into neat tables that can be queried with precision. But this isn’t how humans express themselves.

When a user types in a super natural language query like, “I want a spaceship of the latest model to zip around the city at top speed,” they’re not thinking about product categories or specific labels. They’re expressing a desire based on their experiences, feelings, and the way they perceive the world. As a human, you understand that they’re looking for a sporty city car from the latest year. But how do you translate that into a query your search engine can understand?

The Magic of Embeddings

This is where the power of embeddings comes into play. Just as our brains sort information in a complex, multi-dimensional space based on our experiences and associations, embeddings represent words in a high-dimensional space where the semantic similarity between words corresponds to the geometric distance in the space.

So, how can we teach a machine to understand the meaning of “spaceship of the latest model to zip around the city at top speed”? An embedding is a simple concept that can help us achieve this. Imagine the word “red” being converted into a vector that signifies “color”, “blood”, and “danger” in different contexts. This is the magic of embeddings. They capture the essence of a word and its meaning in a form that machines can understand and process.

To truly appreciate the power of embeddings, it can be helpful to visualize them. The following GIF, created using TensorFlow’s Projector, provides a visual representation of a word embedding. Each point in this space represents a different word. Words that are semantically similar are closer together, while words that are semantically different are farther apart.

In this visualization, you can see how the word ‘red’ might be positioned in relation to words like ‘blue’, ‘yellow’, ‘rainbow’, etc. This gives us a glimpse into how embeddings capture the semantic relationships between words. It’s like a map of human language, charting the subtle nuances and associations that give our words meaning.

The Power and Versatility of Embeddings

Embeddings offer a range of powerful features:

• Storage: These vectors can be stored in special types of databases, making them easy to access and use in various applications.
• Versatility: Not only text, but virtually any multimedia element (videos, images, texts) can be vectorized. This allows for a wide range of data to be processed and understood by machines.
• Multimodal: These embeddings can be multimodal, meaning they can represent multiple types of data simultaneously. This feature is particularly useful when dealing with complex data sets that include various types of data.
• Multilingual: Embeddings can be created for text in any language, making them a powerful tool for multilingual applications and global services.
• Improved Query Handling: The use of embeddings opens up a world of possibilities for handling complex queries and delivering more accurate results.

A Closer Look at Different Types of Embeddings

There are several types of embeddings, each with their own pros and cons:

Word2Vec

Developed by Google, Word2Vec represents words in a high-dimensional space where the semantic similarity between words corresponds to the geometric distance in the space. Word2Vec is great for capturing the semantic meaning of words. It’s simple and computationally efficient.

• Pros: Captures the semantic meaning of words very well.
• Cons: Does not account for the context of words. For example, the word “bank” would have the same vector representation in “river bank” and “financial bank”.
• Use Case: Ideal for tasks where the semantic meaning of words is important, such as semantic search, recommendation systems, and sentiment analysis.

GloVe (Global Vectors for Word Representation)

Developed by Stanford, GloVe excels at creating a global word-word co-occurrence matrix from a corpus. It combines the benefits of two major methods in the field, count-based methods and predictive methods. GloVe captures both the global statistics and local semantics of a corpus. It is also efficient to train.

• Pros: Captures both the global statistics and local semantics of a corpus.
• Cons: Like Word2Vec, GloVe does not consider the context of words.
• Use Case: Useful for tasks that require understanding the global semantic co-occurrence of words, such as document classification and clustering.

BERT (Bidirectional Encoder Representations from Transformers)

Developed by Google, BERT understands the context in which a word is used, making it powerful for a variety of NLP tasks. BERT considers the context of words, capturing nuances that other models miss. It has achieved state-of-the-art results on a variety of NLP tasks.

• Pros: Captures the context of words, capturing nuances that other models miss.
• Cons: BERT is computationally intensive and requires a lot of resources to train. It is also not as interpretable as models like Word2Vec or GloVe.
• Possible Use Case: Ideal for tasks that require a deep understanding of the context, such as question answering, named entity recognition, and sentiment analysis.

You can find a comprehensive comparison of these models in the MTEB leaderboard.

Querying Vectors: The Key to Unlocking the Power of Embeddings

Once we have our embeddings, we can perform queries on them. For example, we can use the dot product to find the similarity between two vectors. The higher the dot product, the more similar the vectors are. Alternatively, we can use cosine similarity, which measures the cosine of the angle between two vectors. This method is particularly useful when the length of the vectors doesn’t matter, and we’re only interested in the direction they’re pointing.

Note: If the thought of working with vectors seems daunting, don’t worry! Nowadays, there are numerous libraries like NumPy, and even databases like PostgreSQL with vector plugins, that make these calculations straightforward and manageable. So, you can leverage the power of embeddings without needing to become a mathematician.

The Wide World of Use Cases

Embeddings can be used in a variety of applications, each with its unique benefits and challenges. Here are a few examples:

• Building Smarter Search Engines: With embeddings, search engines can understand the intent behind a user’s query, not just the specific words they use. For example, a user might search for “spaceship to zip around the city,” and the search engine could understand that they’re looking for a fast, city-friendly car.
• Creating a Q&A System: Embeddings can be used to build a Q&A system for your product that can handle complex queries. For instance, a user might ask, “How do I increase the speed of my spaceship?” and the system could understand that they’re asking about improving their car’s performance.
• Developing a Chatbot: A chatbot powered by embeddings can engage in natural, human-like conversations. It could understand a wide range of inputs and respond appropriately. For example, if a user says, “I’m looking for a spaceship that’s good for city driving,” the chatbot could respond with, “It sounds like you’re looking for a compact, efficient car. Here are a few options…”
• Generating Posts from Multimedia: With multimodal embeddings, you could generate posts from an image or video. For example, a user could upload a picture of a car, and the system could generate a post saying, “Check out my new spaceship! It’s perfect for zipping around the city.”

Exploring Further

If you’re interested in diving deeper into the world of NLP and embeddings, I invite you to check out my GitHub repository where I share various scripts for testing different NLP techniques.

Wrapping Up

I hope this post has given you a good understanding of the power and versatility of embeddings. They are an incredible tool that can help us bridge the gap between human language and machine understanding, opening up a world of possibilities for more natural and intuitive user experiences.

If you found this post helpful, please give it a clap and share it with others who might find it useful. And as always, feel free to reach out to me on LinkedIn if you have any questions or just want to chat about all things NLP!

Understanding Embeddings: A Key to Search with artificial intelligence was originally published in GoPenAI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Este año nuevo comenzó con unos datos bastante interesantes, Google sólo para molestar me envió mi reporte anual de lugares visitados.

Reporte con muchísima información como cuantos lugares he visitado, cuantas veces, incluso cuanto tiempo he pasado en movimiento(caminando, metro etc) esto me llevó a la conclusión de dos cosas.

• Lo mucho que sabe sobre mi: esto ya no es un secreto para nadie pero igual es alarmante 😨 pero creo que esto podría hablarlo en otro post.
• El tiempo que gasto en transportea: ya esto había leído un poco sobre esto, estadísticas, conspiraciónes etc cosas así.

Y es justamente sobre este segundo punto es que trata este post y es que si bien es cierto que gasto gran parte de mi vida en el metro he conseguido la forma de hacer este tiempo un poco más útil. No sin antes aclarar que yo ODIO EL TRANSPORTE PÚBLICO.

Y antes que me digas la típica frase “comprate un auto” sinceramente creo que es peor porque toda mi concentración se va directo a solo conducir 😕.

¿Que hacer?

Mantente ocupado. Si es así de simple, en el transporte público invierto el tiempo en escribir este post por ejemplo. Depende del estado de ánimo un día podría dedicarlo a hacer un buen momazo o atacar a un clan rival en el clash royale.

Tampoco es que ando descubriendo la cura al cáncer mientras me hago espacio para poder entrar al transporte. Pero si que hay distintas actividades que se podrían hacer.

Tengo amigos que escuchan conferencias, cursos, leen un buen libro y te puedo asegurar que su viaje no es un sufrimiento y pérdida de tiempo como lo era para mi hasta hace poco.

Finalmente se trata es de aprender o crear algo en ese tiempo, que quieras o no, tendrás que gastarlo. Esto por supuesto hasta que inventen la teletransportacion o todos vivamos en una utopía donde tooodo el mundo trabaje desde casa.

Créeme el viaje pasa de ser un momento donde el stress tiene la mesa servida a convertirse en un momento que esperas. Dejan de importarte cosas como el particular olor que tiene la persona de al lado o de escuchar quien empujó a quien esta vez.

Entonces, ¿Vas pendiente de hacer algo productivo mientras viajas?

Me atrevo a pensar que todos hemos estado en esa reunión tediosa aburrida ya sea como expositor o público y sin tomar en cuenta la importancia de la misma no logramos encontrar el porque seguimos allí.

Hace unos días tuve un día tan pesado y con tantas reuniones que mi cerebro como una mala broma me colocó de expositor mientras soñaba.

Tenía todos los ingredientes para ser una pésima reunión, me nombraron expositor de un tema el cual no tengo idea, sobre ZAPATOS, ¿es en serio? un auditorio esperando que yo haga un discurso sobre zapatos.

Esto por supuesto no tenía buena pinta. ¿Y como lo tendría? Si me avisan 2h antes, no tengo idea sobre zapatos, excepto lo que el publico en general conoce y sobre todo, es de zapatos, que tema tan extraño para hablar.

La solución

Entonces me vino una idea a la cabeza(estando aún en el sueño) escribamos un stand up sobre esto. Agreguemos un poco de humor a esta exposición.

Y así comencé, lo primero al hablar sobre zapatos seria el ¿por qué usarlos?

Y allí podia hablar un montón de cosas aburridas, pero en vez de eso dividí la exposición en estos puntos.

• Se ven bien: recuerdo haber dicho algo como “ en la mayoría de los casos son más atractivos que esos pies de Hobbit que tienen la mayoría.
• Protección: Díganme a cuantos les encantaría caminar descalzos por un piso lleno de vidrios o tan caliente que podrías cocinar sobre el.
• Personalidad: no hay que negar que cuando ves a alguien con unos zapatos anaranjado fosforescente ese pana debe tener una personalidad suuper original.

Fíjense que seguimos hablando de un tema ultra aburrido pero con un pequeño cambio. El sueño era extraño (como todos) porque era oyente y al mismo tiempo también era expositor y definitivamente disfrute en los dos roles.

Esto por supuesto es porque fue una especie de stand up al que asistí y me mantuve entretenido todo el rato, lástima que acá me cuesta expresar cosas adicionales que sucedieron como el lenguaje corporal, preguntas que por la natureza cómica de la reunión eran graciosas.

Poniendo en práctica

Tenía el clásico sprint review, donde están invitados actores importantes para el proyecto como el owner, usuarios inversionistas personas que por defecto asumimos que quieren que les hables de forma super formal debido a su “importancia”.

Empecé haciendo pequeñas bromas como “esperemos que nada explote”, mencionando objetivos completados para el sprint que no son directamente del proyecto o que ellos le tengan importancia como “ Compramos un filtro de agua”.

En medio de la demo recuerdo que hubo un menú que tenía un brinco raro, funcionaba, pero se sentía raro y comenté algo colo Bueh “acá tenemos una feature nuevo”.

Por primera vez tuvimos una review corta, con gente interesada, donde estoy seguro que todos entendieron como esta la salud del proyecto y su avance, esto sin duda me dejó una gran satisfacción y es el porqué traigo este post.

Conclusión

Ahora la pregunta es ¿por qué no cambiamos la manera en que presentamos avances, reportes incluso problemas. Introducir un poco de humor suavizaria un poco la formalidad del asunto.

Aquí dejo una lista de posibles beneficios:

• Atención: si tu reu logra captar la atención del público por un chiste en el momento adecuado tendrás al público atento en busca de otro.
• Originalidad: no serás el típico expositor aburrido sino al contrario un expositor que quieres ir a escuchar lo que tiene que decir.
• Tiempo: si tienes la atención del público es muy probable que no pierdas tiempo explicando una y otra vez lo mismo.

¿Te atreverías a aplicar esto en tu siguiente reunión?

Como hacer una exposición efectiva was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

As a fanatic of the king of the sports and that at some point as many of you, I used to play until the point of fatigue, and nowadays I can only remember those times.

This is due to the fact that my current hobbies have to do more with trying the latest available JS Framework and with a frequency of at least one per day.

I was, as usual, making programming memes and I couldn’t help but notice the similarities between a programming team and a Football team.

Yes, that’s what happened, so you guys can save your torches, scythes and those posters that says athletes and programmers are different beings since I’ll give you a comparison of all the roles in both teams and you’ll notice I’m not as crazy as it seems.

Fans
Now tell me why would we do a system if there was no audience. This audience are our beloved users.

They will watch our success or failure of the project.

They will be relentless if we fail and they will praise us if the goals are accomplished, my dear friends this a relationship without middle terms. No one says wow this app is great it closes itself 6 times in 10 minutes but I keep loving it.

Defense

Every good team and project must have good grounds because it all gives the same if your project looks like a 5 star but it’s full of issues, drops itself, has a ton of security flaws. It will be very simple for the opposite team to score a 7–1 on you.

Here’s where born backends have they protagonism, they will be in charge of keeping our goal keep safe.

Unfairly for most users and fans this is not a very showy position but its importance in the team is undeniable, or are we going to deny the fact of how many matches has Sergio Ramos put on his back?

Plays like make the rival go offside are easily comparable with a middleware which only allows authorized players.

Midfielder

Midfielder are the equivalent of a full stack, they possess the perfect balance between attack and defense without becoming experts.

It’s very likely that these players have some preference, this being a determining factor when defining their position on the field.

E.g if the team is weak in the offensive you can take your full stack that has preference over the attack and putting him as a deep-lying midfielder during the whole match, this could be your Messi, even though he’s not a forward how many goals hasn’t he dedicated to us?

Forward

What’s a good product worth if you can’t show it to the world? If your fans won’t be able to celebrate it with you?

Here’s where the frontends have their protagonism, inverse to defense this position is criticized by the team for supposedly being easier and at the same time loved by the users because for most people this is the part of the game they like the most because they can understand it.

Manager

I’ve heard many times that this is an unnecessary position because just the players talent is enough to win, well I don’t think the same way as this person is responsible for seeing the macro project and knowing when a change is necessary just in the right time.

In most scenarios, these decisions don’t take enough credit but I consider that there must always be someone who’s watching the whole picture and that can look beyond of what’s in the surface, and this is the project manager.

They also have great ideas like promising the whole fans that we can score 8 goals in 5 minutes.

Team’s captain

Best known as a tech lead, this partner who’s always training a bit more than the other in the field, can take a decision in almost real time, always having into account your team’s capabilities.

An indispensable position because in the middle of the game someone has to be there taking care that everyone follows the good practices that they have studied a lot during training.

Conclusion

What do you think now? Apparently, we have much more in common than what it looks like, we could even compare a match.

Just imagine if every team were two famous technology companies that dedicate to the same business, uber vs cabify for instance, in this sense, it is convenient to have the best team.

So, tell me, what position do you play?

Fluent it’s a nice ORM but sometimes we want features not currently supported by the ORM, for this case I leave you a little example of how you can use the raw query

If you have a problem implemented or a simpler way to do this work please leave a comment below, Regards.

Originally published at Ldmarz blog.

I have the following migration in vapor 3

But maybe we want some default value for the field password… Something like the classic “123456” to do that we must update a little our migration

builder.field(for: \.password, type: .varchar, .default(.literal("123456")))

You can use this syntax to multiples cases like those

Source

Originally published at Ldmarz blog.