<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Krishna on Medium]]></title>
        <description><![CDATA[Stories by Krishna on Medium]]></description>
        <link>https://medium.com/@krishnakantmaurya1616?source=rss-c6110f9aa46e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*6WX4w6SKeWL3kDRbGlaYvw.jpeg</url>
            <title>Stories by Krishna on Medium</title>
            <link>https://medium.com/@krishnakantmaurya1616?source=rss-c6110f9aa46e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 22:49:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@krishnakantmaurya1616/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What is Dependency Injection in Golang? How to Implement It]]></title>
            <link>https://medium.com/@krishnakantmaurya1616/what-is-dependency-injection-in-golang-how-to-implement-it-d228ef202050?source=rss-c6110f9aa46e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d228ef202050</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[dependency-injection]]></category>
            <category><![CDATA[golang-tutorial]]></category>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Krishna]]></dc:creator>
            <pubDate>Sat, 03 Jan 2026 05:08:21 GMT</pubDate>
            <atom:updated>2026-01-03T05:08:21.911Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0WOopWBntWp5cIFlv2yExA.png" /></figure><p>Hello geeks,<br> This is my second Medium article, and I want to share my experience with <strong>Dependency Injection (DI)</strong> in Go what it is, why it matters, and how you can implement it with clean, simple code.</p><p>I’ll try to keep everything easy to understand, with real-world examples and minimal theory. Once you understand this concept, you’ll realize how powerful and clean your codebase can become.</p><h3><strong>Context (My Experience)</strong></h3><p>I was also unaware of this design pattern initially. It was introduced to me by my friend <strong>Eshan</strong> when I first saw his code. At that time, I was genuinely confused about <em>what was going on</em>.</p><p>After that he explained the idea and I did some research on my own, I realized how <strong>clean and flexible</strong> this pattern is. Since then, I’ve implemented Dependency Injection in small projects, and it completely changed how I structure my services.</p><h3><strong>What is Dependency Injection (DI)?</strong></h3><p>Dependency Injection is a design pattern that helps us to <strong>decouple components</strong> by providing their dependencies <strong>from the outside</strong>, instead of creating them internally.</p><p>In simple terms: <em>Instead of a struct creating its own dependencies, those dependencies are </em>injected<em> into it—usually via a constructor.</em></p><p>Important thing to remember:</p><ul><li>Dependency Injection is not a framework or library</li><li>It is a design principle</li><li>The goal is to make a modular, testable, and maintainable code</li></ul><h3>Real-World Problem (Why DI Matters)</h3><p>Imagine you have an <strong>Auth Service</strong> with a register route:</p><ol><li>You connect to the database</li><li>You save the user</li><li>You integrate <strong>Plivo</strong> for email/phone verification</li></ol><p>Later:</p><p>4. You add verification logic to the login route</p><p>5. Then after few year you decide to migrate from <strong>Plivo to SendGrid</strong></p><p>Now the problem arise:</p><p>6. You must search every file</p><p>7. Replace Plivo logic everywhere</p><p>8. High risk of missing something</p><p>9. Painful maintenance as the codebase grows</p><p><strong>This is exactly where Dependency Injection helps. </strong>By injecting dependencies (like DB, Third-party or notification service), you only need to change them <strong>in one place</strong>, not across your entire codebase.</p><p>Before jumping into code, let’s clarify some terms in the Go context:</p><h4>1. Constructor</h4><p>A function used to initialize and return a struct instance.<br> Go doesn’t have constructors like OOP languages, so we use functions like NewService().</p><h4>2. Wiring</h4><p>Centralized initialization of dependencies — usually done in main.go.<br> This is where constructors are called and dependencies are passed around.</p><h4>3. Concrete Type</h4><p>A real implementation (struct) that contains actual logic. Interfaces hide the implementation details.</p><blockquote>Lets begin with code</blockquote><h4>1. Database Layer</h4><pre>type AuthDB struct {<br> dbName      string<br> mongoURI    string<br> usersColl   string<br> mongoClient *mongo.Client<br> logger      *slog.Logger<br>}<br><br>func GetMongoDB(logger *slog.Logger) (*AuthDB, error) {<br> cfg, err := config.GetConfig()<br> if err != nil || cfg.Mongo == nil {<br>  logger.Warn(&quot;config is not loaded&quot;, &quot;error&quot;, err)<br>  return nil, fmt.Errorf(&quot;config is not loaded&quot;)<br> }<br><br> client, err := mongo.Connect(options.Client().ApplyURI(cfg.Mongo.URI))<br> if err != nil {<br>  logger.Error(&quot;failed to connect the mongodb&quot;, &quot;error&quot;, err)<br>  return nil, fmt.Errorf(&quot;failed to connect to mongo&quot;)<br> }<br><br> if err := client.Ping(context.Background(), nil); err != nil {<br>  logger.Error(&quot;Failed to connect with mongodb&quot;)<br>  return nil, fmt.Errorf(&quot;failed to ping the mongo&quot;)<br> }<br><br> logger.Info(&quot;Mongodb connected successfull&quot;)<br><br> return &amp;AuthDB{<br>  dbName:      cfg.Mongo.DBName,<br>  mongoURI:    cfg.Mongo.URI,<br>  usersColl:   cfg.Mongo.Collection,<br>  mongoClient: client,<br>  logger:      logger,<br> }, nil<br>}</pre><p>GetMongoDB acts as a <strong>constructor</strong>, returning a <strong>concrete implementation</strong>.</p><h4>2. Service Layer (Business Logic)</h4><pre>type AuthRepository interface {<br> GetUserByEmail(ctx context.Context, email string) (*models.User, error)<br> InsertUser(ctx context.Context, user models.User) (string, error)<br>}<br><br>type Service struct {<br> db     AuthRepository<br> logger *slog.Logger<br> router *chi.Mux<br>}<br><br>func NewService(authDB AuthRepository, logger *slog.Logger) *Service {<br> r := chi.NewMux()<br> return &amp;Service{<br>  db:     authDB,<br>  logger: logger,<br>  router: r,<br> }<br>}</pre><p>Why we have use an interface for DB?</p><ol><li>To enables <strong>mocking </strong>which make<strong> </strong>testing easier</li><li>Decouples service from database implementation. In future we can easily migrate from MongoDB to PostgreSQL.</li></ol><h4>3. Handler</h4><pre>type Handler struct {<br> svc    *services.Service<br> logger *slog.Logger<br>}<br><br>func InitHandler(svc *services.Service, logger *slog.Logger) *Handler {<br> return &amp;Handler{<br>  svc:    svc,<br>  logger: logger,<br> }<br>}</pre><p>The handler depends on the service — but doesn’t create it</p><h4>4. Main(wiring part)</h4><pre>package main<br><br>func main() {<br> loadEnv()<br><br> cfg, err := config.GetConfig()<br> if err != nil || cfg.Server == nil {<br>  log.Fatalf(&quot;server has yaml error %v&quot;, err)<br> }<br> // calling logger constructor<br> logger := logger.Initlogger(&quot;debug&quot;)<br> svc := setupService(cfg, logger)<br><br> server := setupServer(cfg, svc, logger)<br><br> if err := server.ListenAndServe(); err == nil {<br>  log.Println(&quot;Successfully started the server&quot;)<br> }<br><br>}<br><br>func loadEnv() {<br> if err := godotenv.Load(); err != nil {<br>  log.Printf(&quot;Warn: No env file found&quot;)<br> }<br>}<br><br>func setupService(cfg *config.Config, logger *slog.Logger) *services.Service {<br> // calling DB constructor<br> authDB, err := database.GetMongoDB(logger)<br> if err != nil {<br>  logger.Error(&quot;failed to intialized the database&quot;)<br>  log.Fatal(&quot;failed to init mongo&quot;)<br> }<br> // calling service constructor<br> service := services.NewService(authDB, logger)<br> return service<br>}<br><br>func setupServer(cfg *config.Config, service *services.Service, logger *slog.Logger) *http.Server {<br> if cfg == nil || cfg.Server.Port == &quot;&quot; {<br>  log.Fatalf(&quot;config file not loaded&quot;)<br>  return nil<br> }<br> // calling handler constructor<br> authHandler := handlers.InitHandler(service, logger)<br> // setting up routes and returning the http.Handler<br> handler := routes.SetupRoutes(authHandler, logger, service.GetServiceMux())<br> handlerWithCors := setupCors(handler)<br> server := http.Server{<br>  Handler: handlerWithCors,<br>  Addr:    &quot;:&quot; + cfg.Server.Port,<br> }<br> return &amp;server<br>}<br><br>func setupCors(handler http.Handler) http.Handler {<br> c := cors.New(cors.Options{<br>  AllowedOrigins: []string{<br>   &quot;*&quot;,<br>  },<br>  AllowedMethods: []string{<br>   http.MethodGet,<br>   http.MethodPost,<br>   http.MethodPatch,<br>   http.MethodPut,<br>   http.MethodDelete,<br>  },<br>  AllowedHeaders:   []string{&quot;*&quot;},<br>  ExposedHeaders:   []string{&quot;Content-Length&quot;},<br>  AllowCredentials: true,<br> })<br> return c.Handler(handler)<br>}</pre><p>In the main.go file, we initialize the logger and configuration first, which will be passed to every constructor during initialization. After that, we call the setupService function, which returns a concrete implementation of the service struct.</p><p>Inside the setupService function, we initialize the database and pass it to the service constructor. This allows the service to store the database instance once and reuse it whenever needed. We don’t need to initialize the database every time it’s used because it is passed by reference from the outside.</p><p>Next, we call setupServer, where we pass the service struct to the handler constructor. This function returns a handler struct, which is then passed to SetupRoutes. The SetupRoutes function registers routes like r.Get(&quot;/&quot;, h.GetPlainText) and returns an http.Handler, which is a chi.Mux.</p><p>From the above wiring code, you can see how all dependencies are initialized and passed to other services in a single place. In the future, if we need to change the database from MongoDB to PostgreSQL, we only need to make changes in one place — main.go. The handler methods can still access the service and the database (through the service) without any further modifications.</p><p>Some Disadvantage of DI:</p><ol><li>In main.go, you end up with a large block of code just wiring things together (creating DB, creating Repo, creating Service, creating Handler).</li><li>It can be harder to trace code execution. If you click “Go to definition” on an interface method in your IDE, it might not immediately be clear which implementation is actually running at runtime.</li><li>You have to define interfaces for almost everything. For small scripts or simple apps, this can feel like over-engineering (boilerplate code).</li></ol><p>Reference:</p><ol><li><a href="https://refactoring.guru/design-patterns/singleton">https://refactoring.guru/design-patterns/singleton</a></li><li><a href="https://youtu.be/kF7rQmSRlq0">https://youtu.be/kF7rQmSRlq0</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d228ef202050" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From 4.5 Hours to 11 Minutes: How a Go Worker Pool Transformed My Data Processing]]></title>
            <link>https://medium.com/@krishnakantmaurya1616/from-4-5-hours-to-11-minutes-how-a-go-worker-pool-transformed-my-data-processing-5a4acb776def?source=rss-c6110f9aa46e------2</link>
            <guid isPermaLink="false">https://medium.com/p/5a4acb776def</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[worker-pool]]></category>
            <category><![CDATA[script]]></category>
            <dc:creator><![CDATA[Krishna]]></dc:creator>
            <pubDate>Fri, 26 Dec 2025 03:32:41 GMT</pubDate>
            <atom:updated>2025-12-26T03:32:41.519Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rNpv3njBed0ZfGjvQH4wcw.png" /><figcaption>gopher image</figcaption></figure><p>Hello Techies,<br>This is my first Medium blog, and I am going to talk about the foolish thing I have ever done in my life. When you know the context, you will also say how someone can be so foolish.</p><blockquote><strong>Context:</strong> I had to process records of 3.7 lakh users from a CSV file. When I first opened the CSV file in Microsoft Excel, I saw it and mistakenly assumed that it contained only 37k records, which would be easy to process. That assumption became the worst nightmare of my life.</blockquote><p>So, I had to read the CSV file row by row, clean it up, extract the userId from the CSV, and make a database call to check whether the user exists or not. If the user did not exist, I had to store it in a log file. If the user existed, I had to fetch the data from the database and replace it with whichever data was missing in the CSV file, such as email, phone, or name. I also had to clean up the phone number because it was inconsistent in the records — it could contain +, &quot; symbols, or its length could sometimes be greater than or equal to 13 or 10.</p><p>After processing all this, I had to write a new CSV file that contained clean records of existing users.</p><p>So, to achieve the above task, I wrote a Golang script and processed each record one by one. My script was hitting the database for every user. This is where I made a mistake — I thought there were only 37k records, so it would hit the database only 37k times, which seemed fine. However, after an hour, I realized that I had made a mistake, and by then it was already too late. At that time, I had processed only 55k records 😂.</p><p>To process 3.7 lakh records, it took 4 hours and 36 minutes 😅.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iO_et7tuEHJQEu7JRawGiw.jpeg" /><figcaption><strong>Screenshot of total time taken to process the 3.7 lakh data</strong></figcaption></figure><p>From my mistake, I learned that you should never be in such a hurry; otherwise, you will suffer a lot. If I had not mistakenly assumed 3.7 lakh records to be only 37k, I could have written a much better script that would have taken only 20 to 30 minutes to complete the processing.</p><p>Let’s discuss the better approach that I should have taken from the beginning, whether the records were 37k or 3.7 lakh.</p><h3>Worker Pool Approach</h3><p>This is the best approach I have ever come up with in my life. It is simple and easy to understand. After some time, I decided to write a script that uses dependency injection and a worker pool.</p><p>I am going to discuss only the main logic, so let’s start……</p><ol><li><strong>Worker struct</strong></li></ol><p>This is going to hold all the things related to the worker pool policy.</p><pre>type Worker struct {<br> WorkerID     int<br> JobQueue     chan models.User<br> ResultsQueue chan []models.User<br> Wg           *sync.WaitGroup<br> Processor    *processor.Processors<br>}</pre><p>To initialize the worker struct, I created a constructor function that initializes all the channels and accepts the processor struct, which will call the business logic whenever it is needed.</p><pre>func InitWorker(numJob, numResult int, wg *sync.WaitGroup, <br>processor *processor.Processors) *Worker {<br> jobs := make(chan models.User, numJob)<br> results := make(chan []models.User, numResult)<br> return &amp;Worker{<br>  JobQueue:     jobs,<br>  ResultsQueue: results,<br>  Wg:           wg,<br>  Processor:    processor,<br> }<br>}</pre><p>The function below will be called by the main package to start the worker using a goroutine and pass the worker ID to the StartWorker function.</p><pre>func (w *Worker) GoWorker(numWorker int) {<br> for i := 0; i &lt; numWorker; i++ {<br>  w.Wg.Add(1)<br>  go w.StartWorker(i + 1)<br> }<br>}</pre><p>Now, the SendJobs function will read the CSV file and send the jobs to the Job channel one by one. Since our channel is a buffered channel, it will keep sending jobs until the channel is full and then stop until the channel becomes empty. If we have read all the CSV records and finished sending them into the channel, we will close it for safety to avoid deadlocks or panics.</p><pre>func (w *Worker) SendJobs(input string) {<br> file, err := os.Open(input)<br> if err != nil {<br>  log.Fatalf(&quot;failed to open the file&quot;)<br> }<br> defer file.Close()<br><br> csvReader := csv.NewReader(file)<br><br> headers, err := csvReader.Read()<br> colIndex := make(map[string]int, 0)<br> for idx, val := range headers {<br>  colIndex[strings.TrimSpace(strings.ToLower(val))] = idx<br> }<br><br> userIdx := colIndex[strings.TrimSpace(strings.ToLower(&quot;User ID&quot;))]<br> emailIdx := colIndex[strings.TrimSpace(strings.ToLower(&quot;Email&quot;))]<br> phoneIdx := colIndex[strings.TrimSpace(strings.ToLower(&quot;Phone&quot;))]<br> fNameIdx := colIndex[strings.TrimSpace(strings.ToLower(&quot;First Name&quot;))]<br> lNameIdx := colIndex[strings.TrimSpace(strings.ToLower(&quot;Last Name&quot;))]<br><br> for {<br>  row, err := csvReader.Read()<br>  if err == io.EOF {<br>   break<br>  }<br>  var user models.User<br>  user.ID, _ = bson.ObjectIDFromHex(row[userIdx])<br>  user.Email = row[emailIdx]<br>  user.CountryCode = &quot;91&quot;<br>  user.Name = row[fNameIdx] + &quot; &quot; + row[lNameIdx]<br>  user.Phone = row[phoneIdx]<br><br>  w.JobQueue &lt;- user<br><br> }<br> close(w.JobQueue)<br>}</pre><p>The ProcessResult function will listen to the result channel (a buffered channel of size 1000) and wait for data to come. When it receives data, it will write it to a new CSV file, which will contain clean records of existing users.</p><pre>func (w *Worker) ProcessResult(output string) {<br> file, err := os.Create(output)<br> if err != nil {<br>  log.Fatalf(&quot;failed to create the file %v&quot;, err)<br> }<br> defer file.Close()<br> csvWriter := csv.NewWriter(file)<br> defer csvWriter.Flush()<br> csvWriter.Write([]string{&quot;UserID&quot;, &quot;Name&quot;, &quot;Phone&quot;, &quot;Email&quot;})<br> for res := range w.ResultsQueue {<br>  for _, user := range res {<br>   w.Processor.WriteToCsv(csvWriter, user)<br>  }<br><br> }<br>}</pre><p>This is the main logic where each worker will do their work. Each worker will listen to the jobs channel, and if any data comes in, it will append it to the jobs slice. When its size becomes 100, it will trigger or call the business logic using dependency injection.</p><p>When all the business logic is done, it will give us two things: either an error or results. If the error is nil, we will send the results to the result channel. If the error exists, we will simply print it in the logs. Currently, we are not writing errors to a log file.</p><pre>func (w *Worker) StartWorker(workerID int) {<br> defer w.Wg.Done()<br> jobs := make([]models.User, 0)<br> for job := range w.JobQueue {<br>  jobs = append(jobs, job)<br>  if len(jobs)%100 == 0 {<br>   result, err := w.Processor.StartProcessor(jobs)<br>   if err != nil {<br>    if errors.Is(err, models.ErrNoFound) {<br>     log.Printf(&quot;user not found in db %s&quot;, job.ID.Hex())<br>     continue<br>    }<br>    log.Printf(&quot;failed to fetch the db of userId %s error %v&quot;, job.ID.Hex(), err)<br>   }<br>   w.ResultsQueue &lt;- result<br>   log.Printf(&quot;worker id(%d) has successfully processesd the job total no. processed %d&quot;, workerID, len(result))<br>   jobs = make([]models.User, 0)<br>   time.Sleep(1 * time.Second)<br>  }<br> }<br>// some job might be left over like at last only 50 job left so its never reaches<br>// to the processor to processed it. so we have to explicitily send it <br> if len(jobs) &gt; 0 {<br>  if result, err := w.Processor.StartProcessor(jobs); err == nil {<br>   w.ResultsQueue &lt;- result<br>  }<br> }<br>}</pre><p>This all the worker pool code. Now we will see business logic…</p><p>2. <strong>Business Logic</strong></p><p>I have written all my business logic in the processor package. I have tried to structure the code in such a way that it is easier to make changes. For example, if we need to add another piece of business logic before writing into the new CSV file, we can directly go to the processor file and implement the new logic.</p><p>This is the processor struct, which holds the database struct. It makes it easier to call database functions whenever needed.</p><pre>type Processors struct {<br> db *db.MongoDB<br>}<br><br>func InitProcessor(db *db.MongoDB) *Processors {<br> return &amp;Processors{<br>  db: db,<br> }<br>}</pre><p>The code below contains the main business logic. I perform all the computations, clean up the data, and return the list of users.</p><pre>func (p *Processors) StartProcessor(oldUsers []models.User) ([]models.User, error) {<br> userIds := make([]string, 0)<br> oldUserMp := make(map[string]models.User, 0)<br> for _, user := range oldUsers {<br>  userIds = append(userIds, user.ID.Hex())<br>  oldUserMp[user.ID.Hex()] = user<br> }<br> currUsers := p.db.FindUsers(userIds)<br> resUsers := make([]models.User, 0)<br> for _, currUser := range currUsers {<br><br>  if oldData, ok := oldUserMp[currUser.ID.Hex()]; ok {<br><br>   // checks if csv missing the email logic<br>   if oldData.Email == &quot;&quot; &amp;&amp; currUser.Email != &quot;&quot; {<br>    oldData.Email = currUser.Email<br>   }<br><br>   // check if phone number consisting the + or length is more than 12 or equal to 12 then trim it to 10 digit<br>   oldData.Phone = utils.CheckAndFixPhoneNumber(oldData.Phone, currUser.Phone)<br><br>   resUsers = append(resUsers, oldData)<br>  }<br> }<br> return resUsers, nil<br>}</pre><p>This function basically writes to the new CSV file whenever it receives results in the resultsQueue.</p><pre>func (p *Processors) WriteToCsv(writer *csv.Writer, user models.User) {<br> writer.Write([]string{user.ID.Hex(), user.Name, user.Phone, user.Email})<br>}</pre><p>Now, coming to main.go, for database functions and structs, you can refer to my <a href="https://github.com/krishna102001/worker-pool">GitHub</a> code links.<br> This is where all the initializations happen, and it calls the other functions.</p><pre>const (<br> mongoUri          = &quot;mongodb://localhost:27017/&quot;<br> dbName            = &quot;workerpool&quot;<br> collName          = &quot;users&quot;<br> isSeedEnable      = false<br> isProcessorEnable = true<br> numWorker         = 5<br> numJob            = 500<br> numResult         = 1000<br> inputFile         = &quot;./input/users.csv&quot;<br> outputFile        = &quot;./output/processed_file.csv&quot;<br>)<br><br>func main() {<br> database, err := db.InitMongo(mongoUri, dbName, collName)<br> if err != nil {<br>  log.Fatalf(&quot;failed to connect to monodb %v&quot;, err)<br> }<br> if isSeedEnable {<br>  startTime := time.Now()<br>  database.SeedValue()<br>  log.Println(&quot;time taken to seed the value &quot;, time.Since(startTime))<br> }<br> if isProcessorEnable {<br>  processor := processor.InitProcessor(database)<br>  var wg sync.WaitGroup<br>  worker := worker.InitWorker(numJob, numResult, &amp;wg, processor)<br>  startTime := time.Now()<br>  worker.GoWorker(numWorker)<br>  go func() {<br>   worker.SendJobs(inputFile)<br>  }()<br><br>  writerDone := make(chan bool)<br>  go func() {<br>   worker.ProcessResult(outputFile)<br>   writerDone &lt;- true<br>  }()<br>  wg.Wait()<br>  close(worker.ResultsQueue)<br>  &lt;-writerDone<br>  log.Println(&quot;time taken to Processed the data &quot;, time.Since(startTime))<br> }<br>}</pre><p>This new Golang script is one of the best scripts I have written, and it drastically decreased the time it takes to process the data. I simulated it again with a CSV file containing 3.8 lakh records, while my database held only 1.9 lakh users. Since it iterated over all 3.8 lakh records, it took a total of 7 minutes on my local database. On the cloud database, it took approximately 11 minutes and 39 seconds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SIQ79rKpgUlCvtWsMQznzQ.png" /><figcaption>Total time taken to process the 3.8Lac record</figcaption></figure><p>In the future, even if I increase the business logic, it will barely take half an hour, which is a drastic improvement from 4 hours 36 min → 11 min 39 seconds. I have learned from my mistake, and by reading this blog, you can also learn from it and avoid similar chaos in the future.</p><p>Checkout the code :- <a href="https://github.com/krishna102001/worker-pool">https://github.com/krishna102001/worker-pool</a></p><p>Thank you.<br>In the next blog, we will learn about either Kubernetes or dependency injection.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5a4acb776def" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>