Mageswaran D·Apr 16, 2022How to keep Cloud cost under control?Collect the hardware metrics (CPU/memory utilization, disk I/O)
Mageswaran D·Mar 24, 20222022 : No FileSystem for scheme: s3?How many times did you face this error? S3 or S3a schema not found error while working with AWS S3 + Spark outside the EMR?A response icon5A response icon5
Mageswaran D·Dec 15, 2021Dynamic Programming Patterns to Ace InterviewsFollowing materials are taken from freecodeCamp.org Youtube video course https://www.youtube.com/watch?v=oBt53YbR9Kk strongly recommend to…
Mageswaran D·Dec 8, 2021What if I wanted to submit remote PySpark jobs to AWS EMR without worrying about library dependen…Typically Spark cluster is used to run ETL jobs or some streaming jobs, where everything is managed by developers for developers, where…
Mageswaran D·Nov 29, 2021NLP: What it takes to design a full stack DeepLearning based Receipts form filling system using NER?Online Colab Notebook for model training.
Mageswaran D·Nov 10, 2021NLP: What it takes to model Google search suggestions or auto complete?Dataset : google_wellformed_query 25K examplesA response icon1A response icon1
Mageswaran D·Oct 28, 2021How to build custom NER HuggingFace dataset for receipts and train with HuggingFace Transformers…Disclaimer: It is assumed that you have some working knowledge in Hugging face library and datasets library, to begin with.A response icon1A response icon1
Mageswaran D·Oct 26, 2021A dive into Apache Spark Parquet Reader for small size filesWhile working part of my current project, I was asked how Spark reads Parquet files and how does it achieves the parallelisation. Very…A response icon3A response icon3
Mageswaran D·May 24, 2021Some interesting interview Q&A around “randomness”1 How to sample a random Element from an infinite stream?