From the course: AWS Certified Machine Learning Engineer - Associate (MLA-C02) Cert Prep

Unlock this course with a free trial

Join today to access over 26,500 courses taught by industry experts.

AWS Glue job bookmarks explained

AWS Glue job bookmarks explained

From the course: AWS Certified Machine Learning Engineer - Associate (MLA-C02) Cert Prep

Start my 1-month free trial Buy for my team

AWS Glue job bookmarks explained

“

Picture this. You're watching your favorite Netflix series. You pause at episode 3 minute 21. When you return tomorrow, would you start again from episode 1 or continue right where you left off? That's exactly what AWS GlueJob bookmarks do for your data. While some data pipelines waste time and money reprocessing the same information repeatedly, GlueJob bookmarks let you process only what's new. They help you save compute resources, speed up your workflows, and keep your pipeline smart and efficient. If your ETL jobs keep starting from scratch every time, this feature is what will finally make them remember. So what exactly are AWS Glue job bookmarks? And why should data engineers care? care. AWS Glue job bookmarks are a state-tracking feature that remembers which data has already been processed during previous ETL runs. Think of it as your job's built-in memory. It keeps track of progress and ensures that future runs pick up exactly where the last one stopped. Instead of blindly…

Contents