AI Models & Data — Tryhackme Walkthrough
Task 1: Introduction
Learning Objectives:
- Understand where AI training data comes from and the security risks introduced by poor data provenance
- Recognise how PII and sensitive credentials can become permanently embedded in model weights through large-scale web scraping
- Understand how key model-building decisions (overfitting, quantisation, and federated learning) each introduce distinct security risks
- Understand the inheritance problem and what organisations unknowingly take on when fine-tuning pre-trained models
- Recognise why trained models are opaque black boxes, and what model cards do (and fail to do) to address this
Task 2: Training Data
What term describes the ability to answer where data came from, when it was collected, and whether it has been modified?
Answer: Data Provenance
What is the name of the most widely used public corpus that underpins essentially every major model family?
Answer: Common Crawl
What is the AI equivalent of a Software Bill of Materials (SBOM), used to document dataset sources, licenses, and filtering decisions?
Answer: ML-BOM
Task 3: Building the Model
What term describes one complete pass of the training algorithm through the entire dataset?
Answer: Epoch
What problem occurs when a model memorises training data rather than learning general patterns?
Answer: Overfitting
What post-training optimisation technique reduces the numerical precision of model weights to cut memory and compute requirements?
Answer: Quantisation
What training approach trains a model across decentralised devices, sending only weight updates rather than raw data to a central server?
Get Rotimi Ishola’s stories in your inbox
Join Medium for free to get updates from this writer.
Answer: Federated Learning
Task 4: The Inheritance Problem
What is the process of taking a pre-trained model and continuing to train it on a smaller, task-specific dataset?
Answer: Fine-tuning
What term describes a model that has already been trained on a large general-purpose dataset?
Answer: Pre-trained Model
Task 5: The Black Box Problem
What documentation artifact accompanies a model to describe what it is, how it was built, and where it falls short?
Answer: Model Card
What are the billions of floating-point numbers that make up a trained model collectively referred to as?
Answer: Weights
Task 6: Practical
- Open the view site
- on the hugging face, click on the model card:
- The process to get answer is in this link:
https://vimeo.com/1183660858?share=copy&fl=sv&fe=ci
Task 7: Conclusion
Here’s a summary of this room’s key takeaways:
- AI training data is drawn from poorly documented, unaudited sources, meaning most organisations have no reliable answer to where their training data came from, what it contained, or whether it was tampered with
- PII and live credentials routinely end up baked into model weights through large-scale web scraping and cannot be patched out once the model is deployed
- Model-building decisions such as quantisation and federated learning introduce security trade-offs that are rarely documented, meaning organisations inherit unknown behaviour modifications alongside efficiency gains
- Fine-tuning a pre-trained model inherits everything beneath it: safety alignment erodes with as few as 10 adversarial examples, and fine-tuned models are measurably more susceptible to prompt injection than their base counterparts
- Trained model weights are fundamentally opaque; security testing can only sample behaviour rather than audit it, and model cards (the primary transparency mechanism) remain voluntary, frequently incomplete, and sometimes absent entirely
I hope it was helpful to you. Follow me on Linkedln: https://www.linkedin.com/in/rotimi-ishola/ and X: https://x.com/Timi_phil
