<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by OSTC on Medium]]></title>
        <description><![CDATA[Stories by OSTC on Medium]]></description>
        <link>https://medium.com/@ostradecraft?source=rss-85cc242cc611------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*t89I-O_936hvcxuLTA0ybw.jpeg</url>
            <title>Stories by OSTC on Medium</title>
            <link>https://medium.com/@ostradecraft?source=rss-85cc242cc611------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 11:50:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ostradecraft/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Mosaic Effect: Why “Harmless” Data Isn’t Harmless]]></title>
            <link>https://ostradecraft.medium.com/the-mosaic-effect-why-harmless-data-isnt-harmless-bc63bbd3d072?source=rss-85cc242cc611------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc63bbd3d072</guid>
            <category><![CDATA[open-source-intelligence]]></category>
            <category><![CDATA[opsec]]></category>
            <category><![CDATA[data-privacy]]></category>
            <category><![CDATA[osint]]></category>
            <dc:creator><![CDATA[OSTC]]></dc:creator>
            <pubDate>Tue, 10 Jun 2025 06:37:52 GMT</pubDate>
            <atom:updated>2025-06-10T06:38:19.074Z</atom:updated>
            <content:encoded><![CDATA[<p>In Open‑Source Intelligence (OSINT) and Operational Security (OPSEC), the <strong>Mosaic Effect</strong> refers to how seemingly innocuous, unclassified, or anonymized pieces of data — harmless in isolation — can be aggregated to create a revealing whole. Picture a jigsaw puzzle: a single piece tells you very little, but once assembled, the full image emerges.</p><p>The term originates from intelligence community discourse and has long been used to justify withholding or redacting data under FOIA requests by citing national security concerns (<a href="https://e-pluribusunum.org/2013/05/20/open-data-mosaic-effect/?utm_source=chatgpt.com">e-pluribusunum.org</a>). It’s also one of the five ‘threat vectors’ in the OSINT lifecycle, alongside consent and algorithmic bias (<a href="https://www.shrmonitor.org/assets/uploads/2023/05/article-Millett.pdf?utm_source=chatgpt.com">shrmonitor.org</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*W238_GcKaokZNMVWaB02TA.jpeg" /></figure><h3>Origins &amp; Definitions</h3><p>The Mosaic Effect surfaced in U.S. intelligence and legal contexts in the 1970s as a basis to refuse FOIA disclosures — officials argued that aggregated innocuous data could reveal sensitive operations. It’s now formally defined as:</p><blockquote><em>“Disparate pieces of data or information — although individually of limited utility — become significant when combined with other types of information” (</em><a href="https://centre.humdata.org/glossary-2/mosaic-effect/?utm_source=chatgpt.com"><em>centre.humdata.org</em></a><em>).</em></blockquote><p>It’s also described as a “data linkage attack,” where cross-referencing datasets — even ones that are anonymized — can re‑identify individuals or reveal secret facilities (<a href="https://centre.humdata.org/exploring-the-mosaic-effect-on-hdx-datasets/?utm_source=chatgpt.com">centre.humdata.org</a>).</p><h3>Illustrative Examples</h3><h4>Government Open Data Platforms</h4><p>At the 2014 <em>FOSE</em> conference, officials from Data.gov (then 90,000 datasets strong) warned that aggregating separate safe datasets could expose troop movements or sensor coverage (<a href="https://www.route-fifty.com/cybersecurity/2014/05/worried-about-security-beware-the-mosaic-effect/297335/?utm_source=chatgpt.com">route-fifty.com</a>). While only one dataset was flagged for removal, the potential risk prompted agencies to begin proactively scanning combinations for obscure links.</p><h4>Humanitarian Data Risks</h4><p>In humanitarian settings, combining public demographic surveys with refugee camp maps — even when anonymized — can identify vulnerable individuals or demobilized child soldiers (<a href="https://www.syedpeer.net/infosec/osint-challenges/?utm_source=chatgpt.com">syedpeer.net</a>, <a href="https://centre.humdata.org/exploring-the-mosaic-effect-on-hdx-datasets/?utm_source=chatgpt.com">centre.humdata.org</a>). One study at the HDX platform found that linking age, gender, location, and assistance needs could pinpoint minority groups (<a href="https://centre.humdata.org/exploring-the-mosaic-effect-on-hdx-datasets/?utm_source=chatgpt.com">centre.humdata.org</a>).</p><h4>Counter‑Surveillance &amp; Cover Identity Exposure</h4><p>Undercover police officers in London’s SDS unit warned that tiny leaks — appearance in a newsletter, property records, retired IDs — could be combined by activists or journalists to expose officer identities (<a href="https://policespiesoutoflives.org.uk/they-call-it-the-mosaic-effect/?utm_source=chatgpt.com">policespiesoutoflives.org.uk</a>). What seems trivial — a home sale, school registration — can unravel deep cover identities.</p><h4>Cyber‑Enabled Breaches</h4><p>Cyber attackers regularly use OSINT to enhance weapons systems and target phishing campaigns. The 2015 CENTCOM breach, for example, was enabled by gathering staff data from social media to craft convincing phishing lures (<a href="https://www.cobalt.io/blog/security-breaches-open-source-intelligence-osint-oversights?utm_source=chatgpt.com">cobalt.io</a>).</p><h3>Why This Matters for OPSEC &amp; OSINT</h3><p>The Mosaic Effect isn’t a theoretical threat, it’s a practical tool and a real hazard in today’s OSINT/OPSEC landscape.</p><h4>Defender’s Spotlight: OPSEC Risks</h4><ul><li><strong>Invisible Fingerprints</strong>: OPSEC failures often stem from not recognizing the hidden patterns in public data. For example, sharing routine routines, Wi-Fi SSIDs, or family schedules may seem innocuous — but aggregated with travel logs or forums, they offer adversaries profiles for spear‑phishing or even physical targeting.</li><li><strong>De‑anonymization</strong>: Simply redacting names or personal IDs isn’t enough. Studies show that knowing four data points (e.g. date of birth, ZIP code, gender, occupation) can re-identify up to 95% of individuals (<a href="https://www.shrmonitor.org/assets/uploads/2023/05/article-Millett.pdf?utm_source=chatgpt.com">shrmonitor.org</a>).</li><li><strong>Unforeseen linking across platforms</strong>: A selfie posted to Instagram geotagged to a café, combined with a public forum mention of “at the weekly meetup,” could reveal a secret gathering’s location.</li></ul><h4>Attacker’s Toolkit: OSINT Value</h4><ul><li><strong>Phishing &amp; targeted campaigns</strong>: Aggregated employee data exposes org structures, preferred social networks, and even IT infrastructure — all of which sharpen attack targeting.</li><li><strong>Social network mapping</strong>: Cross‑platform identity linkage (LinkedIn, Facebook, X, Telegram) can reconstruct an individual’s acquaintances, movement patterns, and risk posture — what one analyst called a “comprehensive intelligence mosaic.”</li></ul><h3>Defensive Measures: An OPSEC Framework</h3><h4>Data Threat Modeling</h4><p>Before releasing any dataset — internal or public — conduct a <strong>threat model</strong>: enumerate potential adversaries, the data they may combine, and the sensitivity of the emergent dataset. This mirrors the OPSEC Five‑Step process: identify critical information, threats, vulnerabilities, risks, and countermeasures (<a href="https://www.route-fifty.com/cybersecurity/2014/05/worried-about-security-beware-the-mosaic-effect/297335/?utm_source=chatgpt.com">route-fifty.com</a>, <a href="https://github.com/Jieyab89/OSINT-Cheat-sheet/wiki/OSINT-for-OPSEC-%28Operational-security%29?utm_source=chatgpt.com">github.com</a>).</p><h4>Cross-Referencing (Red Teaming)</h4><p>Have analysts or external auditors simulate attacker behavior — combining redacted datasets with open sources to see what’s revealed. This “mosaic stress test” helps identify hidden linkages.</p><h4>Anonymization Beyond Basics</h4><p>Removing direct identifiers isn’t enough as statistical re-identification remains possible (<a href="https://www.shrmonitor.org/assets/uploads/2023/05/article-Millett.pdf?utm_source=chatgpt.com">shrmonitor.org</a>), thus requiring advanced anonymization methods such as k-anonymity, l-diversity, and noise-injection, among others (<a href="https://www.wallarm.com/what/data-anonymization">wallarm.com</a>).</p><h4>Periodic Review</h4><p>Since public data evolves, routinely reassess older releases for new linkage risks, and once innocuous data might combine with new sensor feeds or crowd-sourced maps to become sensitive. Data leakage from the Strava fitness app is a prime example of this (<a href="https://www.securityweek.com/fitness-app-strava-gives-away-location-of-biden-trump-and-other-leaders-french-newspaper-says/">securityweek.com</a>).</p><h4>Staff Training &amp; Culture</h4><p>Embed OSINT/OPSEC awareness across teams. Encourage caution with personal postings, geotags, travel disclosures, or family mentions that, in aggregate, contribute to a mosaic image.</p><h3>Policy and Ethical Considerations</h3><h4>Transparency vs. Privacy</h4><p>Open‑government advocates worry that Mosaic Effect concerns become excuses for excessive secrecy (<a href="https://e-pluribusunum.org/2013/05/20/open-data-mosaic-effect/?utm_source=chatgpt.com">e-pluribusunum.org</a>). It’s vital that data controls are evidence-based, not blanket bans. Agencies like Data.gov modify only one dataset, suggesting risk is real, but not paralyzing (<a href="https://www.route-fifty.com/cybersecurity/2014/05/worried-about-security-beware-the-mosaic-effect/297335/?utm_source=chatgpt.com">route-fifty.com</a>).</p><h4>Humanitarian Data Ethics</h4><p>Transparency can aid accountability, but it must be weighed against risks to vulnerable populations. In conflict zones, exposing geolocated data might threaten life. Ethical data sharing should include multi-stakeholder review and harms-based licensing.</p><h4>Legal Implications</h4><p>In armed conflict settings, aggregated OSINT may harm civilians or violate privacy laws. International humanitarian law isn’t explicit, but principles of confidentiality and proportionality apply (<a href="https://blogs.icrc.org/law-and-policy/2023/12/05/deploying-osint-in-armed-conflict-settings-law-ethics-theory-of-harm/?utm_source=chatgpt.com">blogs.icrc.org</a>).</p><h3>Seeing the Whole Picture</h3><p>The Mosaic Effect reveals a central truth of modern intelligence: It’s not the data itself, but the <em>patterns</em> across data that matter. For OSINT practitioners, it’s a potent tool; for OPSEC, a potential threat.</p><ul><li><strong>OSINT professionals</strong> should harness mosaic-based analysis but also respect privacy limits.</li><li><strong>Operational security teams</strong> must expand protections beyond the obvious — defending against linkage-based inferences from even seemingly trivial posts or leaks.</li></ul><p>The Mosaic Effect challenges traditional notions of information sensitivity. No longer can we treat data in a vacuum; instead, we must anticipate how it will interconnect, today and tomorrow. By adopting threat modeling, red-team-style linkage testing, rigorous anonymization, and ongoing risk reviews — with clear ethical and policy guardrails — organizations can balance transparency with protection.</p><p>By treating public data as living pieces of a puzzle — not isolated snapshots — OSINT and OPSEC actors alike can navigate this complex intelligence terrain. The Mosaic Effect reminds us: Every piece counts and could reveal more than we ever intended.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc63bbd3d072" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Social Media: OSINT’s Boundless Frontier]]></title>
            <link>https://ostradecraft.medium.com/social-media-osints-boundless-frontier-b492f99394da?source=rss-85cc242cc611------2</link>
            <guid isPermaLink="false">https://medium.com/p/b492f99394da</guid>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[open-source-tradecraft]]></category>
            <category><![CDATA[intelligence]]></category>
            <category><![CDATA[open-source-intelligence]]></category>
            <category><![CDATA[osint]]></category>
            <dc:creator><![CDATA[OSTC]]></dc:creator>
            <pubDate>Mon, 25 Nov 2024 23:58:01 GMT</pubDate>
            <atom:updated>2024-11-26T00:04:48.664Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z9y0JOGHAMrswzhSKEnu8w.png" /></figure><p>Social media stands at the core of Open Source Intelligence (OSINT), serving as an indispensable hub of real-time information and a seemingly endless torrent of user-generated content. From personal details and opinions to multimedia posts and geolocation data, platforms like Facebook, X (formerly Twitter), Instagram, LinkedIn, TikTok, and numerous others, offer a goldmine of information for anyone looking to understand behaviors, trends, and connections. Each platform has its own personality, reflecting different aspects of users’ lives.</p><p>Take Facebook, for example — it’s a treasure trove for gathering information about individuals and their social networks, interests, and affiliations, with billions of profiles, groups, and pages offering a vast reservoir of data. X, with its emphasis on real-time updates, is a go-to source for tracking breaking news, public sentiment, and emerging trends. Instagram and TikTok, primarily focused on visual elements, can provide clues about locations or context through shared photos and videos. While LinkedIn is the place to dig into professional networks, job histories, technical details, and organizational hierarchies.</p><p>Then there are niche platforms like Reddit and specialized forums. These communities often host in-depth discussions on specific topics, offering unique insights into particular interests or ideologies.</p><h3>Key Investigative Techniques</h3><p>The sheer volume and variety of information available on social media make it an unparalleled resource for OSINT. When pieced together thoughtfully, it can help practitioners uncover complex stories and generate actionable insights. Below are some of the most effective techniques used in social media OSINT.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*oiLe-6K_hv2zJvkW0ai1nQ.jpeg" /></figure><h4>Profile Analysis</h4><p>Profile analysis is where many investigations start. By looking at publicly available profiles, it’s possible to gather information about individuals or groups — names, locations, job titles, affiliations, and more. These details can help map relationships and uncover motivations. Digging deeper into interactions between profiles can reveal connections between people, organizations, or even entire networks that might not be obvious at first glance.</p><h4>Network Analysis</h4><p>Social media isn’t just about individual profiles; it’s a web of interconnected networks. Mapping out these relationships is a powerful way to identify key influencers, spot hidden links, or understand how information flows through different ecosystems. Network analysis helps practitioners zero in on the most influential nodes within a network, making it easier to focus on the people or groups that matter most.</p><h4>Content Analysis</h4><p>The content people share — whether it’s text, photos, or videos — can tell a lot. Text-based content like posts, comments, and hashtags can reveal emerging narratives, trends, or even brewing controversies. By monitoring these discussions, practitioners can track sentiment, key themes and topics, and behavior patterns.</p><p>Visual content adds another layer of insight. Photos and videos often contain crucial details, like timestamps, landmarks, or environmental clues. Analyzing these elements can help geolocate people or events, reconstruct timelines, or confirm (or debunk) the authenticity of claims.</p><h3>Challenges and Dilemmas</h3><p>While social media is an incredibly powerful tool, it’s not without its challenges — and its ethical gray areas.</p><p>First off, there’s the sheer amount of data being generated every second. Sorting through this mountain of information to find what’s actually useful can be overwhelming. On top of that, social media content is often fragmented, biased, or outright misleading. Ensuring accuracy means cross-checking and rigorous verification.</p><p>Then there’s the tricky issue of privacy. Sure, a lot of social media data is public, but the line between what’s public and what feels private can get blurry. OSINT practitioners need to be careful not to overstep, infringing on privacy rights or violating ethical standards. Following terms of service and adhering to laws around data use are non-negotiables for maintaining trust and integrity in this field.</p><p>Social media platforms themselves add another layer of complexity. They’re constantly evolving, with updates to algorithms, features, and privacy settings that can change how (or even if) data can be accessed. Staying ahead means adapting methods and tools on the fly.</p><p>Finally, bias is a big concern — both in the data itself and in the perspective of the analyst. Identifying and addressing biases is key to ensuring that intelligence is balanced, objective, and, most importantly, actionable.</p><h3>Adapt or Perish</h3><p>Social media has completely transformed how OSINT is done. It offers real-time access to a wealth of information about individuals, groups, and organizations that was unimaginable around a decade ago. But with great power comes great responsibility. Data accuracy, ethical considerations, and the ever-changing nature of platforms demand careful navigation.</p><p>As new platforms emerge and privacy laws evolve, OSINT tools and techniques will need to keep up. For those who can unceasingly adapt, the opportunities to uncover insights and make an impact are endless.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b492f99394da" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The 6 Pillars of OSINT Documentation]]></title>
            <link>https://osintteam.blog/the-six-pillars-of-osint-documentation-59883c8f5927?source=rss-85cc242cc611------2</link>
            <guid isPermaLink="false">https://medium.com/p/59883c8f5927</guid>
            <category><![CDATA[osint]]></category>
            <category><![CDATA[information-gathering]]></category>
            <category><![CDATA[evidence]]></category>
            <category><![CDATA[open-source-intelligence]]></category>
            <category><![CDATA[documentation]]></category>
            <dc:creator><![CDATA[OSTC]]></dc:creator>
            <pubDate>Thu, 09 Feb 2023 11:38:59 GMT</pubDate>
            <atom:updated>2023-08-31T06:15:47.030Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*Nh_uJ_OxzmJ5hSNLzsq8Dw.jpeg" /></figure><p>If you do any kind of online investigative or research work, then you’re no stranger to the rush of finding hidden information about your subject, connecting pieces of the puzzle, and following the trail of evidence to see where it leads. In all this excitement, who has time to stop and think about such a mundane task as record keeping?</p><p>But let me ask you this, have you ever found yourself in a situation where you had all the information you needed to make your case, but you couldn’t prove its authenticity or retrieve it when you needed it? This is where documentation comes in, and it’s more important than you may think.</p><p>In this article, I’ll take you on a journey to discover the significance of documentation in Open Source Intelligence (OSINT). I want to show you how it can be a valuable tool in your toolkit, and most importantly, how to make it an integral part of your workflow.</p><p>I promise you, with a little effort, documenting your findings can be just as rewarding and (dare I say) fun as the actual hunt itself. So, let’s get started, shall we?</p><h4>What is it?</h4><p>According to the Oxford Dictionary, documentation is “the process of classifying and annotating texts, photographs, etc,” or “material that provides official information or evidence or that serves as a record.”</p><p>In the case of OSINT, both definitions apply because documentation is the process of recording each individual finding, which ultimately results in a full inventory of all your findings. Think of it this way: To create a library you need to collect lots of books in one place. But randomly placing books on shelves won’t do, because what good is a library where you can never find the book you actually need? To make it work, you must carefully label, catalog, and place each book on the correct shelf.</p><h4>Why document?</h4><p>Although it isn’t always the most glamorous of tasks, documentation can pay some serious dividends in the form of…</p><ul><li><strong>Organization: </strong>Much like in our library analogy, documentation helps keep everything in order and makes it much easier to quickly find what you’re looking for.</li><li><strong>Compliance: </strong>Sometimes there are legal or regulatory requirements that investigators must follow. The documentation process can double as a kind of checklist, enabling you to methodically address each requirement and provide proof that you didn’t break any laws.</li><li><strong>Evidence Preservation: </strong>If you ever need to prove where you got your information from or how you came to a certain conclusion, documentation is your best friend. It’s like a paper trail of your research.</li><li><strong>Collaboration: </strong>When you’re working with a team on a joint project, it’s important that everyone is on the same page (quite literally when compiling or editing reports). Good documentation ensures everyone on the team stays aware of all updates and that they’re all using the same version of the data.</li><li><strong>Auditing and Accountability: </strong>It’s important to be able to look back and see what you’ve done and how you’ve done it. Documentation helps with that too, and it also promotes accountability and transparency by forcing you to consider specific policy and methodology questions throughout the process.</li><li><strong>Continuity: </strong>Some OSINT investigations may run for weeks, months, or even years. During that time, the people working on the project may be replaced by others. Good documentation helps keep track of a project’s progress and history regardless of who is doing the research.</li></ul><h4>How do you do it?</h4><p>OSINT documentation involves the use of several methods, including raw data downloads, screen captures, third-party archives, site/page downloads, data hashes, and note-taking.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XzdnNgLnZd4o-rImNEH1KA.png" /></figure><p>These methods aren’t mutually exclusive and you’ll often use many of them in conjunction with one another. But having a clear plan in place to consistently apply them will result in retraceable processes, reproducible results, available/accessible data, and reliable evidence.</p><h4>1. Data/File Downloads</h4><p>This refers to capturing full datasets/files directly from the source without any modification or processing. Having access to complete, original content can be extremely useful for information verification and for checking the authenticity of findings. It also enables you to go back and re-analyze the information whenever necessary.</p><p>You’ll often be able to download images and videos by simply right-clicking and selecting <em>Save as</em>. In other cases, you may need to use browser extensions such as <a href="https://www.downloadhelper.net/">DownloadHelper</a> and <a href="https://chrome.google.com/webstore/detail/image-downloader-imageye/agionbommeaifngbhincahgmoflcikhm">Imageye</a>, or you might even try to dig into a website’s HTML code to uncover downloadable content in a different resolution or higher-quality format.</p><p>To download other file types including data tables or the text of a webpage, you can use browser-based tools such as <a href="https://chrome.google.com/webstore/detail/simple-mass-downloader/abdkkegmcbiomijcbdaodaflgehfffed">Simple mass downloader</a>, <a href="https://chrome.google.com/webstore/detail/instant-data-scraper/ofaokhiedipichpaobibbnahnkdoiiah">Instant Data Scraper</a>, and <a href="https://chrome.google.com/webstore/detail/web-scraper-free-web-scra/jnhgnonknehpejjnehehllkliplmbmhn?hl=en">Web Scraper</a>, or stand-alone applications like <a href="https://www.parsehub.com/">ParseHub</a> and <a href="https://www.octoparse.com/">Octoparse</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xmSeKcFtrqxYcnfLvAhsMA.png" /><figcaption>Simple Mass Downloader dialogue box with options for downloading page files.</figcaption></figure><p>On the other hand, if you know a programming/scripting language like Python, which I highly recommend learning, you can even build your own custom web scrapers and downloaders (among many other things).</p><p><em>For more detailed examples of the numerous uses of Python in OSINT, read this article:</em></p><p><a href="https://ostradecraft.medium.com/a-little-python-can-go-a-long-long-way-in-osint-bb2b3aab1d5b">A little Python can go a long, long way (in OSINT)</a></p><p>The good news is that downloading data/files is an integral part of the collection phase of the OSINT cycle, so you’re probably already doing this anyway. Two things to keep in mind are to always download the best available version of the file or data (including the original metadata if possible) and to keep your downloads neatly organized and categorized to make it easy to find them later on.</p><h4>2. Screen Captures</h4><p>Screen capture is a general term for taking screenshots or screen recordings. It’s particularly useful in cases where you want to capture not only the data or file but also the context in which it was posted. It’s a good method to use when you need to preserve evidence from websites or social media posts, online conversations or interactions, or record a sequence of steps or actions.</p><p>To create a screenshot, you can use built-in functions on your computer or device (such as pressing <em>CTRL + Print Screen</em><strong> or </strong><em>Windows logo key</em> + <em>Shift </em>+ <em>S</em> in Windows<strong>)</strong>, browser add-ons like <a href="https://chrome.google.com/webstore/detail/lightshot-screenshot-tool/mbniclmhobmnbdlbpiphghaielnnpgdp?hl=en">Lightshot</a> and <a href="https://gofullpage.com/">GoFullPage</a> (a great tool for taking a screenshot of the full length of a page), or standalone applications like <a href="https://getgreenshot.org/">Greenshot</a> or <a href="https://www.techsmith.com/store/snagit">Snagit</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HSZvNiHY8uJtltWjPJFG3A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p95gLejdWRPvRrcy_Wd1nQ.png" /><figcaption>GoFullPage browser addon captures the full length of a web page.</figcaption></figure><p>Screen recording is a related process that allows you to create a video of your screen. This is especially useful if you want to capture a process in action or show a certain functionality as it plays out. Much like screenshots, there are default options such as Windows <a href="https://www.tomshardware.com/how-to/screen-record-in-windows">Xbox Game Bar</a>, browser add-ons such as <a href="https://chrome.google.com/webstore/detail/awesome-screenshot-and-sc/nlipoenfbbikpbjkfpfillcgkoblgpmj">Awesome Screenshot and Screen Recorder</a> (which also works for screenshots), and applications like <a href="https://www.nchsoftware.com/capture/">Debut Screen Recorder Software</a>, <a href="https://screenrec.com/screen-recorder/">Free Screen Recorder</a>, and <a href="https://obsproject.com/">OBS Studio</a>.</p><h4>3. Archives</h4><p>Like creating a digital time capsule for web pages, archiving is the process of preserving a copy of a page via a third-party platform. This is extremely useful for keeping an impartial, publicly-available record of the page as it appeared at a specific point in time, particularly if it’s likely to change or be removed in the future.</p><p>It’s also an excellent way to share the archived page with others, without the need to send files or screen captures, by sending them a link to the page on the archiving site.</p><p>Two widely used archiving sites are <a href="https://archive.org/web/">The Wayback Machine</a> and <a href="https://archive.ph/">Archive.today</a>. Pasting the URL of any page into either of these platforms will allow you to create a current copy of the page for future reference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-YG85sxKxZGx0_vAOn8G7Q.png" /><figcaption>Archive.org capture of cnn.com website on August 17th 2000</figcaption></figure><p>An added bonus of this method is that since anyone can archive any page and all archives are public, there’s a chance you’ll find already-archived versions of the page you’re looking for.</p><h4>4. Site (HTML) Downloads</h4><p>Site or page downloading involves saving a web page or website as an offline HTML file or folder, almost like flash freezing the page. This can be useful for preserving a functional copy of the page on your own computer for future reference.</p><p>You can download a page using your browser’s built-in “Save as” function, browser add-ons like <a href="https://chrome.google.com/webstore/detail/save-page-we/dhhpefjklgkmgeafimnjhojgjamoafof">Save Page WE</a> or <a href="https://chrome.google.com/webstore/detail/save-as-mhtml/ahgakckdonjmnpnegjcamhagackmjpei?hl=en">Save as MHTML</a> (which saves a page and all its contents in a single HTML file), or standalone tools like <a href="https://www.gnu.org/software/wget/">Wget</a>, <a href="https://www.httrack.com/">HTTrack</a>, or <a href="https://www.cyotek.com/cyotek-webcopy">WebCopy</a>.</p><h4>5. Hashing</h4><p>This method is used to create a unique and irreversible fingerprint of digital data to make it easily identifiable and tamper-proof. Hashes are created with the help of algorithms that take the data as input and produce a specific string of characters and letters representing that data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*xG7E32zpKiMx3PYp.png" /><figcaption>Hashing a JPG image file (photograph by <a href="https://www.pexels.com/@sigrid-de-greef-1631075/">Sigrid De Greef</a>)</figcaption></figure><p>Because any changes to a hashed file (no matter how minor) will result in an entirely different hash, this method is like a digital wax seal for your data, if anyone modifies or replaces it, you’ll be able to tell.</p><p>You can hash a file using built-in command line applications on your operating system, such as Windows PowerShell for example, or you can choose one of the many web-based, browser-based, or GUI-based tools that are freely available.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*2qL5I3URN6hpIWKC.png" /><figcaption>Windows PowerShell file hashing</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*GMAgr8KeEN6QjiVj.png" /><figcaption>MD5file.com online file hashing</figcaption></figure><p><em>For a more in-depth discussion and other use cases of hashing, check out this article:</em></p><p><a href="https://medium.com/the-first-digit/hashing-the-secret-life-of-data-cab1ac245280">Hashing: The Secret Life of Data</a></p><h4>6. Note-taking</h4><p>This method involves creating a record of the entire investigative process and includes capturing all relevant information and connections as well as your observations and insights. In this sense, note-taking can be considered an overarching documentation technique that captures both your investigative process and your specific findings all in one place.</p><p>You can do note-taking using a variety of hierarchal text-based tools such as <a href="https://evernote.com/">Evernote</a>, <a href="https://www.giuspen.net/cherrytree/">CherryTree</a>, <a href="https://obsidian.md/">Obsidian</a>, or even <a href="https://www.microsoft.com/en-us/microsoft-365/onenote/digital-note-taking-app">OneNote</a>. Other tool options to try out include brainstorming and mind-mapping applications like <a href="https://xmind.app/">XMind</a>, <a href="https://www.mindomo.com/">Mindomo</a>, or <a href="https://www.mindmeister.com/">MindMeister</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*_iX_RP2SxghkwONHvqDy8A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*IO0t3wjy4z-vIlbR.jpg" /><figcaption>CherryTree (left) and XMind (right)</figcaption></figure><p>The key is to choose a tool that works best for your needs and allows you to easily organize and access your information. Note-taking is like creating digital sticky notes for your investigation; you can jot down all your ideas and observations in one place, and then reorganize or replace them as you go through the process.</p><h4>Bottom line</h4><p>Meticulous documentation is the foundation of any good OSINT investigation. By using a variety of methods and tools like raw data downloads, screen captures, third-party archiving, site or page downloading, data hashing, and note-taking, you can consistently apply a clear plan to document your findings, resulting in retraceable processes, reproducible results, available and accessible data, and a reliable chain of evidence.</p><p><em>For more OSINT-related stuff, follow us here on Medium or on </em><a href="https://twitter.com/TRADECRAFT14"><em>Twitter</em></a><em>, </em><a href="https://www.linkedin.com/company/ostradecraft"><em>LinkedIn</em></a><em>, and </em><a href="https://noc.social/@tradecraft"><em>Mastodon</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59883c8f5927" width="1" height="1" alt=""><hr><p><a href="https://osintteam.blog/the-six-pillars-of-osint-documentation-59883c8f5927">The 6 Pillars of OSINT Documentation</a> was originally published in <a href="https://osintteam.blog/">OSINT Team</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The 10 Commandments of Open Source Intelligence]]></title>
            <link>https://osintteam.blog/the-10-commandments-of-open-source-intelligence-8400c2776a90?source=rss-85cc242cc611------2</link>
            <guid isPermaLink="false">https://medium.com/p/8400c2776a90</guid>
            <category><![CDATA[open-source-intelligence]]></category>
            <category><![CDATA[osint]]></category>
            <category><![CDATA[public-information]]></category>
            <dc:creator><![CDATA[OSTC]]></dc:creator>
            <pubDate>Fri, 13 Jan 2023 10:30:29 GMT</pubDate>
            <atom:updated>2023-08-31T06:16:47.675Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cb8KOXXHVp_7_XOuF9yJzw.jpeg" /></figure><p>No doubt about it, OSINT’s moment has arrived. Propelled by an ever-increasing volume of public information and rapid geo-political developments, OSINT has finally reached a critical mass that has thrust it into the limelight. It seems everyone and their grandmother (if she’s computer savvy) is jumping on the bandwagon, and there’s no turning back.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*cJJluc73yohgSTBGZfMU1w.png" /><figcaption>Google web searches for the term OSINT. Jan, 2018 — Jan, 2023.</figcaption></figure><p>Whether you’ve been in the OSINT game for years or are a relative newcomer, it’s always a good idea to have a list of quick-reference guidelines to follow to ensure your process is consistent and your results are as accurate and reliable as possible. To that end and without further ado, I give you <em>The 10 Commandments of OSINT</em>:</p><h4>1. Verify all information</h4><p>It’s essential to always check the credibility of your data. This means verifying the information’s primary source, the date it was published, its textual and audio-visual content, and whether it has been corroborated by other sources.</p><h4>2. Do not rely on only a single source</h4><p>One of the most important principles of OSINT (and closely related to information verification) is to never rely on only a single source. Always cross-reference a source’s information with multiple other sources to ensure its accuracy.</p><h4>3. Utilize all media types</h4><p>Remember that OSINT is not only limited to online media sources. Therefore, it’s also important to take into account physical and offline resources. This could include broadcast media, newspapers, magazines, books, archives, and government records.</p><h4>4. Combine techniques and tools</h4><p>Just as no single source holds all the information, no single method will be enough to conduct a thorough investigation. OSINT involves using an amalgamation of techniques such as advanced search strings, data collection and data mining, link analysis, and network analysis, and a combination of tools.</p><h4>5. Respect privacy and laws</h4><p>OSINT must be done within the boundaries of the law and must respect the privacy of individuals. This means not exposing someone’s personal information without their consent (aka <a href="https://en.wikipedia.org/wiki/Doxing">Doxing</a>) or a legal reason to do so, following relevant data privacy laws, and not breaking the rules of the platforms, sources, and tools you’re using.</p><h4>6. Stay up-to-date with events</h4><p>OSINT analysts must stay up-to-date with current events and their area of focus to understand the context of the information they gather. This means keeping a constant watchful eye on news outlets, social media platforms, and any other relevant sources to know what’s trending and maintain your situational awareness.</p><h4>7. Document everything</h4><p>Meticulous documentation is essential in OSINT. It involves the use of several methods including data downloads, screen captures, site archiving or downloading, <a href="https://medium.com/the-first-digit/hashing-the-secret-life-of-data-cab1ac245280">data hashing</a>, and note-taking. When done right, documentation enables you to retrace your investigative steps, reproduce search results, and present reliable evidence.</p><p>To find out more about OSINT documentation, read our latest article:</p><p><a href="https://ostradecraft.medium.com/the-six-pillars-of-osint-documentation-59883c8f5927">The Six Pillars of OSINT Documentation</a></p><h4><strong>8. Be Aware of your digital footprint</strong></h4><p>Always remain mindful of your digital footprint when conducting research. This means being constantly aware of the information you share publicly (either intentionally or inadvertently), concealing your identity, and masking your IP address and activity.</p><h4><strong>9. Open your mind</strong></h4><p>As an OSINT analyst, you must keep an open mind and be willing to consider different perspectives and theories. This means not dismissing information based on preconceived notions or biases, and being willing to consider alternative viewpoints.</p><h4><strong>10. Continuously learn and improve</strong></h4><p>OSINT is a constantly-developing field, and new technologies and techniques are emerging all the time. And so practitioners must continuously expand their skills, such as <a href="https://medium.com/@ostradecraft/a-little-python-can-go-a-long-long-way-in-osint-bb2b3aab1d5b">learning to code</a> to automate numerous tasks, and educate themselves on the latest tools and techniques, advances in AI, and security and legislation developments to effectively apply them when needed.</p><h4>Go forth and prosper…</h4><p>By following these guidelines, you can conduct effective and ethical OSINT work. However, it’s important to keep in mind that these rules are not static, and as technology and the public domain continue to evolve, so too must these guidelines. After all, OSINT is not an objective but rather a methodology and mindset.</p><p><em>For more OSINT-related stuff, follow us here on Medium or on </em><a href="https://twitter.com/TRADECRAFT14"><em>Twitter</em></a><em>, </em><a href="https://www.linkedin.com/company/ostradecraft"><em>LinkedIn</em></a><em>, and </em><a href="https://noc.social/@tradecraft"><em>Mastodon</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8400c2776a90" width="1" height="1" alt=""><hr><p><a href="https://osintteam.blog/the-10-commandments-of-open-source-intelligence-8400c2776a90">The 10 Commandments of Open Source Intelligence</a> was originally published in <a href="https://osintteam.blog/">OSINT Team</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hashing: The Secret Life of Data]]></title>
            <link>https://osintteam.blog/hashing-the-secret-life-of-data-cab1ac245280?source=rss-85cc242cc611------2</link>
            <guid isPermaLink="false">https://medium.com/p/cab1ac245280</guid>
            <category><![CDATA[open-source-intelligence]]></category>
            <category><![CDATA[data-integrity]]></category>
            <category><![CDATA[verification]]></category>
            <category><![CDATA[osint]]></category>
            <category><![CDATA[data-management]]></category>
            <dc:creator><![CDATA[OSTC]]></dc:creator>
            <pubDate>Tue, 10 Jan 2023 13:22:23 GMT</pubDate>
            <atom:updated>2023-08-31T06:17:27.387Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*MNRASNj4hYNABdRG7gM3yA.jpeg" /></figure><p>You download an important document for a critical task but, for the life of you, can’t figure out if it’s the real deal or not. You’re sending sensitive data over the Internet and losing sleep worrying if it’ll get corrupted during file transfer. You’re working on a joint project yet you have a sneaking suspicion that someone may have messed with your data… The horror! The horror!</p><p>Actually, these kinds of things happen all the time. There’s even an entire field within computing dedicated to dealing with the intentional and unintentional corruption of data — <a href="https://en.wikipedia.org/wiki/Data_integrity">Data Integrity</a> (DI).</p><p>The good news? There’s one small DI trick of the trade that you can use to handle these (and other) sticky situations and put your mind at ease: Hashing!</p><h4>What’s “hashing”?</h4><p>Hashing is a technique that’s used to create a unique and irreversible fingerprint of digital data (a “hash”), making it distinctly identifiable and tamper-proof.</p><p>The concept is simple: A hashing algorithm takes the bytes that make up any digital file and creates a unique string of characters (a digital key) that represents those bytes. The resulting string will only ever match the current composition of bytes within that file at the moment it was hashed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*68Mf5o5A0JKV0Ot6aafurA.png" /><figcaption>Hashing a JPG image file (photograph by <a href="https://www.pexels.com/@sigrid-de-greef-1631075/">Sigrid De Greef</a>)</figcaption></figure><p>This means that changing even a single byte in the file, such as colorizing one pixel in an image (as in the example below), will completely alter its hash value, thus proving beyond a doubt that the old and new files are not one and the same. On the other hand, if no changes have been made to the original file, its hash will remain the same, thereby proving its authenticity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eHQ4B1PK0AntBO3Umo2PCQ.png" /><figcaption>Top: A JPG file and its hash. Bottom: The same file with a red pixel added, resulting in a different hash.</figcaption></figure><h4>What are its uses?</h4><p>Knowing how to create a hash and compare it to a known original enables you to verify the authenticity of files and to check whether your data has been modified or corrupted.</p><p>For Open Source Intelligence (OSINT) practitioners, hashing can be an extremely valuable addition to their workflow, both as a verification tool and a documentation method. For example, they can use it to check if an online photo or video is authentic or has been somehow modified, and to ensure that their findings and evidence have not been tampered with or otherwise distorted after sharing them with clients or other parties.</p><p>In this sense, hashing is like a digital wax seal on your data; if you find that the seal is broken, then the data has been compromised.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O9HF8x8ZjQN3v8ZDFjjTMw.png" /></figure><p>Yet another great use for this technique is the de-duplication of files and records. Let’s say you want to remove all duplicate images from your hard drive. By running each image on your system through a hashing algorithm and storing the resulting hashes in a list, you can easily check the list for duplicates without having to manually check every single image.</p><h4><strong>How is it done?</strong></h4><p>Hashing can be done manually by typing in a simple terminal command in most operating systems (with little or no technical background required) or by using one of the many web-based, browser-based, or GUI-based tools that are freely available.</p><p>Using the command line method in Windows is quite straightforward with the built-in PowerShell terminal. To launch PowerShell, click the Start or Search icon on your Windows desktop, type “powershell” in the search box, and click Open (or Run as Administrator).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*8J_5wNfryggzxFAb7FQF8w.png" /></figure><p>In the PowerShell terminal, type in the following command (replacing the FOLDER and FILE_NAME as needed):</p><pre>Get-FileHash FOLDER\FILE_NAME</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x_mOeS0CKyx4DzCvTZLJAw.png" /><figcaption>Windows PowerShell hash command and output.</figcaption></figure><p>In the MacOS terminal, use the command:</p><pre>shasum -a 256 FILE_NAME</pre><p>And in Linux terminal, use the command:</p><pre>sha256sum FILE_NAME</pre><p>Among the web-based tools are websites such as <a href="https://xorbin.com/tools/sha256-hash-calculator">xorbin.com</a> that will allow you to hash texts, and other sites such as <a href="https://emn178.github.io/online-tools/sha256_checksum.html">emn178.github.io</a> and <a href="https://md5file.com/calculator">md5file.com</a>, which will allow you to also drag and drop a file and hash it in multiple formats.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mUsKLEo9dTrmvlUr05iiHw.png" /><figcaption>MD5file.com</figcaption></figure><p>Alternatively, you can choose from among the different browser add-ons or extensions to do your hashing directly from within your browsing session. A couple of options in this category are <a href="https://chrome.google.com/webstore/detail/secure-hash-generator/eaoopiddkidbkpimlfkkmkkddjbpemop">Secure Hash Generator</a> (in Chrome and <a href="https://addons.mozilla.org/en-US/firefox/addon/secure-hash-generator/">Firefox</a>) to hash files, and <a href="https://chrome.google.com/webstore/detail/hash-generator/gpoanfafbmmhifpmbefgllakooeokdgl">Hash Generator</a> (in Chrome) to hash texts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-UfUSXv0gWO3_kZanoDN-w.png" /><figcaption><a href="https://chrome.google.com/webstore/detail/secure-hash-generator/eaoopiddkidbkpimlfkkmkkddjbpemop">Secure Hash Generator</a> Chrome add-on.</figcaption></figure><p>Finally, if for privacy or other reasons you&#39;d prefer not to upload your files to a website or browser extension, you can instead install a standalone GUI-based tool on your own system.</p><p>Some good options in this category are <a href="https://www.quickhash-gui.org/downloads/">QuickHash</a>, which works with Windows, Linux, and MacOS, and also allows you to compare the hashes of two files or even entire directories, and <a href="https://securityxploded.com/hashgenerator.php">Hash Generator</a> and <a href="http://www.abelhadigital.com/multihasher/#downloads">MultiHasher</a>, both of which are designed for Windows systems and offer hashing for numerous file types.</p><h4>Limitations</h4><p>Like pretty much everything in OSINT, here too the devil is in the details. And while there’s no denying that hashing can be extremely useful, it won’t always give you all the information you need.</p><p>For example, even though finding a mismatched hash tells you that your file has been modified, tampered with, or corrupted, it still doesn’t reveal specifically which element in the file was changed.</p><p>And because hashing algorithms look at all components of a file, including its data and metadata, you’ll most likely have to dive into the file itself to compare and contrast it with the original version to find the exact differences.</p><h4>Bottom line</h4><p>Whether you’re a no-nonsense OSINTer looking to take your verification, documentation, and evidence preservation skills to the next level, or just a person who doesn’t particularly like it when modified data is passed off as the genuine article, hashing is a simple yet effective technique to ensure the authenticity and integrity of your files and records. So hash at it!</p><p><em>For more OSINT-related stuff, follow us here on Medium or on </em><a href="https://twitter.com/TRADECRAFT14"><em>Twitter</em></a><em>, </em><a href="https://www.linkedin.com/company/ostradecraft"><em>LinkedIn</em></a><em>, and </em><a href="https://noc.social/@tradecraft"><em>Mastodon</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cab1ac245280" width="1" height="1" alt=""><hr><p><a href="https://osintteam.blog/hashing-the-secret-life-of-data-cab1ac245280">Hashing: The Secret Life of Data</a> was originally published in <a href="https://osintteam.blog/">OSINT Team</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Isn’t OSINT just glorified Googling?]]></title>
            <link>https://osintteam.blog/isnt-osint-just-glorified-googling-4f79e3f32d8e?source=rss-85cc242cc611------2</link>
            <guid isPermaLink="false">https://medium.com/p/4f79e3f32d8e</guid>
            <category><![CDATA[osint]]></category>
            <category><![CDATA[web-search-engine]]></category>
            <category><![CDATA[web-intelligence]]></category>
            <category><![CDATA[open-source-intelligence]]></category>
            <dc:creator><![CDATA[OSTC]]></dc:creator>
            <pubDate>Thu, 05 Jan 2023 13:02:59 GMT</pubDate>
            <atom:updated>2023-08-31T06:17:48.773Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lo1LX-edBlDA_7hu7v5ncg.png" /></figure><p>When I tell people what I do for a living, the usual reaction is: “Wait a minute, you mean you get paid to just Google stuff all day?!” Well, no, not quite.</p><p>Though it’s true that Open Source Intelligence (OSINT) does involve gathering information using the Internet, it’s much more than just Google (or Bing, or Yahoo, or Yandex) searches on steroids.</p><p>Not convinced yet? Fair enough; here’s my list of 8 key differences between OSINT<em>-ing</em> and Googling:</p><h4>1. Scope</h4><p>A search engine is a useful tool for finding information on the Internet and specifically on the Surface Web (the 4% of the Internet that’s actually visible and accessible to search engines), OSINT is a more systematic and comprehensive approach to gathering and analyzing information from the entire Internet and beyond.</p><p>Besides the Surface Web, an OSINT investigation typically involves <a href="https://en.wikipedia.org/wiki/Deep_web">Deep Web</a> sources, which aren’t accessible to Google (and friends) and therefore don’t appear in search results, as well as <a href="https://en.wikipedia.org/wiki/Dark_web">Dark Web</a> sources that can only be accessed using special browsers, such as <a href="https://en.wikipedia.org/wiki/Tor_(network)">TOR</a> for example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WMw84Emx9YuYzY3aXW4XKw.png" /></figure><p>To find the information they need, OSINT investigators may spend hours, days, or even weeks trawling through social media and content-sharing platforms, e-commerce and review sites, public records and databases, leaked information and breached data, crypto exchanges and marketplaces, FTP servers, Dark Web sources, hardcopy publications (books, newspapers, magazines, and even pamphlets/fliers on occasion), radio and television broadcasts, or various other public sources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kE35iJtjTll7V7klqmh9Dw.png" /></figure><h4>2. Uses</h4><p>Most people use search engines to quickly answer a simple question or find a specific piece of information such as events in the news, information about celebrities, sports results, recipes, and so on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*s5d87FsU1z4Z3hzx5y26aQ.png" /><figcaption><a href="https://trends.google.com/trends/yis/2022/GLOBAL/">Google Trends</a> — Global searches in 2022</figcaption></figure><p>OSINT is used in cases where a quick search isn’t enough because the questions it seeks to answer are more complex or because the information needed to answer them is more difficult to find.</p><p>For example, OSINT is frequently used to investigate issues like human and civil rights violations, environmental and other disasters, criminal activity including cybercrime, and even military actions and <a href="https://www.ostradecraft.com/osint-case-studies?pgid=kkrnyo8q-05b06f57-992a-4c9d-8062-85b4ab49c2d9">war crimes</a>, as has been made abundantly clear during the current war between Russia and Ukraine.</p><h4>3. Verification</h4><p>Any intelligence is only as good as the data it’s based upon. The massive amount of false information that’s generated daily on every topic imaginable, from celebrity gossip to critical health and safety advisories, means that OSINT must verify the accuracy and reliability of all its information before reaching any conclusions.</p><p>So while most people don’t normally question the truthfulness or accuracy of the search results they’re looking at, skepticism and a critical eye are second nature to OSINT investigators, who do everything possible to confirm the accuracy of any information they find by checking multiple sources and using various fact-checking techniques.</p><h4>4. Analysis</h4><p>Unlike search engines that tend to find individual pieces of information, OSINT’s primary goal is to gather all relevant information about a subject from any available sources and analyze the data to find connections, trends, and patterns that lead to useful intelligence. After all, that’s what puts the <em>INT</em> in OSINT.</p><p>For example, to find out whether a person is affiliated with a particular group or organization, you might seek specific news articles mentioning such an affiliation. An OSINT investigator, on the other hand, might decide to map out that person’s entire social network based on a detailed exploration of their online activity and their social media contacts and interactions to conclusively prove such ties exist and how deep they run.</p><h4>5. Methods</h4><p>OSINT also frequently involves using techniques such as platform-specific search strings; image and video analysis; metadata and source code exploration; network analysis; and link analysis, sometimes combined with specialized reconnaissance tools such as <a href="https://www.youtube.com/watch?v=n0XZ6lFYGWw">Maltego</a>, <a href="https://www.spiderfoot.net/documentation/">SpiderFoot</a>, and <a href="https://www.kali.org/tools/recon-ng/">Recon-ng</a>.</p><p>Search engines, on the other hand, rely on algorithms to generate lists of results that they deem most relevant to your search query.</p><p>For example, while an average searcher might use reverse image search to find where on the web a particular photograph appeared, an OSINT investigator could pick up on details in the photo’s foreground, background, or metadata that tell them a lot about where and when the photo was taken, and in some cases even <a href="https://www.youtube.com/watch?v=-G_NxUy5aLc&amp;t=0s">reveal the identities</a> of the people involved.</p><h4>6. Expertise</h4><p>To effectively do their work, OSINT practitioners usually need at least some knowledge of the different tools and techniques that are out there. And since the OSINT landscape is fluid and constantly evolving, most professionals in this field spend a considerable amount of time educating themselves on the latest changes and developments to stay at the top of their game.</p><p>Sometimes, investigators may have more advanced training in skills like data mining, imagery analysis, or programming or even have specific subject matter expertise in disciplines such as geopolitics, cybersecurity, or criminology.</p><p>Google searches, by comparison, don’t really require any special expertise and can be done by anyone with an Internet connection.</p><h4>7. Anonymity</h4><p>To avoid compromising their investigations or unintentionally revealing their identity, OSINT practitioners will usually take extra precautions to protect their anonymity and privacy, minimize their digital footprint, and prevent the subjects of their investigations from becoming aware of their presence and activity.</p><p>These precautions include the use of things like <a href="https://en.wikipedia.org/wiki/Virtual_private_network">VPNs</a> to mask or spoof IP addresses, <a href="https://en.wikipedia.org/wiki/Virtual_machine">Virtual Machines</a> to keep investigations separate and to protect the host computer against malware, and “sock puppets” — i.e. fake social media accounts — to avoid using real accounts.</p><p>Average web searchers don’t usually resort to such measures (even though using a VPN is highly recommended for everyone these days).</p><h4>8. Documentation</h4><p>For all but the simplest searches, an average search engine user can’t easily track their search steps to show how they went from point A to point B to point X, nor what the full results were at each step. They might be able to dig through their search engine or browser history, but it might still be quite difficult to accurately retrace their path.</p><p>In OSINT, documentation is an integral part of the process that enables investigators or their clients to access the original information later on (and usually see it just as it appeared when it was first discovered), to show how the investigative process unfolded, and to preserve the findings as evidence for use in legal or other proceedings.</p><p>To achieve such comprehensive and accurate documentation, an OSINT investigator will use methods and tools such as timestamped screenshots, webpage snapshots and archiving, and note-taking and mind-mapping applications.</p><h4>OSINT ≠ Googling</h4><p>The bottom line? Doing OSINT and “Googling it” are not equivalent activities. The first is a process, the second is an event.</p><p>And though OSINT may (and often does) use search engines, it takes searches many steps further and goes much deeper than a regular web search. It usually also involves many more sources, approaches every step methodically (including documenting its processes and findings), and requires a unique blend of expertise, skills, and tools.</p><p><em>For more OSINT-related stuff, follow us here on Medium or on </em><a href="https://twitter.com/TRADECRAFT14"><em>Twitter</em></a><em>, </em><a href="https://www.linkedin.com/company/ostradecraft"><em>LinkedIn</em></a><em>, and </em><a href="https://noc.social/@tradecraft"><em>Mastodon</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4f79e3f32d8e" width="1" height="1" alt=""><hr><p><a href="https://osintteam.blog/isnt-osint-just-glorified-googling-4f79e3f32d8e">Isn’t OSINT just glorified Googling?</a> was originally published in <a href="https://osintteam.blog/">OSINT Team</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A little Python can go a long, long way (in OSINT)]]></title>
            <link>https://ostradecraft.medium.com/a-little-python-can-go-a-long-long-way-in-osint-bb2b3aab1d5b?source=rss-85cc242cc611------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb2b3aab1d5b</guid>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[osint]]></category>
            <category><![CDATA[open-source-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[python-programming]]></category>
            <dc:creator><![CDATA[OSTC]]></dc:creator>
            <pubDate>Tue, 27 Dec 2022 12:03:51 GMT</pubDate>
            <atom:updated>2023-08-31T06:18:46.069Z</atom:updated>
            <content:encoded><![CDATA[<h3>In OSINT, a little Python can go a long way</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*7bDNnKuxXEQkMz7-jcPGOQ.jpeg" /></figure><p>A lot of different people get into Open Source Intelligence, or <a href="https://www.ostradecraft.com/post/the-amazing-power-of-public-information">OSINT</a>, for a lot of different reasons. Whether it’s because you’re an intrepid detective on a quest for justice, an investigative journalist looking to verify a key source, or merely an inquisitive [*<em>cough* </em>nosy] individual hoping to find interesting tidbits of information on people or groups [<em>*cough* </em>dig up some dirt], knowing how to program (even a little) can give you a serious advantage in the OSINT game.</p><p>Imagine being able to automatically gather huge amounts of data from almost anywhere on the open Internet, analyze vast datasets, map out complex networks, and even track people’s locations with just a few lines of code. That’s where Python comes in.</p><p>This versatile (yet surprisingly accessible/readable) programming language is a kind of Swiss Army Knife for OSINT practitioners, allowing you to automate all sorts of tasks and uncover information and patterns that might otherwise remain hidden.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OK9bs06tW7hArjpejI72GA.png" /><figcaption>Python tasks for OSINT [image by <a href="https://www.ostradecraft.com/">TRADECRAFT</a>]</figcaption></figure><p>So how can you use Python in your OSINT work? Let’s have a quick look.</p><h4><strong>Web Scraping</strong></h4><p>This is the process of extracting data from online sources using automation. Python has a number of pre-packaged modules, or libraries, that allow you to write code that can navigate to an online source, identify the data you’re interested in, and extract and structure it.</p><p>For example, if you’re interested in a particular Twitter user, you can use the <a href="https://www.tweepy.org/">Tweepy</a> library to gather all available information about them from the Twitter API, including the content of their tweets and any associated media files or links. And if you’re looking to collect data from a website, you could use the <a href="https://requests.readthedocs.io/en/latest/">Requests</a> and <a href="https://beautiful-soup-4.readthedocs.io/en/latest/">Beautiful Soup</a> libraries to extract information from the site and save the data to a file or database for later analysis.</p><p>Which brings me to my next point…</p><h4><strong>Data Mining and Analysis</strong></h4><p>Python has a number of libraries that can be used to mine, analyze, and visualize data. These libraries allow you to import and process large datasets and create charts, plots, and other visualizations to help you understand and communicate the data.</p><p>For example, let’s say you have a dataset containing information about a person’s social media activity, including the dates and times of their posts, the number of likes and comments they received, and the content of their posts. You could use libraries such as <a href="https://spacy.io/">spaCy</a>, <a href="https://radimrehurek.com/gensim/">Gensim</a>, and <a href="https://textblob.readthedocs.io/en/dev/">TextBlob</a> to mine and analyze the textual data to find patterns and to answer questions like: What are the most common topics the person writes about? Which hashtags do they use? and What are their feelings on a specific issue?</p><p>You could then use libraries like <a href="https://pandas.pydata.org/">Pandas</a> and <a href="https://matplotlib.org/">Matplotlib</a> to quantify, measure, and visually represent the data to answer questions such as: When is this person most active online? and Which posts gained them the most followers in the shortest amount of time?</p><h4><strong>Network Analysis</strong></h4><p>Libraries such as <a href="https://networkx.org/">NetworkX</a> and <a href="https://pygraphviz.github.io/">PyGraphviz</a> allow you to analyze the relationships between individuals or groups by creating visualizations of their connections and mathematically assessing the importance of a specific person or entity within the network.</p><p>For example, let’s say you want to understand the relationships between a group of people on social media. You could use Python and a network analysis library to create a graph representing the connections between these people, with each person represented as a node and their connections represented as edges. You could then use the library to calculate various metrics, such as degree centrality (the number of connections a person has) or betweenness centrality (the number of times a person acts as a bridge between other people). You could also use the library to identify groups or communities within the network or to identify key influencers or leaders.</p><h4><strong>Geolocation</strong></h4><p>Libraries such as <a href="https://github.com/DenisCarriere/geocoder">Geocoder</a> and <a href="https://geopy.readthedocs.io/en/stable/">Geopy</a> allow you to convert physical addresses or GPS coordinates into latitude and longitude, and vice versa. This can be useful for identifying the location of individuals or organizations based on their online activity.</p><p>For example, let’s say you want to understand the locations of a person’s social media activity. You could use a geolocation library to convert any location data you can find in their posts (e.g. geotags, place names, addresses, street signs, landmarks, etc.) into latitude and longitude coordinates. You could then use a mapping library, such as <a href="https://towardsdatascience.com/creating-a-simple-map-with-folium-and-python-4c083abfff94">Folium</a>, to create a map showing the locations of the person’s activity. This could help you understand where the person spends their time, and potentially even identify patterns in their movements.</p><p>These are just some of the ways that Python can support OSINT work, making it an excellent choice for gathering and analyzing information from publicly-available sources.</p><h4>But wait, there’s more!</h4><p>With its large community of developers, Python has an ever-expanding list of useful libraries and tools such as <a href="https://pypi.org/project/selenium/">Selenium</a> for<em> </em>automating web browsing tasks, <a href="https://pypi.org/project/python-whois/">Whois</a><em> </em>for querying web registrar databases to gather information about domain names, <a href="https://pypi.org/project/phonenumbers/">Phonenumbers<em> </em></a>for parsing and validating phone number data, and <a href="https://scikit-learn.org/">Scikit-Learn</a> for machine learning and predictive analytics.</p><p>So should you add <em>Python Programming </em>to your new year’s resolutions? Well, let me put it this way: If learning OSINT gives wings to your curiosity, then learning Python straps a warp drive on its backside. Nuff said?</p><p>For more OSINT and Python tools, tips, and topics, follow TRADECRAFT on <a href="https://twitter.com/TRADECRAFT14">Twitter</a>, <a href="https://www.linkedin.com/company/ostradecraft">LinkedIn</a>, and <a href="https://noc.social/@tradecraft">Mastodon</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb2b3aab1d5b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>