Avoiding Webscraping Throttling Using Python and Tor as a Proxy | Bored Hacking
In order to hide your IP address and to try and limit throttling when web scraping, it might be a good idea to use a proxy such as Tor. This post describes how to use Python based web scraping tools and Tor to hide yourself when scraping websites for data. However, it can also be used more broadly when using Python to make requests to hide your actual IP. I do not condone the use of this information for any illegal or illicit activity.
How to Find Historical Imagery of Russia's Faked Satellite Photos - bellingcat
In Bellingcat’s latest report, Forensic Analysis of Satellite Images Released by the Russian Ministry of Defense, we exposed how Russia had falsely claimed satellite imagery from June 2014 was from July 14th and July 17th. As with nearly all of Bellingcat’s work we used open source information, satellite imagery from Google Earth, to expose the fake […]
This article was originally posted to the AutomatingOSINT.com blog. Recently I was listening to an episode of the Freakonomics podcast titled, “Making Sex Offenders Pay – And Pay and Pay” which you can listen tohere. The premise behind the episode was that sex offenders are a bit unique in how they are punished by the […]
Unveiling Hidden Connections with Google Analytics IDs - bellingcat
Imagine you are researching an anonymous website with no clues as to its authorship. A single string of code could link that site to another, which perhaps reveals the identity of the owner. With the right tools, this information can be surprisingly easy to uncover. Site fingerprinting with embedded code Google Analytics is a popular […]
This article was originally posted on the AutomatingOSINT.com blog. One of my Automating OSINT students Michael Rossi (@RossiMI01) pinged me with an interesting challenge. He had mentioned that the Common Crawl project is an excellent source of OSINT, as you can begin to explore any page snapshots they have stored for a target domain. Michael wanted […]
Automatically Discover Website Connections Through Tracking Codes - bellingcat
This article originally appeared on the AutomatingOSINT.com blog. Fellow Bellingcat contributor Lawrence Alexander did some really interesting OSINT work on analyzing the hidden links between websites using tracking and analytics codes. In his how-to titled “Unveiling Hidden Connections With Google Analytics IDs” he shows how you can begin to see how websites are connected to […]
Yachtspotting: OSINT Methods in Navalny's Corruption Investigation - bellingcat
Recently, Russian President Vladimir Putin’s press secretary, Dmitry Peskov, found himself embroiled in a scandal over his wristwatch, which was prominently displayed during his wedding to former Olympic ice dancer Tatiana Navka. Anti-corruption activist and Kremlin opponent Aleksey Navalny revealed that a watch worn by Peskov was worth upwards of $600,000, despite Peskov’s approximate annual […]
This article was originally published on the AutomatingOSINT.com blog. Lots of OSINT investigations involve looking at companies, their structure, and of course their directors. Just yesterday, CBC News here in Canada did a story about a wealthy family allegedly using tax havens as a means to avoid paying taxes. As part of the news story […]
Gangs of Detroit: OSINT and Indictment Documents - bellingcat
This article was originally published on the AutomatingOSINT.com blog. VICE News ran a story about a gang in Detroit, Michigan that was nabbed partly due to their use of social media. This of course caught my attention so I clicked the link to the indictment papers and began to have a read. I find court documents completely […]
This post originally appeared on the AutomatingOSINT.com blog. Eliot Higgins had an excellent post where he shows how to grab preview images from YouTube videos and use them for verification, or potentially to find additional sites where that video might be shown or linked to. This technique is called reverse image searching which I have covered […]
Wix is a popular free website hosting platform, that has 73 million users across 180 countries (stats from Wix themselves). Recently while working on an investigation, I was spending time hunting around a website that is hosted on Wix and I discovered an interesting anomaly.
Automatically Finding Weapons in Social Media Images Part 1 - bellingcat
This article was originally published on the AutomatingOSINT.com blog. As part of my previous post on gangs in Detroit, one thing had struck me: there are an awful lot of guns being waved around on social media. Shocker, I know. More importantly I began to wonder if there wasn’t a way to automatically identify when […]
Automatically Finding Weapons in Social Media Images Part 2 - bellingcat
This post was originally published on the AutomatingOSINT.com blog. In the first part of this series we examined how to write some code that can slice and dice and image and then submit it to Imagga. By examining the tags that come back we can make a fairly accurate guess as to whether there is […]
Automated Reverse Image Search Part 2: Vimeo - bellingcat
This article was originally published on the AutomatingOSINT.com blog. In a previous blog post I covered how to utilize the YouTube API to find the preview images for videos and then reverse search them using the TinEye API. In this blog post we will cover how to use the same techniques for Vimeo to retrieve […]
When People Sleep: Determine Facebook Activity Using Google Chrome, Javascript and Python - bellingcat
This article was originally published on the AutomatingOSINT.com blog. It is always an interesting question and one that was posed over on the IntelTechniques.com forums. How can you tell what hour of the day people are most active on Facebook? There are tools out there for Twitter and other platforms, but due to Facebook’s limited […]
Expanding Skype Forensics with OSINT and Python: IP Addresses - bellingcat
This article was originally published on AutomatingOSINT.com. In the first part of this series we covered how to extract email accounts from SQLite databases and pull additional information from FullContact.com to find social media accounts or other online profiles that are associated with those email addresses. In this post we will apply much the same technique except […]
Expanding Skype Forensics with OSINT: Email Accounts - bellingcat
This article was originally published on the AutomatingOSINT.com blog. I will be the first to tell you that I know little about forensics compared to most law enforcement or private forensic examiners. One thing that I always found amazing was looking at the result of a forensic acquisition and seeing all of that magical data flowing […]
Dark Web OSINT With Python and OnionScan: Part One - bellingcat
You may have heard of this awesome tool called OnionScan that is used to scan hidden services in the dark web looking for potential data leaks. Recently the project released some cool visualizations and a high level description of what their scanning results looked like. What they didn’t provide is how to actually go about […]
Dark Web OSINT with Python Part Two: SSH Keys and Shodan - bellingcat
This entry was originally posted on the AutomatingOSINT.com blog. Welcome back good Python soldiers. In Part One of this series we created a wrapper around OnionScan, a fantastic tool created by Sarah Jamie Lewis (@sarajamielewis). If you haven’t read Part One then go do so now. Now that you have a bunch of data (or you […]