Developer dev-lu has created a web application called the “OSINT Toolkit.” While I feel the name is a mismatch to its functionality, this is still a very useful little application. With a tiny modification in the docker-compose.yml, it can be stood up on a server and used to investigate potential threats. It’s also very light-weight. Unlike OpenCTI (which consumes about 10Gb of memory on my server), this takes less than 2Gb of memory. Certainly it doesn’t replace OpenCTI in totality, but it does share the most important features for my usage.
What It Is
This utility is a web application that runs on a server (or locally), using the “bring your own keys” model. Under the Settings tab, you configure all the API keys to 3rd parties like “AbuseIPDB” or “AlienVault.”
The only feature that doesn’t require a key is the Newsfeed. The Newsfeed is currently not configurable. It pulls security related news articles from a variety of sources for reading.
IOC Analyzer
The first tab (IOC analyzer) is where you input an indicator of compromise. This can be an IP address, domain, URL, email, hash, or CVE. Clicking the Analyze button will pass this value in the field to your 3rd party API plugins. So if you input a suspicious IP picked up by your IDS, and hit Analyze, it will pass it to AbuseIPDB.
Results will display from all API’s configured in a stacked list:
Quickly we get an idea of what is RED or GREEN and we can dig deeper by opening up a line item:
If you’re using the CrowdSec API you can get even more detail, like the type of attacks from this threat actor… in the case of this IOC:
Email Analyzer
Exporting an email to .eml format allows for the app to pass it to a 3rd party for analysis and display the results. It does a very deep dive into the email itself.
The first window shows General Info, such as the HASH of the email (which could be referred to the IOC lookup), as well as basic info (subject, to, from, etc.)
The second window shows security data:
The 3rd window shows info on any attachments present.
The 4th window parses out all URLs from the email – which can include advertising URLs, tracking URLs, Social Media URLs and more. Each of these URL’s can be further analyzed (although that button didn’t work for me):
The 5th window on the email parse shows how many hops it took the email to get to you and where those hops are.
The final window shows the full email parsed by fields.
Finally, they offer one more option: To Analyze via Open-AI.
IOC Extractor
I’m not quite sure how to use this tab.
Domain Monitoring
This tab allows you to check domains related to your own. The usefulness here is that the plugin hookup renders screenshots (if available) of each domain it finds similar to the one searched.
AI Assistant
If you feel comfortable enough to pass your log data (I don’t) to Open.AI, it will parse it and give you some result to this application. There’s no data storage of the result, so it’s a one and done. Other options on this tab are phishing analyzer, code helper and code deobfuscator. The last being perhaps the most interesting, as it can take IOC code and try to de obfuscate it using OpenAI.
While having OpenAI here is interesting, I don’t see the need to consume API points, when I could just do these actions in OpenAI directly.
What It Isn’t
I really don’t consider this an OSINT toolkit. OSINT would normally have information discovery tools on people, and corporations… The OSINT focus would be finding source images (reverse image search), EXIF data on images, Geo locating tools, phone number lookup, real estate data lookup, dark web data searches, public camera feeds by location, cell tower info, ADSB aircraft info, real-time ship info, Gmail GAIAID querying, and Public Info. Databases (i.e. LittleSis).
This tool doesn’t handle any of the above mentioned OSINT needs. Instead it focuses almost exclusively on a Cyberthreat model. I noticed the author of this tool is working on a Go based CTI utility, which may expand beyond the capabilities of this utility.
Production Readiness
As of this writing, there are warnings that this is not yet production ready. Compiling source, I got several notices of library vulnerabilities that need addressing. As such, I’m not sure I’d install it on a work VM. As a Proof of Concept at home, sure it’s great.
Key Takeaways
For me the most useful part of the tool is the IOC Analyzer. The CrowdSec and AbuseIPDB pairing go very well for a Cyber Threat Intelligence utility. However, the name just isn’t the right fit. There’s not much OSINT happening and more IOC focus.
I like the fact it’s not a heavy install. It isn’t dumping tons of Gigs of data on my filesystem, or consuming 10+ Gigs of ram. OpenCTI can tend to be heavy handed in that regard.