// cyber security researcher
askkemp
I build tools for threat intelligence, malware detection, and internet-scale data collection — mostly in Python. Below is a selection of my open-source work.
View GitHub Profile →Projects
-
This prototype uses Google Gemini to automatically write YARA-X rules for brand monitoring. It crawls a list of URLs with a headless browser (Browsertrix), pulls the HTML out of the resulting WARC files, and asks Gemini to turn that content into four kinds of signatures: notable brand phrases, tracking/tag identifiers, ownership-verification meta tags, and unique HTML/JS fingerprints. Each generated rule is automatically validated by compiling it and re-scanning the original WARC to confirm it still matches. As a bonus step, it can ask Gemini for lists of related, same-domain, and unrelated URLs, crawl those too, and rescan everything to confirm the rule doesn't fire on unrelated sites — a quick way to sanity-check a signature's precision before using it at scale.
-
A set of Python scripts for scanning Web ARChive (WARC) files with YARA-X, built around the idea of scanning huge web crawls such as Common Crawl's monthly ~2.3 billion page dataset. Each record's HTTP payload is extracted and matched against YARA-X rules, and results include which record matched, the matching content, and the byte offset needed to pull the full record back out later. The repo walks through four usage scenarios, from generating your own small WARCs with a browser extension, to rapidly crawling a list of URLs, to downloading Common Crawl files locally for testing, up to a multiprocessed AWS EC2 workflow that can scan the entire ~80TB Common Crawl archive in well under a day. A companion script can then download just the matching payloads straight out of S3 without pulling down the full WARC files.
-
A single Python script that downloads both RFC 6962 "classic" and static-tile Certificate Transparency logs, then extracts useful metadata from each X.509 certificate such as fingerprint, issuer, subject, validity dates, and SAN entries. It tracks download progress and log tree size in DynamoDB, streams the downloaded records to S3, and shows a live progress bar with an ETA — even shutting down gracefully and flushing its logs to S3 if interrupted. Parsing primarily uses pyca/cryptography but falls back to PyOpenSSL and PKI linting for out-of-spec certificates that would otherwise fail. It's meant to be run as one long-lived process per CT log, and the resulting JSON logs can be loaded into ElasticSearch or a similar search engine for fast querying across millions of certificates.
-
A cloud-hosted website downloader that goes further than a plain crawl: it performs a MitM TLS-intercept while downloading a site so you get the full picture — files, PCAP, decrypted proxy logs, and the x509 certificates presented — not just the rendered pages. The system deploys to AWS with SAM and is built from a client script, a Lambda function behind API Gateway, an autoscaling EC2 group running SSLsplit and Wget, and an install script that wires up the certificates and iptables rules needed for interception. A job can target a specific AWS region or fan out to all of them at once, supports single-page or recursive downloads, lets you force IPv4/IPv6 and pick a user-agent, and returns everything as a single compressed archive via a pre-signed S3 URL. It's designed for parallel, disposable, short-lived jobs, with full Cloudwatch logging of every step for later review.
-
Shodan Search Results to MISP Jupyter Notebook
A prototype that takes the results of a Shodan search and organizes them into MISP as clean, de-duplicated threat intelligence objects rather than a pile of raw JSON. For every IP address found, it also queries the Shodan host history API and merges everything for that IP into a single MISP event, using custom and modified MISP objects (shodan-report, intel-collection, http-metadata, domain-ip, x509, geolocation, ftp-meta, and more) so the results are useful to an analyst and graph well inside MISP. It's careful about not creating duplicate objects — for example it ignores tiny lat/long drift in geolocation data that would otherwise spam near-identical objects — and uses its own identifier scheme since Shodan's own IDs aren't always present. The result is a repeatable way to turn ad-hoc Shodan searches into structured, searchable intelligence.
-
Downloads files and URL analysis results from the VirusTotal feeds API and writes them to disk, then produces small JSON "summary" files containing just the notable fields you'd actually want to search on. Logstash picks up those summaries and loads them into ElasticSearch, while Elastic APM tracks the script's throughput and errors along the way, so the whole pipeline is observable in Kibana. It avoids reprocessing work: files already on disk aren't re-downloaded, and summaries aren't regenerated if they're already indexed in ElasticSearch, with a back-population mode to catch up on anything missed. A one-time setup flag bootstraps the required ElasticSearch index template and lifecycle policy, making the whole feed-to-searchable-index pipeline simple to stand up.
-
Converts an entire folder of community Sigma detection rules into Splunk search syntax and stitches all of them (over 2,000 searches) into one big classic Splunk dashboard. The generated searches are designed to run one at a time, top to bottom, so a single dashboard load effectively walks through the whole Sigma ruleset against your data. Users can scope every search to a specific host via a dashboard input field, or leave it as a wildcard to run against all logs, and the dashboard is built and tested against the Sysmon and Windows Splunk add-ons. It's a fast way to get broad detection coverage from the open-source Sigma project into Splunk without converting and wiring up each rule by hand.
-
A scoring tool that combines several reputation sources — MaxMind GeoIP Insights and minFraud, FraudLabs Pro, GreyNoise, and Shodan — into a single allow/block disposition for an IP address. It's built around merchant-fraud data as the differentiator from typical IP reputation tools, and it reasons about direction and service type separately, so the same IP might be blocked as an inbound VPN connection while still being allowed for outbound web traffic. Each API's raw response, a curated subset, and a vote-style recommendation are all returned as JSON (or optionally flattened to CSV), and the repo includes worked examples like a Tor exit node, a malware-hosting IP, and a known spam sender to show how the different sources agree or disagree. The README also notes real-world observations, like fraud signals fading back to "normal" over time and Shodan's vulnerability data being passive rather than confirmed.
-
A proof-of-concept for looking up ARIN network WHOIS/RDAP records for a large list of IPv4 addresses in parallel by fanning requests out to an AWS Lambda function. Starting from ARIN's daily bulk delegation file, you extract the IPv4 blocks you care about, feed them into the script, and it queries the Lambda-backed lookup with a configurable thread count, logging both its own progress and the raw WHOIS results as JSON. Because it can generate a very large number of lookups quickly, the README is upfront that it's for educational/proof-of-concept use, and points people who need full ARIN data at ARIN's official bulk WHOIS access instead. It's a small, practical example of using serverless functions to parallelize an otherwise slow, rate-limited lookup.
-
A small script for pulling Rapid7 Forward DNS (FDNS) study files from the scans.io/Rapid7 Open Data API. It lists the files available for the study, skips anything already downloaded, and uses Wget (with a configurable rate limit) to fetch the rest, checking each file's SHA-1 hash against the API's metadata and flagging any mismatch by appending ".ERROR" to the filename. All activity is written to a log file, and the only requirements are a Rapid7 API key and Python, making it a straightforward, low-maintenance way to keep a local mirror of the FDNS dataset in sync.
-
A utility that scans every file inside a TAR archive with a YARA rule entirely in memory, without ever extracting the archive to disk first. When a file inside the archive matches the rule, the script can optionally extract just that matching file to a chosen output directory, which makes it handy for pulling a single interesting file out of a large archive without unpacking everything, and it can also print the matching strings for more visibility into why a file matched. It's a lightweight, single-purpose tool for triaging archived file collections against known YARA signatures.