[ UPDATE: Our NSRL lookup service is no longer operational ] Needle in a Stack of Needles
How much of your job as a lethal forensicator is spent searching for needles in haystacks? Known File Filters can be helpful in this regard . . . until the filters get too large to be useful. Reducing the data set you have to look through in order to find that precious needle (ouch) is one of those Holy Grail problems forensics people have been trying to address for years.
NIST on the Case
NIST is kind enough to distribute the National Software Reference Library (NSRL). A collection of hashes of known software (usually provided by vendors themselves), it contains over 78 million hashes, 21 million of which are unique. It is arguably the best resource almost no one uses because 78 million is enough hashes to choke almost any forensics tool you’re using. Go ahead and try it. We’ll wait . . .
One Step Beyond
Starting today we are offering (in Beta) the Kyrus NSRL Lookup Service (NSRLookup), which is based on the hard work of Rob Hansen at Red Jack, who coded nsrlquery. You can download a Windows binary or the *nix source code at SourceForge. Once installed, you can use the output of Jesse Kornblum’s md5deep to query the server. The output can be piped directly: C:\> md5deep -r * | nsrllookup -s nsrl.kyr.us . . . or a saved file can be used C:\> md5deep -r * > known.txt C:\> nsrllookup -s nsrl.kyr.us < known.txt By default the server responds with the hashes of unknown files. You can get the hashes of known files by adding the -k flag, like this: C:\> nsrllookup -s nsrl.kyr.us -k < known.txt To see a help screen and a list of all command line options, use the -h flag: C:\> nsrllookup -h nsrllookup for Windows version 1.0.6-1 nsrllookup [-hvukx] [-U FILE] [-K FILE] [-s SERVER] [-p PORT] -h: display this help message -v: display version information -u: show only unknown hashes (default) -k: show only known hashes -U FILE: write unknown hashes to FILE -K FILE: write known hashes to FILE -s SERVER: connect to a specified nsrlquery server -p PORT: connect on a specified port
Please report bugs in the Sourceforge discussion forum.
We hope that NSRLookup proves useful to the forensics community. It isn't quite a “find evidence” button, but with people like Rob trying to tackle the problem, its close.