I've been working on an application that checks for duplicate files. One of the better ways to test whether files are identical is to hash them. MD5 hashing is common, but it has been known to cause collisions. I elected to use the SHA1 algorithm instead.
In my testing, I've found that I can has a ~1GB file within about 7 seconds without consuming an equal share of memory.
You will need the following namespaces:
SHA1 Hash Example Code:
When I started on this task, I thought it would a very simple thing to do. I mean its a similar concept for both; rows and columns. Well I spent two days banging my head on the desk before coming up with a solution.