24 Million entries and I need to what?
S. Dale Morrey
sdalemorrey at gmail.com
Sat Dec 28 01:59:50 MST 2013
Yep the experiment itself is a bad idea.
Still if you had 25 million distinct entries each 32 bytes long and you
needed the fastest lookup and comparison time possible, what would you use?
FYI putting 24 million distinct entries into a single directory even /tmp
completely corrupted my file system to a point it wouldn't even boot. I'm
trying to figure out how that's possible. My understanding was that /tmp
isn't supposed to persist after a reboot, but everything was there anyways
(near as I could tell). Took an hour to rm -rf that directory from a
On Sat, Dec 28, 2013 at 1:27 AM, Levi Pearson <levipearson at gmail.com> wrote:
> On Fri, Dec 27, 2013 at 11:12 PM, Joshua Marsh <joshua at themarshians.com>
> > On Fri, Dec 27, 2013 at 7:54 PM, Levi Pearson <levipearson at gmail.com>
> >> Is helping someone further along the wrong path
> >> really helpful?
> > It sounded to me like he was experimenting, which is great. I don't feel
> > like that's a wrong path. The collision attacks I've studied all were
> > discovered through iterations of analysis and experimentation.
> Experimentation is essential, but part of that is figuring out the
> *right* experiment to do and knowing how to analyze the result.
> Misguided experiments are at best a waste of time, and at worst a
> source of false assurance about the subject under experiment.
> Furthermore, coming up with good experiments is *hard*, certainly
> harder than running this particular one, and so advice regarding the
> experiment itself can be correspondingly more helpful.
> Given your knowledge of cryptography and the 256-bit variant of SHA-2,
> can you think of any useful information that could be derived from the
> experiment in question? Do you think it's a useful experiment for
> determining the correctness of a SHA-2 implementation? You are almost
> certainly more knowledgeable about cryptography than I am, and I'll
> defer to your judgement if you would consider this particular
> experiment to have any value beyond the sort of "hard knocks" learning
> you get from wasting a lot of time on bad experiments. From what I can
> see, though, the experiment will provide *no* usable information
> unless a collision is actually discovered. That's vanishingly unlikely
> to happen if the implementation is correct, but is also highly
> unlikely for an implementation that is incorrect, but not blatantly
> > I'm fairly certain that I'm not the only one on this list with more than
> > hobbyists knowledge of cryptography. If he simply wanted to know about
> > correctness of an implementation of SHA-256 he probably could have asked
> > and gotten the answer. I'm not saying that anything you have said about
> > cryptography is wrong. I just find it ironic that you are calling
> > out for their inability to answer the question when it seems to me like
> > you've tried to answer a question he never had.
> I didn't call anyone out for inability to answer a question. I
> suggested that it was clear there was some fundamental
> misunderstanding about the usefulness of the experiment being set up,
> and that it's far more *helpful* to root out and correct the
> misunderstanding than it is to give correct answers on how to run a
> bad experiment more quickly. I do recognize that this is a debatable,
> but it ought to be clear now which side I stand on.
> If you read what I've written, I have nowhere provided any "answers"
> to unasked questions. I pointed out that the answers that others had
> given were not going to be *helpful* in any experiments devised to
> answer the *stated questions*, which turned out to be stated, if not
> inaccurately, at least vaguely. And in response the *actual question*
> was clarified, and now it's even more apparent that the experiment was
> poorly devised to investigate the question!
> We all have gaps in our reasoning at times and sometimes start off in
> bad directions, and it's very helpful to have someone who notices this
> point it out before too much effort is expended in the wrong
> direction. Not everyone listens to this sort of advice, but it's nice
> to have it there in case the questioner is prepared to take it and
> save some time and effort.
> So, you may think that my approach seems rude, but so far it seems to
> me that it's produced more information about the actual goal of this
> enterprise and has put everyone in a better position to offer sound
> advice. Of course, I admit it's possible that Dale has again left out
> some details that, if revealed, would show how the proposed experiment
> makes sense. Maybe he's also got some good reasons for being
> tight-lipped about the details. But based on the information given, I
> stand by my judgement that this is a poor experiment and much better
> one could be devised based simply on the nature of functions and
> algorithms instead of hard-to-quantify properties of this specific
> If anyone finds me asking a question that reveals (or seems to reveal,
> even) an underlying misunderstanding, I hope that they'll take the
> time to help me identify what's wrong with the question (or how I
> asked it) instead of just answering the poorly founded question.
> PLUG: http://plug.org, #utah on irc.freenode.net
> Unsubscribe: http://plug.org/mailman/options/plug
> Don't fear the penguin.
More information about the PLUG