The Gut-Ripping Question

Last month, someone was telling me about some setbacks in their faith. On the surface, it would seem that the God of the Old Testament and the God of the New Testament are not one and the same. Why is He so cruel in the Old Testament, yet so kind in the New?
But perhaps a better question would be, why is God so merciful in both the Old and New Testaments? How many times did He turn away from destroying us all? Even from the very beginning, He was merciful. The worst thing that He could have done - the thing we most deserved - was to be left in the mess we made. But He provided a way.
But ignoring particular acts of justice or mercy, the million dollar question came up. Why did God, in His foreknowledge that a large portion of His creatures would choose eternal suffering - why when He knew all this before time began, went ahead anyway with the plan? Is the joy of the few worth the condemnation of the many?

It's interesting that this question should spring from Genesis 3. The above question implies that God is insensitive to humans, or is cruel. Adam and Eve fell because they chose to believe that God didn't have their best interests in mind, which is the rub of this line of questioning.

It all makes me think of Job...
Why is life given to those in misery,
and life to the bitter soul,
to those who long for death that does not come,
who search for it more than for hidden treasure,
who are filled with gladness
and rejoice when they reach the grave?
-Job3:20-22

The Bible may not give an complete explanation of why God went ahead with creation. But He didn't explain Himself to Job either. Rather, He posed Job with a series of pointed questions.

Who is this that darkens my counsel with words without knowledge? Brace yourself like a man; I will question you, and you shall answer me. -Job38:2-3
Who endowed the heart with wisdom or gave understanding to the mind? -Job38:36

Do we have the capacity to grasp God's motives for creating us? If we do not understand the motives, how then can we assume they are inadequate or cruel? Can we say that our morality is superior to the Creator's?