Monday, July 28, 2008

The 64 Bit and premature optimaization anomaly !

I am currently working to improve my undergrad research on Neural Network (which has been accepted in Fifth International Symposium On Neural Networks ).We have done a lot of brain storming in last few months. And then I was selected to translate the whole idea to code ! Having the sole responsibility for developing the whole idea does involve twist and turns, but it is fun too ! After finishing developing , I have a feeling that it is gonna be a good work.

And last night, my research partner informed me that , result has improved for every case but for benchmark heart attack problem. For this data set the program falls into an infinite loop.

In constructing neural network, instead of maintaining a explicit relationship of which node connected to which one, I have packed the connection information in unsigned integer. So storing and retrieving the connection information is easy, just a few left or right shift and masking .

As a matter of fact, this optimization is not very necessary here, because

1) Storing and retrieving connection information do not occur that much.

2) As it is a research on how neural network can predict result more accurately , result is of more concern not the speed.

But now, as heart attack benchmark problem has 35 input nodes, way more than other benchmark problems, I am left shifting an unsigned integer by 35 bits !. Phew ! Guess what Donald. E. Knuth has told us

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." (Knuth, Donald. Structured Programming with go to Statements, ACM Journal Computing Surveys, Vol 6, No. 4, Dec. 1974. p.268.)

But as changing it would cause an avalanche of more changes, I did the obvious, changed the data type from unsigned integer to unsigned long long. And the result is same, trapped in the infinite loop.

So after more debugging for 2 hours, when I am cursing under my breath, the following line just hit me



Here, 1 is of type int having length 32 bits, so shifting it by 32 or more bits, nothing retains, the net result is a big fat 0 !. So what i need to do is making 1 as unsigned long long type ( 1ULL ).

And, YAY ! everything works !