This is why Reincarnation is a problem and not rational ! :

The human and animal population on earth grows exponentially(this is proven)
SO, how we could explain the introduction of extra souls?

If reincarnation is/was a fact then the population growth rate would be steady and flat! There would be a balance in the growth and death rate.

Now, for the reason why I think people would remember past lives:

Simply, DNA is crossed or passed from one generation to another. The genetic information would also contain memory or that memory genes are also passed to another generation or something similar would be passed on.

Or every time you touch someone those information from that person is passed on to you electro-magnetically. Since some from of electricity is involved in the behaviour of cells in the body. That electricity contains information which passed on by touching or genes.