The Hungarian physicist Eugene Wigner once wrote a paper on "The unreasonable effectiveness of mathematics in the natural sciences" in Communications in Pure and Applied Mathematics, vol. 13, No. I (February 1960). In it he wrote that ``the enormous usefulness of mathematics in the natural sciences is something bordering on the mysterious and that there is no rational explanation for it''. I know there have been many times when I have been utterly incredulous that any of my theories or calculations could actually have any connection to reality. However, after learning more about biology and particularly neuroscience, I have come to see this as less of a conundrum.
One of the premises of Wigner's thesis is that mathematics is purely invented and hence cannot possibly have any connection to the outside world. To him, it is then uncanny that something as arcane as complex numbers or abstract algebra could have any applications to reality. I believe that mathematics is not a fanciful invention but comes comes directly from a brain that has been shaped by the natural world through evolution. For me, mathematics is a reflection of the mind and science is a mapping of that mind to the external world.
I started to think this way when I found out that mice and other supposedly lower animals know the basics of set theory and logic. For example, Howard Eichenbaum at Boston University showed that rats understand transitivity i.e. if A is bigger than B and B is bigger than C then A is bigger than C. Logic is not only a product of the human mind but is an innate skill that is built in and confers a benefit for survival. With basic logic operations hard wired, all that is needed to generate mathematics is the ability to combine logical statements. Perhaps, the ability to recursively combine arbitrarily complex propositions may be what distinguishes humans from other species. The ability for language may be inextricably linked to the ability for mathematics.
If mathematics is a product of a brain that was shaped by the external world then it is not surprising that we can describe that world with mathematics. To me, the unreasonable effectiveness of mathematics is a tautology. The real question is Why is the world regular enough to allow for mathematics to arise? That I have no answer. However, if a universe has enough consistency that being able to develop logic has a survival advantage, then mathematics will arise and be the natural language to describe it.