A temporary lives, as the name suggests, only briefly. It is normally destroyed at the end of the expression of which it is a part, that is, at the next semicolon or, rarer, at the end of the controlling expression in if statements etc.
There is a single way to extend the temporary's life: By binding it to a const ref (a normal ref won't do the job.).
Therefore, the formal reason why you cannot bind temporaries to non-const refs is:
Because the reference would be immediately dangling.
Now you may ask "why do non-const refs not extend a temporary's life as well?" The design decision was probably made because it would be too error-prone. Even extending the lifetime through a const ref is error-prone: People may return that reference and use it to initialize another one which will be dangling since the original reference's scope was left. With a non-const ref, people might additionally think that writing through the reference would have an effect, for example, when it usually doesn't. Of course one could use the non-const ref like any local variable: Changes made through it will only be visible through it, and short of copying the referenced object there is no way to access it outside the ref's scope. But for that use case we have a true and tested zero-overhead solution which everybody understands immediately: Local variables ;-).
error: invalid initialization of non-const reference of type ‘double&’ from an rvalue of type ‘double’
.