I'll try to explain it in a different way: The question of endianness touches on the difference between definition vs. notation vs. technical representation
Starting with definition: the number formats we know are comprised of several digits. Each digit has a place. In decimal format, the least significant digit would be the single numbers 0-9, the digit on the next higher place would be the tens, then the hundreds and so on. In binary format, the least significant bit would be bit-0, which is 2^0, the bit on the next-higher place would be bit-1, which is 2^1
Because of the commutativity of addition the following holds:
1025 == 1*2^0 + 1*2^10 == 1*2^10 + 1*2^0
In a technical representation, each bit corresponds to some cell in an electrical circuit. For operations implemented in hardware, like e.g. addition, there is special wiring which (among others) will transport the carry of an addition to the next-higher cell. On this low, technical level, the actual meaning of endinanness is simple to understand: it defines how processing cells are wired to the memory bus: If we have a parallel bus of 64bit width, the question is: to which end of that bus is the least significant bit wired? Because, on the other side of that bus, there is a mapping to memory cells. In a little endian system, the wiring is such that the least significant bit will be connected to that line in the bus which maps to the lowest address in the memory.
However, most of the confusion arises from notation (more precisely, from a clash of notation systems). The point is, when humans do occidental mathematics, they always use some kind of notation, which is more similar to big endian. More specifically, the important point to note is the writing direction on paper. In all those cultures of the Occident, we are writing from left to right. Thus we conceptualise a digit on the left side as "first" and a digit on the right side as "later". And we have the habit of writing the most significant digits first. (Please note: in computer hardware, a cell or bus lane does not have the property of being "left" or "right" — it just depends on how you are looking at the circuity board)
Now comes the part which makes up most of the confusion. This stems from an completely ephemeral historical habit: At that time when computers first became more widely known and discussed in academia, it just happened that 8-bit registers were quite common. There is no deeper meaning in this, it is a random historic fact. But this led to the notion of a "Byte" to become something of a fad deeply entrenched in engineering culture. Unfortunately it was that very time, when also the foundational ideas of modern programming languages were formed and conceptualised. This causes the notion of a "Byte" (or a char in C parlance) to become deeply ingrained into the definition of our programming languages. And, as we all know, language shapes the way of our thinking.
Together, this shapes our habit of grouping the bits into groups of eight bits, and also the habit of addressing positions in the memory by multiples of eight bits. Now, combine this with the aforementioned habit of humans to notate the number representation from right to left. And voilà — total confusion results. We group the bits of a little-endian number in compounds of eight bits, but then we write them in our customary right-to-left notation:
1025 = 00000001 00000100 00000000 00000000
You see: here we notate each group of eight bits (a "byte") in descending order from right to left, but we arrange the groups in ascending order into this notation. Oh well. Admitted, this notation is well defined. Admitted, it is also a well established habit since 40 years or so. Yet to put it bluntly, this notational habit obscures and confuses the actual meaning.
It would be much more in line with the actual circuity and wiring to the bus, if we would notate the same little-endinan number as follows:
10000000 00100000 00000000 00000000
But it's probably too late to change such deeply ingrained habits, and so we'll have to live with that confusing "flip" in the direction within each Byte. And with the further confusion that the direction of Shift operations just happens to be defined in the mathematical notation with the most significant bit first, most left, and thus a "left shift" moves the bits up towards places with higher significance.