Understanding Big and Little Endian Byte Order

Problems with byte order are frustrating, and I want to spare you the grief I experienced. Here's the key:

Problem: Computers speak different languages, like people.
- Some write data "left-to-right" and others "right-to-left".
- A machine can read its own data just fine - problems happen when one computer stores data and a different type tries to read it.
Solutions
- Agree to a common format (i.e., all network traffic follows a single format), or
- Always include a header that describes the format of the data. If the header appears backwards, it means data was stored in the other format and needs to be converted.

Numbers vs. Data

The most important concept is to recognize the difference between a number and the data that represents it.

A number is an abstract concept, such as a count of something. You have ten fingers. The idea of "ten" doesn't change, no matter what representation you use: ten, 10, diez (Spanish), ju (Japanese), 1010 (binary), X (Roman numeral)... these representations all point to the same concept of "ten".

Contrast this with data. Data is a physical concept, a raw sequence of bits and bytes stored on a computer. Data has no inherent meaning and must be interpreted by whoever is reading it.

Data is like human writing, which is simply marks on paper. There is no inherent meaning in these marks. If we see a line and a circle (like this: |O) we may interpret it to mean "ten".

But we assumed the marks referred to a number. They could have been the letters "IO", a moon of Jupiter. Or perhaps the Greek goddess. Or maybe an abbreviation for Input/Output. Or someone's initials. Or 10 in base 2, aka 2 in binary. The list of possibilities goes on.

The point is that a single piece of data (|O) can be interpreted in many ways, and the meaning is unclear until someone clarifies the intent of the author.

Computers face the same problem. They store data, not abstract concepts, and do so using a sequence of 1's and 0's. Later, they read back the 1's and 0's and try to recreate the abstract concept from the raw data. Depending on the assumptions made, the 1's and 0's can mean very different things.

Why does this problem happen? Well, there's no rule that all computers must use the same language, just like there's no rule all humans need to. Each type of computer is internally consistent (it can read back its own data), but there are no guarantees about how another type of computer will interpret the data it created.

Basic concepts

Data (bits and bytes, or marks on paper) is meaningless; it must be interpreted to create an abstract concept, like a number.
Like humans, computers have different ways to store the same abstract concept. (i.e., we have many ways to say "ten": ten, 10, diez, etc.)

Storing Numbers as Data

Thankfully, most computers agree on a few basic data formats (this was not always the case). This gives us a common starting point which makes our lives a bit easier:

A bit has two values (on or off, 1 or 0)
A byte is a sequence of 8 bits
- The "leftmost" bit in a byte is the biggest. So, the binary sequence 00001001 is the decimal number 9. 00001001 = (2³ + 2⁰ = 8 + 1 = 9).
- Bits are numbered from right-to-left. Bit 0 is the rightmost and the smallest; bit 7 is leftmost and largest.

We can use these basic agreements as a building block to exchange data. If we store and read data one byte at a time, it will work on any computer. The concept of a byte is the same on all machines, and the idea of "Byte 0" is the same on all machines. Computers also agree on the order you sent them bytes -- they agree on which byte was sent first, second, third, etc. so "Byte 35" is the same on all machines.

So what's the problem -- computers agree on single bytes, right?

Well, this is fine for single-byte data, like ASCII text. However, a lot of data needs to be stored using multiple bytes, like integers or floating-point numbers. And there is no agreement on how these sequences should be stored.

Byte Example

Consider a sequence of 4 bytes, named W X Y and Z - I avoided naming them A B C D because they are hex digits, which would be confusing. So, each byte has a value and is made up of 8 bits.

 Byte Name:    W       X       Y       Z
 Location:     0       1       2       3
 Value (hex):  0x12    0x34    0x56    0x78

For example, W is an entire byte, 0x12 in hex or 00010010 in binary. If W were to be interpreted as a number, it would be "18" in decimal (by the way, there's nothing saying we have to interpret it as a number - it could be an ASCII character or something else entirely ).

With me so far? We have 4 bytes, W X Y and Z, each with a different value.

Understanding Pointers

Pointers are a key part of programming, especially the C programming language. A pointer is a number that references a memory location. It is up to us (the programmer) to interpret the data at that location.

In C, when you cast (convert) a pointer to certain type (such as a char * or int *), it tells the computer how to interpret the data at that location. For example, let's declare

void *p = 0; // p is a pointer to an unknown data type
             // p is a NULL pointer -- do not dereference
char *c;     // c is a pointer to a single byte

Note that we can't get the data from p because we don't know its type. p could be pointing at a single number, a letter, the start of a string, your horoscope, an image -- we just don't know how many bytes to read, or how to interpret what's there.

Now, suppose we write

c = (char *)p;

Ah -- now this statement tells the computer to point to the same place as p, and interpret the data as a single character (1 byte). In this case, c would point to memory location 0, or byte W. If we printed c, we'd get the value in W, which is hex 0x12 (remember that W is a whole byte).

This example does not depend on the type of computer we have -- again, all computers agree on what a single byte is (in the past this was not the case).

The example is helpful, even though it is the same on all computers -- if we have a pointer to a single byte (char *, a single byte), we can walk through memory, reading off a byte at a time. We can examine any memory location and the endian-ness of a computer won't matter -- every computer will give back the same information.

So, what's the problem?

Problems happen when computers try to read multiple bytes. Some data types contain multiple bytes, like long integers or floating-point numbers. A single byte has only 256 values, so can store 0 - 255.

Now problems start - when you read multi-byte data, where does the biggest byte appear?

Big endian machine: Stores data big-end first. When looking at multiple bytes, the first byte (lowest address) is the biggest.
Little endian machine: Stores data little-end first. When looking at multiple bytes, the first byte is smallest.

The naming makes sense, eh? Big-endian thinks the big-end is first. (By the way, the big-endian / little-endian naming comes from Gulliver's Travels, where the Lilliputans argue over whether to break eggs on the little-end or big-end. Sometimes computer debates are just as meaningful )

Again, endian-ness does not matter if you have a single byte. If you have one byte, it's the only data you read so there's only one way to interpret it (again, because computers agree on what a byte is).

Now suppose we have our 4 bytes (W X Y Z) stored the same way on a big-and little-endian machine. That is, memory location 0 is W on both machines, memory location 1 is X, etc.

We can create this arrangement by remembering that bytes are machine-independent. We can walk memory, one byte at a time, and set the values we need. This will work on any machine:

c = 0;     // point to location 0 (won't work on a real machine!)
*c = 0x12; // Set W's value
c = 1;     // point to location 1
*c = 0x34; // Set X's value
...        // repeat for Y and Z; details left to reader

This code will work on any machine, and we have both set up with bytes W, X, Y and Z in locations 0, 1, 2 and 3.

Interpreting Data

Now let's do an example with multi-byte data (finally!). Quick review: a "short int" is a 2-byte (16-bit) number, which can range from 0 - 65535 (if unsigned). Let's use it in an example:

short *s; // pointer to a short int (2 bytes)
s = 0;    // point to location 0; *s is the value

So, s is a pointer to a short, and is now looking at byte location 0 (which has W). What happens when we read the value at s?

Big endian machine: I think a short is two bytes, so I'll read them off: location s is address 0 (W, or 0x12) and location s + 1 is address 1 (X, or 0x34). Since the first byte is biggest (I'm big-endian!), the number must be 256 * byte 0 + byte 1, or 256*W + X, or 0x1234. I multiplied the first byte by 256 (2^8) because I needed to shift it over 8 bits.

Little endian machine: I don't know what Mr. Big Endian is smoking. Yeah, I agree a short is 2 bytes, and I'll read them off just like him: location s is 0x12, and location s + 1 is 0x34. But in my world, the first byte is the littlest! The value of the short is byte 0 + 256 * byte 1, or 256*X + W, or 0x3412.

Keep in mind that both machines start from location s and read memory going upwards. There is no confusion about what location 0 and location 1 mean. There is no confusion that a short is 2 bytes.

But do you see the problem? The big-endian machine thinks s = 0x1234 and the little-endian machine thinks s = 0x3412. The same exact data gives two different numbers. Probably not a good thing.

Yet another example

Let's do another example with 4-byte integer for "fun":

int *i; // pointer to an int (4 bytes on 32-bit machine)
i = 0;  // points to location zero, so *i is the value there

Again we ask: what is the value at i?

Big endian machine: An int is 4 bytes, and the first is the largest. I read 4 bytes (W X Y Z) and W is the largest. The number is 0x12345678.
Little endian machine: Sure, an int is 4 bytes, but the first is smallest. I also read W X Y Z, but W belongs way in the back -- it's the littlest. The number is 0x78563412.

Same data, different results - not a good thing. Here's an interactive example using the numbers above, feel free to plug in your own:

The NUXI problem

Issues with byte order are sometimes called the NUXI problem: UNIX stored on a big-endian machine can show up as NUXI on a little-endian one.

Suppose we want to store 4 bytes (U, N, I and X) as two shorts: UN and IX. Each letter is a entire byte, like our WXYZ example above. To store the two shorts we would write:

short *s; // pointer to set shorts
s = 0;    // point to location 0
*s = UN;  // store first short: U * 256 + N (fictional code)
s = 2;    // point to next location
*s = IX;  // store second short: I * 256 + X

This code is not specific to a machine. If we store "UN" on a machine and ask to read it back, it had better be "UN"! I don't care about endian issues, if we store a value on one machine, we need to get the same value back.

However, if we look at memory one byte at a time (using our char * trick), the order could vary. On a big endian machine we see

Byte:     U N I X
Location: 0 1 2 3

Which make sense. U is the biggest byte in "UN" and is stored first. The same goes for IX: I is the biggest, and stored first.

On a little-endian machine we would see:

Byte:     N U X I
Location: 0 1 2 3

And this makes sense also. "N" is the littlest byte in "UN" and is stored first. Again, even though the bytes are stored "backwards" in memory, the little-endian machine knows it is little endian, and interprets them correctly when reading the values back. Also, note that we can specify hex numbers such as x = 0x1234 on any machine. Even a little-endian machine knows what you mean when you write 0x1234, and won't force you to swap the values yourself (you specify the hex number to write, and it figures out the details and swaps the bytes in memory, under the covers. Tricky.).

This scenario is called the "NUXI" problem because byte sequence UNIX is interpreted as NUXI on the other type of machine. Again, this is only a problem if you exchange data -- each machine is internally consistent.

Exchanging Data Between Endian Machines

Computers are connected - gone are the days when a machine only had to worry about reading its own data. Big and little-endian machines need to talk and get along. How do they do this?

Solution 1: Use a common format

The easiest approach is to agree to a common format for sending data over the network. The standard network order is actually big-endian, but some people get uppity that little-endian didn't win... we'll just call it "network order".

To convert data to network order, machines call a function hton (host-to-network). On a big-endian machine this won't actually do anything, but we won't talk about that here (the little-endians might get mad).

But it is important to use hton before sending data, even if you are big-endian. Your program may be so popular it is compiled on different machines, and you want your code to be portable (don't you?).

Similarly, there is a function ntoh (network to host) used to read data off the network. You need this to make sure you are correctly interpreting the network data into the host's format. You need to know the type of data you are receiving to decode it properly, and the conversion functions are:

 htons()--"Host to Network Short" 
 htonl()--"Host to Network Long" 
 ntohs()--"Network to Host Short" 
 ntohl()--"Network to Host Long"

Remember that a single byte is a single byte, and order does not matter.

These functions are critical when doing low-level networking, such as verifying the checksums in IP packets. If you don't understand endian issues correctly your life will be painful - take my word on this one. Use the translation functions, and know why they are needed.

Solution 2: Use a Byte Order Mark (BOM)

The other approach is to include a magic number, such as 0xFEFF, before every piece of data. If you read the magic number and it is 0xFEFF, it means the data is in the same format as your machine, and all is well.

If you read the magic number and it is 0xFFFE (it is backwards), it means the data was written in a format different from your own. You'll have to translate it.

A few points to note. First, the number isn't really magic, but programmers often use the term to describe the choice of an arbitrary number (the BOM could have been any sequence of different bytes). It's called a byte-order mark because it indicates the byte order the data was stored in.

Second, the BOM adds overhead to all data that is transmitted. Even if you are only sending 2 bytes of data, you need to include a 2-byte BOM. Ouch!

Unicode uses a BOM when storing multi-byte data (some Unicode character encodings can have 2, 3 or even 4-bytes per character). XML avoids this mess by storing data in UTF-8 by default, which stores Unicode information one byte at a time. And why is this cool?

(Repeated for the 56th time) "Because endian issues don't matter for single bytes".

Right you are.

Again, other problems can arise with BOM. What if you forget to include the BOM? Do you assume the data was sent in the same format as your own? Do you read the data and see if it looks "backwards" (whatever that means) and try to translate it? What if regular data includes the BOM by coincidence? These situations are not fun.

Why are there endian issues at all? Can't we just get along?

Ah, what a philosophical question.

Each byte-order system has its advantages. Little-endian machines let you read the lowest-byte first, without reading the others. You can check whether a number is odd or even (last bit is 0) very easily, which is cool if you're into that kind of thing. Big-endian systems store data in memory the same way we humans think about data (left-to-right), which makes low-level debugging easier.

But why didn't everyone just agree to one system? Why do certain computers have to try and be different?

Let me answer a question with a question: Why doesn't everyone speak the same language? Why are some languages written left-to-right, and others right-to-left?

Sometimes communication systems develop independently, and later need to interact.

Epilogue: Parting Thoughts

Endian issues are an example of the general encoding problem - data needs to represent an abstract concept, and later the concept needs to be created from the data. This topic deserves its own article (or series), but you should have a better understanding of endian issues. More information:

Interactive example:

108 Comments

Posted September 19, 2006, under Programming
Tags:
Related Posts:

Unicode and You

A little diddy about binary file formats

Explanations for everyone.

The Quick Guide to GUIDs

A Simple Introduction To Computer Networking

Like it? Share on:

RSS or Email Subscription |

Print This Post

108 Comments »

Trackbacks & Pingbacks

Pingback by Unicode and You | BetterExplained — February 21, 2007 @ 6:04 am
Pingback by A little diddy about binary file formats | BetterExplained — February 21, 2007 @ 6:10 am
Pingback by network « life ideas — November 13, 2007 @ 4:07 pm
Pingback by Twitted by zjtopspeed — August 4, 2009 @ 2:20 am

Comments

Great post! Thank you!

Michael P — January 12, 2007 @ 8:41 am
how do you write 999 in little endian

angela — January 25, 2007 @ 10:21 am
Hi Angela, it depends on whether you are using a 2-byte or 4-byte integer. I’ll assume you are using a 4-byte integer because they are more common.

In hex, 999 is 0×03e7. But we need to pad it out to 4 bytes, so it becomes

0×00 00 03 e7

(Broken into 1-byte groups for easier reading). On a little-endian machine, the little end (e7) comes first, so it would be stored as

e7 03 00 00

To double-check, if you plug these numbers into the endian calculator it should give 999 for little-endian.

Hope this helps!

Kalid — January 26, 2007 @ 8:12 pm
This is really really helpful! Big Thanks!

Vivi — April 16, 2007 @ 10:39 pm
In Byte Example you say that 00100010 is 0×12 or dec. 18. Is that true?

jakob — May 15, 2007 @ 1:05 am
Whoops, that was a typo — thanks! It should be:

0×12 = binary 0001 0010 = decimal 18

Putting the space helps make it more clear, I’ll make the change now. Appreciate the comment.

Kalid — May 15, 2007 @ 1:38 am
thanks a million..a superb explanation about byte ordering…
mr. jakob: there is no typo… its right in doc. as 00010010(0×12) and 18(dec)..

shekhar — May 24, 2007 @ 11:42 pm
Hi Shekhar, I’m glad you found it useful. Jakob actually caught the typo, I fixed up the original article

Kalid — May 25, 2007 @ 1:02 am
Thank you boss… just when i thought that Endianess is a complete waste in the world of Computer arithmatics!!!.. you proved it works…

Sumeash — July 19, 2007 @ 5:15 am
Hi Sumeash, always happy to help!

Kalid — July 31, 2007 @ 1:05 am
i want to know about the coding … how can we change the number present in big endian to little endian and vice-versa

Navin — September 12, 2007 @ 7:21 am
How will following structure look in big-endian and little-endian systems?
struct STRUCT {
unsigned int a:12;
unsigned int b:10;
unsigned int c:10;
};
struct STRUCT myStruct= {0xFED, 0×345, 0×3AB};

emile — October 4, 2007 @ 1:01 am
Was so very educative. Now I think i can go easy with my programming.

Giridhar — November 29, 2007 @ 2:09 pm
@emile: I’ll have to take a closer look at that one and remember how padded items are laid out .

@Giridhar: Thanks, glad you liked it.

Kalid — November 29, 2007 @ 3:28 pm
Awesome post!
thank you

yogesh — December 5, 2007 @ 10:03 pm
Thanks Yogesh!

Kalid — December 5, 2007 @ 10:11 pm
Question from Doran: “How can 2 chars allocate to an unsigned short? It just doesn’t make sense to me.
I’ve heard about NUXI few times before, and still I can’t get it. Please can you explain it for me. (even in C)

My reply:

On a 32-bit computer, a short is composed of 16 bits (2 bytes). In order to set the value, you an specify the short using two hex characters (in C):

short a = 0×12;
short b = 0×34;

So short a has 0×12 (18 decimal) and b is similar. Now, instead of using the characters “0-A”, let’s just use U N I and X. For example, U could be 3, N could be 4, I could be 5, and X could be 6.

short a = 0xUN;
short b = 0xIX;

On any machine, these shorts would be stored consecutively in memory. Address 0 and 1 would be “a”, and address 2 and 3 would be “b”. [again, each short takes up 2 bytes].

On a big-endian machine, the data would look like this:

Addr 0: U
Addr 1: N
Addr 2: I
Addr 3: X

On a little-endian machine, we store the *smallest* part of the number first. That is, in a = 0xUN, we store “N” first, which are the low-order bits. So in memory it would look like this:

Addr 0: N
Addr 1: U
Addr 2: X
Addr 3: I

Hence the “NUXI” problem. On a big-endian machine the data looks like UNIX, on a little-endian machine the data looks like NUXI. This isn’t a problem if you stay on the same machine (each machine knows how to convert appropriately), but can be a problem if you are exchanging binary data between machines.

Hope this helps,

-Kalid

Kalid — December 10, 2007 @ 6:58 pm
Well information is good!!!
I have one query if there is not much advantage of Big-Endian over Little Endian then why Network Byte order is Big-Endian????

SHAAN — December 17, 2007 @ 12:17 am
While there are advantages to each, I don’t think one is clearly better than the other. I think they just had choose one or the other — Big Endian may have been a more popular format at the time .

Kalid — December 17, 2007 @ 12:19 am
Thank you so much. I’ve been lucky thus far doing high end coding, but having rolled my sleeves up to start mucking about with bit and bytes this has been very helpful indeed.

kudos!

Steve — December 20, 2007 @ 2:53 pm
Thanks Steve, glad you found it useful! It’s fun to dip into bits & bytes every once in a while .

Kalid — December 20, 2007 @ 4:22 pm
That was damn good explanation. Thanks a lot for the post

Ramkumar — December 28, 2007 @ 12:20 am
Hi Ramkumar, you’re welcome. Glad you liked it.

Kalid — December 28, 2007 @ 1:12 pm
this is a great read

socal — December 29, 2007 @ 6:12 pm
Thanks Socal!

Kalid — December 30, 2007 @ 1:52 am
How can I change byte order from Big Endian to Little Endian and vice versa without breaking structure. When we are sending any structure.

avvy — January 8, 2008 @ 3:21 am
Hi avvy, you can use the “host to network” and “network to host” functions to convert data (more info here: http://linux.die.net/man/3/htons). You’d have to convert each field in the structure separately.

Kalid — January 8, 2008 @ 2:38 pm
Hi Kalid,

I cannot get a better picture of this topic wherever i search. I was searching about endianness in a hurry as was looking for some stuff which can atleast give the details in short and i feel lucky to find your post. Your post came like an angel as i had some urgency to find about endianness faster.
Very briefly you told whole story about it. Details perfect…Flow of explanation perfect! Kudos!
I’ll appreciate if you drop a small mail whenver you post stuff like this.

Thanks a tonn!

Shweta — January 17, 2008 @ 7:41 am
Hi Shweta, glad you liked the article! If you’d like to receive emails when new posts appear, just enter your email address in the “subscribe” form on the upper-right of the page. Thanks for the comment.

Kalid — January 17, 2008 @ 10:23 am
Hey thanks for info..done it
Hoping to see something informative soon.

Shweta — January 18, 2008 @ 4:06 am
You have a ‘locaiton’ in there, if you care to fix it.
I second everyone else and say that this is awesome. I didn’t even know this issue existed and now I understand it well (I think).

I think you should mention explicitly that if you store ‘UNIX’ in little-endian it will end up as NUXI in big-endian. Not strictly necessary, but I think it would Explain it Better. (Even Better.) Or perhaps lainExp it terBet. (enEv terBet.)

Whenever I see the word endian I think first of Ender Wiggin. The enemy is down and so forth.

Alrenous — March 14, 2008 @ 7:54 pm
Thanks Alrenous, glad you’re enjoying it. Appreciate the tip — went through and cleaned up a bunch of typos (it’s a bit embarrassing how many were in there).

Good suggestion on the explanation, I’m always looking for ways to make things clearer (that’s why this isn’t best explained ).

I hadn’t made the Ender Wiggin association, but I love Ender’s game… maybe there’s a way to fit him and Bean into this article somewhere.

Kalid — March 14, 2008 @ 8:10 pm
Gr8 post. really helpful.

Breeson — April 6, 2008 @ 10:48 pm
Thanks Breeson, glad it was useful.

Kalid — April 28, 2008 @ 10:24 pm
Thanks alot Kalid ur explanation about endianness is awesome. I hve one question that need to be answered Is there is a way In a mixed binary file with 4 byte Integers and single byte characters to identify whether the byte we read from the file is a character or is a part of 4 byte integer data.

It would be very helpful if you can answer me

Kumar — April 30, 2008 @ 12:08 pm
Hi Kumar, glad you enjoyed it. Offhand, I don’t think there’s a way, looking at the raw data, to tell whether it’s supposed to be a character or integer.

I think you’d need the file format spec to figure out the structure of the data — for example, the TCP header defines the byte ranges for each field.

Kalid — April 30, 2008 @ 12:20 pm
hi,
actually i know nothing about the c-language, but i have a question.
i have a riddle. it is: 71+, OÄE
i have to get numbers of this riddle, that is, i have to “translate” this riddle somehow into numbers.
and i know, that it has something to do with the c or c++ language and also something with intel byte order…
can you please help me? i would be most thankful.

joonas — May 4, 2008 @ 2:14 am
Hi Joonas, I’m not exactly sure of the question — you have raw data and need to extract a value?

If you have 4 bytes (for example), I would examine it as a 4-byte integer, 4-byte unsigned integer, 2 shorts, and 4 characters (all in the big and little endian varieties). I’d then try to look for some pattern in the results.

Kalid — May 4, 2008 @ 11:00 am
Very nice article! crisp and to the point……

hema — May 19, 2008 @ 6:27 am
Thanks Hema, glad you liked it.

Kalid — May 19, 2008 @ 9:52 am
Ahh… Finally i got it.
What an explanation!
Concepts, Simplicity, Analogies, Humor and Writing style – first Question then Answer.
Really, Thankx A Lot.

Apurva — May 30, 2008 @ 10:25 am
Awesome Apurva, glad you enjoyed it!

Kalid — June 1, 2008 @ 12:47 pm
Do you have any idea what machine uses which endian? For example, how about PC, big endian or little endian? Unix, big endian or little endia?

Shaoji — June 11, 2008 @ 11:49 am
excellent job. the way you explained it is the way things should be explained in this world.

barn — June 12, 2008 @ 1:19 pm
Simple and interesting presentation

Basheer — June 18, 2008 @ 5:54 am
Thanks for an excellent explanation. Just one question: you only mention integers – what about floating point values? How are they converted between endianness?

Paul Dowd — June 19, 2008 @ 11:39 am
@Shaoji: Good question, there is a partial list here: http://en.wikipedia.org/wiki/Endianness#Endianness_and_hardware. The endian issue is more with the processor (x86 Pentium) than the OS (Unix).

@Barn: Thanks, glad you liked it.

@Basheer: Thanks!

@Paul: Appreciate the kind words. Great question — I believe you would treat a floating point number as a 4-byte data type (on a 32 bit machine). Between different endian machines you’d have to reverse the bytes before reading the float (not positive but that’s my guess — data is data).

Kalid — June 20, 2008 @ 10:11 am
Here’s sample code for floating point / byte array conversion. We’ll start with a number (3.14159) and flip the sign bit (more info on floating point).
```
#include <stdio.h>

int main(int argc, char** argv){
  char *a;
  float f = 3.14159;  // number to start with
 
  a = (char *)&f;   // point a to f's location

  // print float & byte array as hex
  printf("float: %f\n", f);
  printf("byte array: %hhX:%hhX:%hhX:%hhX\n", \
    a[0], a[1], a[2], a[3]); 

  // toggle the sign of f -- using the byte array
  a[3] = ((unsigned int)a[3]) ^ 128;

  //print the numbers again
  printf("float: %f\n", f);
  printf("byte array: %hhX:%hhX:%hhX:%hhX\n", \
    a[0], a[1], a[2], a[3]); 

  return 0;
}
```
And it outputs this on my machine:
```
float: 3.141590
byte array: D0:F:49:40
float: -3.141590
byte array: D0:F:49:C0
```
As you can see, we can access the floating point number as a byte array and change individual portions of it. This machine is little-endian. Notice how a[3], the highest byte containing the sign bit, is last. That means the little end, a[0], comes first.

Theoretically, on a big-endian machine the order of bytes would be reversed, but some sources say IEEE 754 floats are always stored little-endian. I don’t have a big-endian machine to confirm this .

Kalid — July 13, 2008 @ 12:29 am
Keep posting…Very good article.

Ajay — July 21, 2008 @ 10:03 am
How will following structure look in big-endian and little-endian systems?
struct STRUCT {
unsigned int a:12;
unsigned int b:10;
unsigned int c:10;
};
struct STRUCT myStruct= {0xFED, 0×345, 0×3AB};

anil — August 13, 2008 @ 3:16 am
Why are sequence of characters not converted into network byte order when sending them to a remote UNIX server?
Example : – ‘Article’ is sent without converting to ‘elcitrA’.

Sharada Chanda — August 22, 2008 @ 9:28 pm
Interesting Presentation!!
Just one question.. Whether the byte order conversion for a 8-byte variable is similar to a 4-byte variable?

Ajish — September 7, 2008 @ 10:58 pm
Excellent Boss!!! thnaks a billon…for presenting such a complex thing in a very lucid manner.

pramod — September 24, 2008 @ 10:02 pm
@anil: I’m not certain how structs are laid out on big/little endian systems (in the order variables are declared or otherwise). The easiest method may be to use the byte-walking method above.

@Sharada: Most network protocols should convert everything to “network order” when transmitting — it’s probably a bug if this doesn’t happen! (Or if the client & server agreed not to use the standard protocols).

@Ajish: Yes, I imagine 8-byte information is stored similarly (as the jump from 2 to 4, extrapolate from 4 to 8; it always pays to double check though!).

@pramod: Thanks, glad you enjoyed it.

Kalid — October 1, 2008 @ 11:31 am
Good article. I am having trouble talking to SD cards which are little-endian. I assume that, when sending the data serially, the LSB of byte 0 goes first.

Delron — October 21, 2008 @ 6:24 pm
Thanks Delron. When reading a stream of data for something as modern as an SD card, I’d assume that it would be given to you in 1-byte chunks, so hopefully individual bits aren’t an issue. (I.e. you could read/write 1 byte at a time — just my guess).

Kalid — October 21, 2008 @ 7:07 pm
Thanks for your reply. It let me take another run at the problem. It turns out to be a good example of the confusion that can result in “endians”. I see now that the SD card runs on the SPI protocol which is big-endian, however, the data that the big_endian SPI is transferring to the card uses the FAT file system which is little_enidian. I just have to remember that a byte is a byte, as you so correctly emphasize.

Delron — October 21, 2008 @ 10:22 pm
@Delron: Wow, I thought I had it rough when having to tinker around with networking . Yeah, it helped me to realize that the endian craziness stops at the byte level, and a byte’s a byte.

Kalid — October 21, 2008 @ 10:34 pm
Hello guys wassup?

I have a doubt…earlier this guy said that to fix the endianness issue we can use a magical number before every piece of data, n if the we read the magic number n he is 0xFEFF we have a data with the same format else if 0xFFFE we have to swap memory, but i don get the idea very well

So e.g. if i have a pixel (his order is RGBA) on a little endian machine i can use a rmask n make a AND?
Someone can help me to understand better? ^^

Shanx — November 13, 2008 @ 12:00 pm
Kalid, hi man!

U can explain the thing about put a magic number before every data? I don get it very well =\

If i have a vector of pixels that i pick n my machine is little endian e.g., n the format of pixels is RGBA (bing endian), like this

0 X RR GG BB AA

what is the idea? How i put a magic number before this pixel n mak a test?

Thanks! ^^

Shanx — November 13, 2008 @ 12:22 pm
Hi Shanx, good question. The “magic number” is a special header everyone agrees to write before the data.

If you’re reading data and the magic number appears backwards (0xFEFF instead of 0xFFFE), you know the data was stored in a different format and therefore you must byte-swap the data.

This is the strategy used in some Unicode documents — include a header field so people can tell when someone else wrote it differently. So, you just need to check whether the header == 0xFFFE (same byte order) or header == 0xFEFF (different byte order).

Kalid — November 13, 2008 @ 6:49 pm
awesome is the word…plesae try to include the solution to the problem in detailed!
thanks,.

dada — November 26, 2008 @ 5:07 am
Awesome post! Thank you for writing this!

mike — December 13, 2008 @ 9:23 am
@mike: Thanks!

Kalid — December 13, 2008 @ 10:12 am
Wow, that explained everything so thorougly. Other articles gave the basics, yet none gave you the practice.

Thanks!

Michael — January 3, 2009 @ 1:30 am
Thanks for the note Michael! Yes, I find it helps to cement understanding with a real example of what’s going on.

Kalid — January 3, 2009 @ 1:52 am
Excellent article! Very informative with great examples, and just enough light-heartedness thrown in to make it a fun read.

I have read many other articles on endianess and somewhat understood the topic. But this was the first article I came away from feeling like I understand it well enough to explain it to someone else (the ultimate test IMHO as to whether you truly understand a topic).

*Thank you* for taking the time to write this. I can honestly say this knowledge will help me in my career; which in turn will help my earning potential. So you should feel very good knowing you have made such an impact in someones life via your efforts.

Mark — January 16, 2009 @ 11:46 am
Thanks Mark, your note made my day . I know what you mean, it took a while for me to really get why endian problem happened, and why only for multi-byte data (single bytes and the order of bits are a common denominator that all computers agree on, otherwise we’d have the “bit ordering problem” too).

Regarding writing, I find it’s no fun for the writer or reader when done in a stuffy style. Thanks again for the comment.

Kalid — January 16, 2009 @ 12:33 pm
thnks for this explaination I always had problem with big and little endian

vikas — January 19, 2009 @ 12:29 am
many thanks for your great post.

but where should I set the commend of “setenv/export F_UFMTENDIAN=unit”? in the program, in the compile file or ?

Anonymous — January 19, 2009 @ 9:24 am
many thanks for your great post.

but where should I set the command of “setenv/export F_UFMTENDIAN=unit”? in the program, in the compile file or ?

Anonymous — January 19, 2009 @ 9:25 am
Great post! It was really helpful.
Thanks a lot!

Faisal Shaikh — February 5, 2009 @ 12:46 am
great job ….. thanks

vijay — February 9, 2009 @ 5:29 am
this is good man..even i can only understand 1/4 of it..this should be enough..cheers man.

salaz — February 11, 2009 @ 1:00 pm
Thanks, very nice explanation …

Thura — February 16, 2009 @ 12:06 am
Excellent explanation! Much Better Explained than other sources I’ve seen. Well done!

Bob — February 20, 2009 @ 1:51 pm
Excellent explanation! Much *Better Explained* than other sources I’ve seen. Well done!

Bob — February 20, 2009 @ 1:52 pm
@salaz, Bob, Thura: Thanks!

Kalid — February 20, 2009 @ 2:24 pm
Cảm ơn rất nhiều! / Thank You very much! / Merci beaucoup! / Danke Sehr! / …
But I still *wonder* that in Little-Endian, ‘UNIX’ will be stored as ‘XINU’, not ‘NUXI’ (??). Because the *number* UNIX is a double-word (4-byte): HIGH-word is ‘UN’ (2-byte), LOW-word is ‘IX’ (also 2-byte). And due to Little-Endian, I have the LOW-word stored first, then the HIGH-word second –> ‘IX’, ‘UN’.
Next, in each WORD-number, still due to Little-Endian, the *number* IX (2-byte) will be stored in usual order: X I. Same thing for the 2-byte UN *number* : N U.
So, a 4-byte ‘UNIX’ will be stored (in memory -of course) as: X I N U (?!?!). Do I confuse something? Hope to hear from You soon… ^_^

chipvang — February 25, 2009 @ 1:22 am
@chipvang: You’re welcome, gracias, danke!

If UNIX was stored as a double-word (4 bytes) then it would become XINU. But, the problem refers to storing UNIX as two shorts: UN and IX. So if you switch the endian on each short, it becomes NU and XI, or NUXI.

Great question — it confused me too.

Kalid — February 27, 2009 @ 1:39 am
thanks a lot.

truonglvx — March 1, 2009 @ 8:51 pm
Great article … detailed view right from Basics !

samasat — March 9, 2009 @ 3:30 pm
Great article! However (there’s always a however for engineers…), I’d like to suggest a couple of additions:

1) talk a bit about mixed-endian, like some ARM (and IBM?) processors have used. They can be BE for 16-bit values, but have two 16-bit values LE as a 32-bit read (or was that 32 and 64 – I still get confused).

2) Also, your explanations and example are all software/firmware based/targeted. For us hardware designers, mostly using Verilog, LE naturally makes sense and is assumed by the language. For example, if I declare a 32-bit vector and assign a value to it, it’s naturally LE:

wire [31:0] my32bitregister;
assign my32bitregister = 32′h12_34_56_78;

In this case,
my32bitregister[ 7: 0] is 8′h78
my32bitregister[15: 8] is 8′h56
my32bitregister[23:16] is 8′h34
my32bitregister[31:24] is 8′h12

I think most of my big design headaches over the years have been byte/word swapping to convert endianness!

David Rogoff — March 18, 2009 @ 3:26 pm
Hi
Your approach to the confusing topic is really simple.you made it easy to understand.
Thanks

Amey — March 30, 2009 @ 11:18 pm
pls tell me why the word address is jumping from 0 to 4..

Anonymous — April 10, 2009 @ 12:59 am
Hey,
thank you very much for that introduction. It is way nicer to read than the respective Wikipedia entry.
One thing you may want to change are the pointer initialized to 0. It might be a little confusing because they’re actually NULL pointers. You mention that, but why not just let the addresses start at 1 instead of 0?

Keep up the good work
JAAG

Another anonymous guy — April 12, 2009 @ 2:24 pm
very nice article keep up the good work

rajKumar — April 25, 2009 @ 12:26 pm
I’m trying to interact from a little endian machine to a big endian – Motorola machine. I get data from the big endian machine in bytes. I never know the type of data packed in it. So I’m don’t know if I’m receiving integer or shot int or whatever type.

How can I rightly interpret the data in the little endian machine?

Lakshminarayanan — May 5, 2009 @ 8:10 am
Great Article, I could understand the byte ordering better than any other source of Information. I still have one question which i am not able to understand. Say we have a binary data ( a JPEG Image) created in little endian machine and its transmitted to a big endian machine. If we compute a SHA1 hash of the file in both the machine do they give same result of they would give different hash.
If i read the file as byte by byte in base64 encoded format and hash them in both the machine. Please see i am not rendering the file, i am just interested in computing the hash of the file.

Mani

Mani — June 10, 2009 @ 9:33 am
@Lakshminarayanan: Unfortunately, you need to know the data types in the byte stream if you want to convert from big to little endian.

@Mani: Thanks. SHA1 looks at data as a stream of bytes, so big and little endian shouldn’t matter. Also, file formats like JPEG usually have a standard format they agree on, so it will be the same file on both systems.

Kalid — June 10, 2009 @ 1:08 pm
AUBE FUTI LY0L PLEX NEAI 000D.

( BEAUTIFULLY EXPLAINED. )

KUDOS.

Harish — June 11, 2009 @ 2:19 am
really a gud one….

neeraj — June 17, 2009 @ 3:02 am
great post, my lecturer couldn’t explain as good as you do

Johanes — June 29, 2009 @ 4:10 pm
Boy… You cleared my issues which I was struggling with for two hours! trying to read a big endian based data from a file into C…. Thanks…

rahul — July 15, 2009 @ 2:53 pm
@Harish, Neeraj, Johanes, Rahul: Thanks!

Kalid — July 15, 2009 @ 2:59 pm
re. floats: If they are the same kind of IEEE float on your source machine and destination, then the bytes will be compatible, but the byte order may not be the same.
Unlike 8, 4, and 2 byte integers, floats do not have an official network order, but I think common practice is to treat the bytes like a 4 byte int when converting from host to network order.

David Whalen-Robinson — September 4, 2009 @ 6:19 pm
re. floats (part 2):
You can save yourself a lot of pain by looking at how the numbers are stored in memory. So if your number’s bytes looks like 01 02 03 04 in memory, and your buffer from the network has 04 03 02 01, you know you are going to have to flip the order somehow.

David Whalen-Robinson — September 4, 2009 @ 6:26 pm
@David: Thanks for the details!

Kalid — September 10, 2009 @ 1:04 am
Here’s where the NUXI analogy falls apart for me. Consider (using a 16 bit value from Kalids example):

0×03 e7 (BigE)

0xe7 03 (LittleE)

When viewed from the context of the UNIX.. I have
U = 0
N = 3
I = E
X = 7
When viewed from a NUXI perspective I get:
N = 3
U = 0
X = 7
I = E

0×30 7E is not the same as 0xE7 03. Having a hard time drawing synergy between the two

MP — September 30, 2009 @ 4:39 am
@MP: Hi there, in the original NUXI analogy, the data was stored as two shorts (so “UN” “IX” became “NU” “XI” -> each short was reversed). This example was picked, I believe, because it makes UNIX turn into another readable word.

In your example, the data is is stored as a single 4-byte integer, so as you notice it would transform UNIX to XINU, not UNIX to NUXI. Hope this helps!

Kalid — October 2, 2009 @ 2:09 pm
Hi Kalid! I’m rather a noob at programming and I need your help in programming for a Cryptography problem.

How to come up with the pseudocode and program(using C++ or Java) for this problem:

In its binary form of Big-Endian byte-order format, create a program that computes “sig” for a given data from a file. Whereby

sig = SHA-1(m)^d mod N

where “m” is the byte stream formed by the content of the file; “d” and “N” are given as well.

I’m not sure of converting to binary form in computer language and whether this SHA-1 function is given in library.

Appreciate your help.

Shah — October 30, 2009 @ 1:05 am
First, thanks a lot!

I have a question. Can U just tell me if I get this right?
If I have a hexadecimal number 12A4382
and I want to represent it on BigE and LittleE
and addresses start from 1000, which byte would be on each address?
I think:
BigE:
01 on 1000
2A on 1001
43 on 1002
82 on 1003

LittleE:
82 on 1000
43 on 1001
2A on 1002
01 on 1003

is this right?

taz — November 2, 2009 @ 5:48 am
Nice article!

I have a doubt. The article says that the storage is same for little and big endian machines.

But when i check the following code in Intel Pentium 4, it displays as little endian:

(snip)

#include

void main()
{
short Var = 0×1234 ;
char *ptr ;

ptr = (char *) & Var;

if( *ptr == 0×12)
{
ptr++ ;
if(*ptr == 0×34)
{
printf(”\nBig Endian\n”) ;
}
}
else if( *ptr == 0×34)
{
ptr++ ;
if(*ptr == 0×12)
{
printf(”\nLittle Endian\n”) ;
}
}

}

(snip end)

My doubt is, when i see the memory, the value is stored in the order “34 12″. That is, the order corresponding to little endian.

As per the article, it should have been stored as “12 34″.

Pls clear my doubt.

Many thanks in advance.

Murugesh — November 12, 2009 @ 7:25 am
@Murugesh: Hi, Pentium chips are little endian, so you’re getting the right answer! (It should be 34 12 in this case). Hope this helps.

Kalid — November 13, 2009 @ 10:21 am

RSS feed for comments on this post. TrackBack URI

Have a question? Know an explanation that caused your own a-ha moment? Write about it here.