• 11
name

A PHP Error was encountered

Severity: Notice

Message: Undefined index: userid

Filename: views/question.php

Line Number: 191

Backtrace:

File: /home/prodcxja/public_html/questions/application/views/question.php
Line: 191
Function: _error_handler

File: /home/prodcxja/public_html/questions/application/controllers/Questions.php
Line: 433
Function: view

File: /home/prodcxja/public_html/questions/index.php
Line: 315
Function: require_once

name Punditsdkoslkdosdkoskdo

Python how to write to a binary file?

I have a list of bytes as integers, which is something like

[120, 3, 255, 0, 100]

How can I write this list to a file as binary?

Would this work?

newFileBytes = [123, 3, 255, 0, 100]
# make file
newFile = open("filename.txt", "wb")
# write to file
newFile.write(newFileBytes)

This is exactly what bytearray is for:

newFileByteArray = bytearray(newFileBytes)
newFile.write(newFileByteArray)

If you're using Python 3.x, you can use bytes instead (and probably ought to, as it signals your intention better). But in Python 2.x, that won't work, because bytes is just an alias for str. As usual, showing with the interactive interpreter is easier than explaining with text, so let me just do that.

Python 3.x:

>>> bytearray(newFileBytes)
bytearray(b'{\x03\xff\x00d')
>>> bytes(newFileBytes)
b'{\x03\xff\x00d'

Python 2.x:

>>> bytearray(newFileBytes)
bytearray(b'{\x03\xff\x00d')
>>> bytes(newFileBytes)
'[123, 3, 255, 0, 100]'
  • 127
Reply Report
      • 1
    • Nice use of builtin types. Just note that bytearray was added in 2.6, if you want to support legacy systems, it should be avoided.
    • @Perkins: Sure, and you should avoid generator expressions if you need to work on 2.3, be careful with both str.encode and struct.pack if you need to work on 2.2. But 2.6 has been out for 5 years now; all three Ubuntu LTSs still in support, all three OS X versions in support, the previous major version of CentOS/RHEL, etc., all come with it built in. If you need to support 2.5 or 2.1 or 1.6 or whatever, you probably know…
      • 1
    • With Python 2 on Windows, I found that writing a bytearray still converts to , making it unsatisfactory for binary data, if the "b" flag is not passed when opening the file.
      • 1
    • @feersum: Of course; that's what binary vs. text mode means in 2.x. It doesn't matter what type your bytes come from. (In 3.x, of course, binary vs. text mode means that you write bytes vs. unicode, and the feature is part of the universal newlines options for text.)
    • I'm not sure bytearray() is a good choice for file writing. You would need to limit the size to manageable chunks. Otherwise once your filesizes get too high you will run out of memory.

Use struct.pack to convert the integer values into binary bytes, then write the bytes. E.g.

newFile.write(struct.pack('5B', *newFileBytes))

However I would never give a binary file a .txt extension.

The benefit of this method is that it works for other types as well, for example if any of the values were greater than 255 you could use '5i' for the format instead to get full 32-bit integers.

  • 29
Reply Report
    • You're right, struct.pack is also the way to go if you are going to be writing data with values above 255, since neither bytearray nor chr can handle larger integer values.
      • 2
    • @MarkRansom: Well, this is still definitely a good solution to the more general problem of "I have a list of integers of some arbitrary but fixed size, how can I write them to a binary file?" and I can see people searching for that question and finding this one…
      • 1
    • .txt is fine if you have some way to knowing that the data you are writing all falls inside the printable ascii range. However, you are correct I think in this case, since the example data includes non printable characters.
    • @Perkins I didn't make the assumption that the values would even be less than 256 much less in the ASCII range. Even if they are, .txt files should be reserved for those that make sense to a human which never applies to binary data.

To convert from integers < 256 to binary, use the chr function. So you're looking at doing the following.

newFileBytes=[123,3,255,0,100]
newfile=open(path,'wb')
newfile.write((''.join(chr(i) for i in newFileBytes)).encode('charmap'))
  • 12
Reply Report
      • 1
    • You must mean < 128. As python3 complains: UnicodeEncodeError: 'ascii' codec can't encode character '' in position 0: ordinal not in range(128)
    • No, I mean < 256, but the encoding should be charmap rather than ascii, and works in python2 as well as python3. The ascii encoding only works in python2.

As of Python 3.2+, you can also accomplish this using the to_bytes native int method:

newFileBytes = [123, 3, 255, 0, 100]
# make file
newFile = open("filename.txt", "wb")
# write to file
for byte in newFileBytes:
    newFile.write(byte.to_bytes(1, byteorder='big'))

I.e., each single call to to_bytes in this case creates a string of length 1, with its characters arranged in big-endian order (which is trivial for length-1 strings), which represents the integer value byte. You can also shorten the last two lines into a single one:

newFile.write(''.join([byte.to_bytes(1, byteorder='big') for byte in newFileBytes]))
  • 9
Reply Report

You can use the following code example using Python 3 syntax:

from struct import pack
with open("foo.bin", "wb") as file:
  file.write(pack("<IIIII", *bytearray([120, 3, 255, 0, 100])))

Here is shell one-liner:

python -c $'from struct import pack\nwith open("foo.bin", "wb") as file: file.write(pack("<IIIII", *bytearray([120, 3, 255, 0, 100])))'
  • 8
Reply Report

Use pickle, like this: import pickle

Your code would look like this:

import pickle
mybytes = [120, 3, 255, 0, 100]
with open("bytesfile", "wb") as mypicklefile:
    pickle.dump(mybytes, mypicklefile)

To read the data back, use the pickle.load method

  • 1
Reply Report
      • 2
    • This does not produce a binary file of 5 bytes length, where the only content is 120, 3, 255, 0, 100. In a closed system, this may be acceptable though.

Trending Tags