pack: Binary encoding and decoding

This module provides low-level facilities for converting Alore objects to and from packed binary representations (this is called packing and unpacking, respectively). The packed representations are narrow strings that can written to files, sockets and other narrow streams. Strings containing character codes larger than 255 (wide strings) must be encoded before passing them to the packing methods (using the encodings module, for example).

This module is typically used for accessing and generating binary data using standard binary formats, for implementing binary network protocols and interoperating with lower-level languages such as C.

Pack formats

Binary data can be packed in several different formats. Some formats can only pack integer data, others floating point numbers, and finally there is a format for string data. There are no separate formats for different endiannesses — instead there are separate classes for packing and unpacking big endian and little endian data.

The table below contains descriptions of the supported formats, including sizes of the packed representations and the corresponding Alore types that can be packed into and unpacked from using each format (note that N in a format name must be replaced by a valid integer):

Format Description Packed bits Packed bytes Alore type
Byte unsigned byte1 8 1 Int
WordN unsigned integer N (16, 32 or 64) 2, 4, or 8 Int
IntN signed integer2 N (8, 16, 32 or 64) 1, 2, 4 or 8 Int
FloatN floating point number3 N (32 or 64) 4 or 8 Float
Str narrow string variable4 variable4 Str

Notes:

 1   The Byte format is thus named for convenience; for consistency's sake it could equivalently have been named Word8.
 2   Signed integer formats (Int8 to Int64) are encoded using the standard 2's complement encoding.
 3   The Float32 and Float64 formats correspond to single and double precision IEEE floating point numbers, respectively.
 4   The width of a Str format is always specified with an additional integer argument specifying the length of the packed string in bytes. When packing, input strings may be smaller than the length; they will be padded with spaces so that the desired length is reached.

Packer classes

There are two concrete packer classes, one for the big-endian (network) byte order and other for the little-endian byte order. They both support the same methods; the common abstract superclass PackerBase defines these.

Class Packer

Inherits PackerBase

class Packer()
Construct a packer that packs and unpacks data in the big-endian (network) byte order.

Class LittleEndianPacker

Inherits PackerBase

class LittleEndianPacker()
Construct a packer that packs and unpacks data in the little-endian byte order.

Class PackerBase

This is an abstract base class of the packer classes.

Methods

Note: Instances of N in method names below must be replaced with an integer valid for the format. See the above table for details. For example, the actual methods for packing words are packWord16, packWord32 and packWord64.

packByte(int as Int) as Str
packWordN(int as Int) as Str
packIntN(int as Int) as Str
packFloatN(float as Float) as Str
packStr(str as Str, len as Int) as Str
Pack a value in a specific format. The argument must have a compatible Alore type for the format (integers are also valid when floats are expected). The result is a narrow string.
packBytes(array as Array<Int>) as Str
packWordsN(array as Array<Int>) as Str
packIntsN(array as Array<Int>) as Str
packFloatsN(array as Array<Float>) as Str
packStrs(array as Array<Str>, len as Int) as Str
Pack an array of values in a specific format. The argument must be an array, and each array item must have a compatible Alore type for the format. The result is a narrow string that is the concatenation of the packed items.
unpackByte(str as Str) as Int
unpackWordN(str as Str) as Int
unpackIntN(str as Str) as Int
unpackFloatN(str as Str) as Float
unpackStr(str as Str) as Str
Unpack a narrow string encoded in the format specified by the method name. The argument must have the correct length for the format. The Alore type of the result value is specified by the format.
unpackBytes(str as Str) as Array<Int>
unpackWordsN(str as Str) as Array<Int>
unpackIntsN(str as Str) as Array<Int>
unpackFloatsN(str as Str) as Array<Float>
unpackStrs(str as Str, len as Int) as Array<Str>
Unpack a narrow string that contains any number of values encoded in the format specified by the method name. The argument must have a valid length for the format (any multiple of the single item length). The result is an array of values whose type is specified by the format.

Examples

var p = Packer()
p.packWord16(7)               -- Result: "\u0000\u0007"
p.packBytes([16, 128])        -- Result: "\u0010\u0080"
p.unpackBytes("\u0010\u0080") -- Result: [16, 128]

var pl = LittleEndianPacker()
pl.packWord16(7)              -- Result: "\u0007\u0000"

Class PackStream

Inherits Stream

class PackStream(stream as Stream[, packer as PackerBase][, bufMode as Constant])
Construct a PackStream instance that uses another stream for input/output. The packer argument, if present, must be a packer object. If omitted, the big endian packer will be used. The bufMode argument may be Buffered, LineBuffered or Unbuffered. If omitted, full buffering will be used.

PackStream methods

Note: Instances of N in method names below must be replaced with an integer valid for the format. See the above table for details. For example, the actual methods for writing words are writeWord16, writeWord32 and writeWord64.

writeByte(int as Int)
writeWordN(int as Int)
writeIntN(int as Int)
writeFloatN(float as Float)
writeStr(str as Str, int as Int)
Pack data and write to the stream. The arguments are similar to the corresponding pack* methods (see above).
writeBytes(array as Array<Int>)
writeWordsN(array as Array<Int>)
writeIntsN(array as Array<Int>)
writeFloatsN(array as Array<Float>)
writeStrs(array as Array<Str>, int as Str)
Pack an array and write to the stream. The arguments are similar to the corresponding pack* methods (see above).
readByte() as Int
readWordN() as Int
readIntN() as Int
readFloatN() as Int
Read and unpack data from the stream. The format specifies the number of characters (bytes) to read and the type of the return value.
readBytes(n as Int) as Array<Int>
readWordsN(n as Int) as Array<Int>
readIntsN(n as Int) as Array<Int>
readFloatsN(n as Int) as Array<Float>
readStrs(n as Int, len as Int) as Array<Str>
Read and unpack multiple data items from the stream. The format and the length argument specify the number of characters (bytes) to read and the length of the Array return value and the type of its items.
eof() as Boolean
seek(n as Int)
close()
Call the corresponding method of the underlying stream (if it is supported).

The inherited writing and reading methods (write, readLn, etc.) behave normally and can be used in addition to the PackStream specific input/output methods.

Class PackFile

Inherits PackStream

class PackFile(path as Str[, packer as PackerBase], ... as Constant)
Open a binary file for reading and/or writing. The supported additional arguments are the same as for io::File. The packer argument, if present, must be a packer object (see above). If omitted, the big endian packer will be used.

Example: 

var f = PackFile("file.bin", Output)
f.writeWord32(1234)
f.writeBytes([1, 2, 3, 4])
f.close()

Constants

HostByteOrderPacker as def () as PackerBase
This constant refers to either Packer or LittleEndianPacker depending on the native byte order of the host system.

Exceptions

class PackError
Raised when a packer class gets invalid input and cannot perform packing or unpacking. Inherits from std::ValueError.