[gambit-list] A couple of random questions
feeley at iro.umontreal.ca
Mon Jul 25 07:23:10 EDT 2011
On 2011-07-22, at 4:41 AM, mikel evins wrote:
> Question 2:
> (I asked this one before, but didn't see an answer.)
> Suppose I have a file containing data serialized by calling object->u8vector and then written to a file. Suppose, further, that I happen to know that an object of interest to me starts at a specific byte offset within the file.
> What's the right way to read that object from the file?
> An additional constraint is that I don't want to have to read the entire contents of the file in order to read the object; I want to be able to read just enough bytes to reconstitute the object whose serialized representation starts at the given offset.
The object encoding used by object->u8vector (and u8vector->object) does not contain a length. The stream of bytes has to be parsed to know how long the object encoding is. To facilitate I/O, I usually prefix the encoding with a 32 bit integer which indicates the length of the encoding.
A second problem is that the object encoding supports shared structures. This means that the object created with
(let ((shared (cons 1 2))) (cons shared (cons 3 shared)))
will be encoded with something like this:
OBJ1=[pair 1 2]
OBJ2=[pair 3 OBJ1]
or if you want all the details:
(let ((shared (cons 1 2))) (cons shared (cons 3 shared))))
#u8(100 100 81 82 100 83 129 0)
^ ^ ^ ^ ^ ^ ^
| | | | | | |
| | 1 2 | 3 reference to shared object 0
| | |
| | "pair" (index 2)
| "pair" (index 1)
"pair" (index 0)
So you can't just start parsing an object from an arbitrary point inside the encoding. Some context will be missing.
The approach I would take is to dump each object independently in the file and keep a separate index of where each object starts, and its length (then use input-port-byte-position to seek to the right place). Even better, store the serialized objects in a database and let it do the indexing for you (it also simplifies deleting objects).
More information about the Gambit-list