Editing binary files

bbloke

bbloke

Registered
I would greatly appreciate any help that anyone could offer with regards to a problem I am working on. I need to edit data that is outputted from one program, running under IRIX, in a 32-bit long integer format. I want to be able to divide that data into segments (for instance, into 8 K segments) and then feed it back into the originating program.

Now, I found "od" (more specifically "od -S") was a great help when wanting to convert the data to ASCII and THEN process it. However, I could not find a way to convert the data back to 32-bit long integer format, so I now want to work with the original data. "head," "cut," "sed," and so on do all work with the binary data but I can't find a way of using them to divide the data into segments.

Does anyone have any ideas? :(
 
Thanks, but I don't know any perl (or tcl, c, or the like!). I'm fairly new to Unix and only know shell scripting, really. I've found that using "head" or "tail" does indeed cut the raw data, but that I do not have as much control as I'd like. For instance, using "head -1" gives me 103 (?!) data points rather than a nice even number. Similarly, "sed" does not allow me to work by character position (in bytes), only line numbers. An alternative to segmenting the original data would be to find a way of getting the ASCII back into 32-bit long integer format!

Do you have any further details about the perl features that might be of use?

Thanks for the rapid feedback, by the way! :)
 
I didn't understand a word you said. (Maybe I shouldn't post in that case... oh well.)
 
Originally posted by arden
I didn't understand a word you said. (Maybe I shouldn't post in that case... oh well.)
Click to expand...

Erm...

OK, I was trying to keep it short, as it is a very long story! Trust me.

The basic idea is that there is a program running under IRIX (another variant of UNIX) which produces our raw data for our research in a particular format. I want to "chop" this data into a series of smaller, evenly sized segments (eg taking 512 K data sets and chopping them into sixteen 32 K files, perhaps) so that I can analyze each bit.

I have found various commands for editing text do indeed work with the raw data, but that I do not have the precision I need in determining where the data is segmented. This is because the standard text editors do not seem capable of cutting files at specific size-determined positions, they can only make changes based on searches for specific characters. So, I am after a method for snipping my raw data at precisely the right points, without having to convert it to ASCII first. If I converted it to ASCII, I'd need a way of converting it back to the raw data format again, and I have not yet found a way of doing so.

Is that any better? I can give the full story if necessary, but I doubt people would want it! :)
 
is there any regular pattern? all on one line or a certain number of bytes per line? is it just a stream with no spaces or carraige returns? I was thinking maybe something as simple as vi would work, since you can specifiy a number before most actions such as hitting 5l moves you five spaces, maybe you could do a 32768l or whatever the cut command is. just a thought
 
Oh, what you want to do is easy then :)

You have a 512k file, and you want a bunch of 32k files. Use the split command. split takes a file and splits it into files of X size. So,

split -b 32k filename newname

That will make a bunch of files name newname.aa newname.ab newname.ac, etc, each of 32k, with the last one being the remainder if it doesn't divide evenly.

Brian
 
:D

You're never going to believe this, but I found out about the "split" command today whilst at work and was going to post this fact this evening, but you beat me to it!

Yes indeed, the split command does what I need, I have tested it, and the program running under IRIX is able to read the files (with a little assistance, but that is another matter).

Thanks for all the help, it was much appreciated! :)
 
You must log in or register to reply here.
Back
Top