Download Shareware and Freeware Software for Windows, Linux, Macintosh, PDA

line Home  |  About Us  |  Link To Us  |  FAQ  |  Contact

Serving Software Downloads in 976 Categories, Downloaded 29.540.328 Times

Parsing binary files with regular expressions 1.0

  Date Added: May 10, 2013  |  Visits: 351

Parsing binary files with regular expressions

Report Broken Link
Printer Friendly Version


Product Homepage
Download (29 downloads)



This script allows you to use the regular expression engine to parse binary files, especially those for which the struct module alone is inadequate.The typical way to parse binary data in Python is to use the unpack method of the struct module. This works well for fixed-width fields, but becomes more complicated when you need to parse variable-width fields. Perl's implementation of unpack accepts "*" as the field length, and even allows grouping with parentheses, which mitigates this problem. Python does not currently offer these features. Although you can dynamically generate a format string for unpack with a lot of slicing and calls to calcsize, the resulting code will likely be hard to read and error-prone.Fortunately, in some cases there is a simpler way to do it: use the regular expression engine to grab each field, and use struct.unpack on the results.First, you construct a regular expression (RE) describing the entire record structure, grouping each field you'd like to extract with parentheses, and compile it. To create the regular expression, you just have to remember that one character in the RE equals one byte in the record. So, the expression ".." would match any short (2 bytes). To match a variable-width field, the REengine will have to be able to recognize where the field ends. In a null-terminated string, for example, the field ends with a zero byte. You'd therefore look for any number of characters followed by a null byte: "(.*?)". Note the use of the non-greedy qualifier "?" -- this way, we only match up to the first null, rather than the last null in the buffer.When compiling, make sure to pass the re.DOTALL flag to the compiler, or it will consider bytes that happen to match ASCII '' to be newlines. Then, you use the findall method of the compiled expression object on your buffer. findall finds all non-overlapping matches, one match for each record. It returns a list of tuples, one for each match; each tuple will contain one element for each field you grouped in the RE.You still need to unpack the fields in the tuples before using them, since they're still strings rather than usable values. Generally, you'll call unpack once for each field, with only one format character. (You can also group multiple consecutive fixed fields in one set of parentheses in the RE, and then unpack them in one call. But that may get confusing.)The code above demonstrates how to unpack a binary file that has an indeterminate number of variable-width records, each consisting of a little-endian short, a null-terminated string, and two more shorts. It drops the resulting values into a list and also into a dictionary.This technique is useful when your variable-width fields are terminated with a sentinel, such as the zero-terminated strings described above. If your field length is embedded in the data, and you can't use the "p" (Pascal string) modifier, you'll probably have to resort to slicing the buffer up manually.This technique is also applicable even if your fields are all fixed-width. The findall method will operate on the entire buffer at once with a single regular expression, which saves you from having to dynamically create a long format string encapsulating all your data, or alternatively iterating over slices of the buffer.

Requirements: No special requirements
Platforms: Windows, Mac, *nix, Mac OSX, Linux, Python , BSD Solaris
Keyword: Binary Binary Files Parser Files Files Management Parser Regular Expressions
Users rating: 0/10

License: Freeware
USER REVIEWS
More Reviews or Write Review


PARSING BINARY FILES WITH REGULAR EXPRESSIONS RELATED
Programming  -  TestGen4J 0.1.4-alpha
TestGen4J automatically generates JUnit test cases from your own Java class files or source files. Its primary focus is to exercise boundary value testing of the arguments passed to the method. It uses rules, written in a user-configurable XML...
849.92 KB  
Graphical Text Tools  -  AutoCAD DWG Viewer 3.4.1
FREE AutoCAD Drawing Viewer is a lightweight powerful utility for viewing, editing, printing, converting and saving AutoCAD drawing files. Features include: Print drawings, create PDF files from drawing files in batches, convert drawings to...
26.13 MB  
Graphical Text Tools  -  AutoCAD Drawing Viewer 3.7
FREE AutoCAD Drawing Viewer is a lightweight powerful utility for viewing, editing, printing, converting and saving AutoCAD drawing files. Features include: Print drawings, create PDF files from drawing files in batches, convert drawings to...
134.75 KB  
Programming  -  Convert ANSI files to UNICODE files 0.0.1
Console program to convert ANSI files to UNICODE files. Including Recursive option and filter.
4.32 KB  
Automation Tools  -  Delete Lines 1.1.0
Delete/Filter lines from big text files (for example log files). If you have ever been confronted with the problem that you want to delete or filter particular lines of a big text file, this free program is exactly what you are looking for. With...
1.25 MB  
Miscellaneous  -  pkgconfig parser 1.0
This script creates a class to allow access to variables stored in pkg-config files ( or '.pc' files ). This is usefull in conjunction with distutils to get correct information for compiling external C/C modules. Variable substitution is performed...
 
Networking Tools  -  Mibble 2.8
Mibble is an open-source SNMP MIB parser (or SMI parser) written in Java. Mibble project can be used to read SNMP MIB files as well as simple ASN.1 files. Mibble is distributed as a Java library to make it possible to include it in your...
3.7 MB  
Utilities  -  Compress Files '08 for Mac OS 4.5
Compress Files is an universal program for Mac OS X that enable Mac users to create Zip compressed and encrypted files; XAR archive files, 7zip and 7zip crypted archives, TAR archives; Gzip compressed archives; Bzip2 compressed archives and...
3.2 MB  
Book Collection Managers  -  Find and Remove Duplicate Files 1.0.0.88
Many of peoples are unable to find duplicate files on Windows computers. Here is the best way solve this problem i.e. download and employ Find and Remove Duplicate Files software that helps you to find duplicate files within few seconds. It is...
25.05 MB  
Web Authoring  -  Log Parser Lizard 1.0
Log Parser is a very powerful, free and versatile tool that provides universal query access to text based data, such as log files, XML files, and CSV files, as well as key data sources on the Microsoft Windows operating system, such as the event...
4.89 MB  
NEW DOWNLOADS IN SCRIPTS, FILE MANAGEMENT
Scripts  -  Free Ecommerce website creator 1.2
Free Ecommerce website creator is a free PHP shop creating script. This allows you to put a online shop on your own website. Create your own free ecommerce website for Your Business. Create an online shop using easyGUI online shop creator. The...
1.44 KB  
Scripts  -  MochiGames PHP Script ZDR 1.00
MochiGames PHP Script ZDR is web site, ready for use, for flash games. These flash games are downloaded automatically by "MochiGames PHP Script ZDR" from MochiGames media. The use of the games is free, you can use your own Mochi Publisher ID and...
368.54 KB  
Scripts  -  Php Chat 2.0
Add a free php site, single sign-on and multiple skins, 100% free 1. Server Modes: The chat server has paid mode and free mode. If the free chat mode, a free chat room will be assigned to your website with your domain as the room name. 2....
938.87 KB  
Scripts  -  Nibbleblog 3.0.1
Nibbleblog it's a powerful engine for creation and manipulation of BLOG's completely free. Very simple to install and configure (Only 1 step). The database used is based on XML files and this way it is not necessary to use MySQL or similar DBMS....
371.09 KB  
Scripts  -  PHP File Manager | CloudOsys 2.9b8
CloudOsys is a PHP file manager, a tool that allows your visitors upload files such as media content directly to your website. Your visitors will upload files directly to your website, where they can share and comment on them. Through cloud...
1.41 MB  
File Management  -  YetiShare - File Hosting Script Free 2.1
YetiShare is a PHP script that allows you to create your own professional file sharing service that you can earn a revenue from. The script has a wide range of features including an extensive admin area, user accounts, and more. This is a free...
6.2 MB  
File Management  -  XtraUpload for File sharing 1.6
XtraUpload has all the features you would expect from a file Hosting Script. Free users as well as premium users have the ability to upload files but premium users get a lot more features such as viewing files, instantly downloading files and...
4.8 MB  
File Management  -  PHPGnuCacheII for File sharing 2.1.1
PHPGnuCacheII is a PHP/MySQL WebCache script for the gnutella/gnutella2 Peer-to-Peer network. It implements the Version 2 GWebCache specifications.
20.48 KB  
File Management  -  TorrentFlux for Download Managers 2.3
TorrentFlux is a FREE PHP based Torrent client that runs on a web server. It allows you to manage all of your Torrent downloads through a convenient web interface from anywhere. Features:- Upload Torrents via URL or File Upload- Start, Stop, and...
1.29 MB  
File Management  -  FastBeats for File sharing 2.0.95
FastBeats is an all-in-one BitTorrent Tracker content management system written in PHP.It uses a MySQL database to store all of your site content such as News, Torrents, Descriptions and much more. FastBeats is dedicated only for FastWeb users.
1.09 MB