cmdfs for Linux 0.3

cmdfs is a FUSE-based virtual filesystem which applies an arbitary filter command to selected files in a source directory tree to create the destination files. Includes configurable caching and monitoring of the source tree to limit CPU use and prefetch data.<br /><br />Build<br /><br /> 1. Unpack archive<br /><br /> tar zxvf cmdfs-0.1<br /><br /> 2. Configure build<br /><br /> cd cmdfs-0.1<br /> ./configure<br /><br /> 3. Build<br /><br /> make<br /><br /> 4. Install<br /><br /> sudo make install<br /><br />Synposis<br /><br />cmdfs < source-dir > < mount-dir > [options]<br /><br />options<br /> -o extension=ext1[;ext2[;...]]<br /> Specify matching file extension(s) to apply command to<br /> <br /> -o path-re=< regular expression ><br /> Specify regexp of pathnames to apply command to<br /><br /> -o mime-re=< regular expression ><br /> Specify regexp of mime types to apply command to (as returned by file -b --mime-type)<br /><br /> -o link-thru<br /> Pass unmatched files through to filesystem as symbolic links [default: files not listed or accessible]<br /> <br /> -o monitor<br /> Watch source directory tree for new files and cache them as they appear. New directories will be monitored [default: not monitored]<br /> <br /> -o cache-dir=< dir ><br /> Directory to save cache files [default: /tmp/cmdfs-cache.< USER >]<br /> <br /> -o cache-size=< size in Mb ><br /> Size to attempt to limit cache directory to. Files removed on least-recent access basis.[default: no limit]<br /> <br /> -o cache-entries=< count ><br /> Number of entries to attempt to limit cache directory to. Files removed on least-recent access basis.[default: no limit]<br /><br />How It Works - An Example<br /><br />Given a source tree, which includes, say, jpg images, we can generate a view filesystem which contains the same files resized to email size.<br /><br />The filter command is set up as a fileystem option at mount time. In this case our filter option might be something like (using ImageMagick's convert command):<br /><br />command=convert - -resize 20% -<br /><br />Note we follow the UNIX convention of taking the standard input and filtering to standard output. Being a FUSE filesystem, the child process will be run under the group/user of the owner. The command is run within a shell (/bin/sh) process, as if started by /bin/sh -c ''<br /><br />Obviously there'll be files in the source tree which we dont want to apply the filter to. In this case we identify files of interest by extension:<br /><br />extension=jpg;gif;png<br /><br />To mount the filesystem as above for a particular user:<br /><br /> cmdfs /home/bob/images /mnt/myphotos "-ocommand=convert - -resize 20% -,extension=jpg;gif;png"<br /><br />Selection Filters<br /><br />As well as just matching on extension, files of interest can be selected by mime type, matching a regexp (cmdfs uses the file command to determine the mimetype):<br /><br />mime-re=image/*<br /><br />Or just a plain path regexp (this allows us to exclude whole directory subtrees etc<br /><br />path-re=.*/my-images/<br /><br />These options can be combined to get the required filter selection.<br /><br />Unmatched Files<br /><br />By default, files which don't match a filter won't appear in the filesystem view. We may want to see the original files though, so the<br /><br />link-thru<br /><br />option may be specified. This will generate symbolic links to the original files.<br /><br />Monitoring<br /><br />The image resize example is a good example of an application where we might want the files in the source filesystem to be filtered as they are added, rather than just on-demand. Cmdfs uses inotify to monitor all the directories in the tree, and automatically monitors new subdirectories as they are added:<br /><br />monitor<br /><br />When you copy the latest batch of files from your camera the resized versions will automatically be generated.<br /><br />Caching<br /><br />cmdfs only recreates view files if the modification date changes on the source. All files created using the filter are held in a cache directory, with a name created as a hash from the full path. By default this directory is located in /tmp/cmdfs-cache.< user name >, but can be changed:<br /><br />cache-dir=< dir ><br /><br />Note that some systems will clear the /tmp directory tree between reboots, which may or may not be desired.<br /><br />The size of the cache directory can be limited using:<br /><br />cache-size=< size in mb ><br /><br />and/or<br /><br />cache-entries=< number of files ><br /><br />The least recently accessed cached files will be removed in the background to maintain these limits.<br /><br />Mounting With fstab<br /><br />To aid in mounting directories for multiple users, cmdfs accepts the source base directory as the first argument so it can be mounted using the conventional mount.fuse script from fstab. This allows supplying cmdfs# as the fs_spec (see man 5 fstab). The user_allow_other option needs to be enabled in /etc/fuse.conf to allow normal users to mount (see fuse.conf). An example fstab entry:<br /><br />cmdfs#/media/myphotos /home/bob/images fuse user,allow_other,command=convert??N*?*40-??N*?*40-resize??N*?*4020%??N*?*40-,monitor,cache-size=500,extension=jpg;gif;png 0 0<br /><br />Note that the spaces in the command need to be escaped using ??N*?*40 for /etc/fstab to parse correctly.

Requirements:	No special requirements
Platforms:	*nix, Linux
Keyword:	Apply,  Cache,  Cmdfs,  Command,  Default,  Directories,  Directory,  Files,  Filesystem,  Filter,  Limit,  Mount,  Option,  Regexp,  Resize,  Source
Users rating:	0/10