hashedassets 0.3.1.1

Copies files to filenames based on their contents<br /><br />hashedassets is a command-line tool that copies files to filenames based on their contents. It also writes a map of what was renamed to what, so you can find your files.<br /><br />Main purpose of this is that you can add a far future Expires header to your components. Using hash based filenames is a lot better than using your $VCS revision number, because users only need to download files that didn't change.<br /><br />Creating some source files<br /><br />First, we create a file to be hashed:<br /><br />>>> system("mkdir input/")<br />>>> system("mkdir input/subdir/")<br />>>> system("mkdir maps/")<br />>>> with open("input/foo.txt", "w") as file:<br />... file.write("foo")<br /><br />>>> with open("input/subdir/bar.txt", "w") as file:<br />... file.write("bar")<br /><br />>>> system('touch -t200504072213.12 input/foo.txt')<br /><br />Simple usage<br /><br />>>> system("hashedassets maps/map.txt input/*.txt input/*/*.txt output/")<br />mkdir 'output'<br />cp 'input/foo.txt' 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />>>> print open("maps/map.txt").read()<br />subdir/bar.txt: Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt<br />foo.txt: C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt<br />< BLANKLINE ><br /><br />>>> system("ls output/")<br />C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt<br />Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt<br /><br />>>> system("ls maps/")<br />map.txt<br /><br />Logging level<br /><br />If we tell the command to be quiet, it does not print what it is doing:<br /><br />>>> system("hashedassets -q maps/map2.txt input/*.txt input/*/*.txt output/")<br /><br />If we tell the command to be more verbose, it logs more information:<br /><br />>>> system("hashedassets -vvv maps/map3.txt input/*.txt input/*/*.txt output/")<br />Debug level set to 10<br />cp 'input/foo.txt' 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />Modification time is also preserved:<br /><br />>>> old_stat = os.stat("input/foo.txt")<br />>>> new_stat = os.stat("output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt")<br />>>> [(getattr(old_stat, prop) == getattr(new_stat, prop))<br />... for prop in ('st_mtime', 'st_atime', 'st_ino',)]<br />[True, True, False]<br /><br />We can easily do this with multiple formats:<br /><br />javascript<br /><br />>>> system("hashedassets -n my_callback maps/map.js input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />>>> print open("maps/map.js").read()<br />var my_callback = {<br /> "foo.txt": "C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt",<br /> "subdir/bar.txt": "Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt"<br />};<br /><br />JSON<br /><br />>>> system("hashedassets -n my_callback maps/map.json input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />>>> print open("maps/map.json").read()<br />{<br /> "foo.txt": "C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt",<br /> "subdir/bar.txt": "Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt"<br />}<br /><br />JSONP<br /><br />>>> system("hashedassets -n my_callback maps/map.jsonp input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />>>> print open("maps/map.jsonp").read()<br />my_callback({<br /> "foo.txt": "C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt",<br /> "subdir/bar.txt": "Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt"<br />});<br /><br />Sass<br /><br />Sass is a meta language on top of CSS.<br /><br />>>> system("hashedassets -n my_callback maps/map.scss input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />>>> print open("maps/map.scss").read()<br />@mixin my_callback($directive, $path) {<br /> @if $path == "subdir/bar.txt" { #{$directive}: url("Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt"); }<br /> @else if $path == "foo.txt" { #{$directive}: url("C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt"); }<br /> @else {<br /> @warn "Did not find "#{$path}" in list of assets";<br /> #{$directive}: url($path);<br /> }<br />}<br /><br />PHP<br /><br />>>> system("hashedassets -n my_callback maps/map.php input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />>>> print open("maps/map.php").read()<br />$my_callback = array(<br /> "subdir/bar.txt" = > "Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt",<br /> "foo.txt" => "C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt",<br />)<br /><br />Sed<br /><br />We can also generate a sed script that does the replacements for us:<br /><br />>>> system("hashedassets -n my_callback maps/map.sed input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />>>> print open("maps/map.sed").read()<br />s/subdir/bar.txt/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt/g<br />s/foo.txt/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt/g<br />< BLANKLINE ><br /><br />We should also be able to use this directly with sed<br /><br />>>> with open("replaceme.html", "w") as file:<br />... file.write('< a href=foo.txt >bar< /a >')<br /><br />The script is then applied like this:<br /><br />>>> system("sed -f maps/map.sed replaceme.html")<br />< a href=C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt >bar< /a ><br /><br />However, '.' is not treated as wildcard, so the following does not work<br /><br />>>> with open("replaceme2.html", "w") as file:<br />... file.write('< a href=fooAtxt >bar< /a >')<br /><br />>>> system("sed -f maps/map.sed replaceme2.html")<br />< a href=fooAtxt >bar< /a ><br /><br />Specifying the type with -t<br /><br />The type of the map is guessed from the filename, but you can specify it as well:<br /><br />>>> system("hashedassets -t js cantguessmaptype input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />Specifying the length of the filename with -l<br /><br />>>> system("hashedassets -l 10 maps/shortmap.json input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/C-7Hteo_D9.txt'<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IO.txt'<br /><br />>>> system("rm output/C-7Hteo_D9.txt output/Ys23Ag_5IO.txt")<br /><br />Specifying the digest with -d<br /><br />>>> system("hashedassets -d md5 maps/shortmap.json input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/rL0Y20zC-Fzt72VPzMSk2A.txt'<br />cp 'input/subdir/bar.txt' 'output/N7UdGUp1E-RbVvZSTy1R8g.txt'<br /><br />>>> system("rm output/rL0Y20zC-Fzt72VPzMSk2A.txt output/N7UdGUp1E-RbVvZSTy1R8g.txt")<br /><br />Re-using a map<br /><br />The program reads in maps it created in a prior run to only copy files that haven't changed since. So, the following commands do not copy any files:<br /><br />>>> system("hashedassets maps/map.scss input/*.txt input/*/*.txt output/")<br />>>> system("hashedassets maps/map.php input/*.txt input/*/*.txt output/")<br />>>> system("hashedassets maps/map.js input/*.txt input/*/*.txt output/")<br />>>> system("hashedassets maps/map.json input/*.txt input/*/*.txt output/")<br />>>> system("hashedassets maps/map.sed input/*.txt input/*/*.txt output/")<br />>>> system("hashedassets maps/map.jsonp input/*.txt input/*/*.txt output/")<br />>>> system("hashedassets maps/map.txt input/*.txt input/*/*.txt output/")<br /><br />If we touch one of the input files in between, the file will be read but not copied because the hashsum is the same:<br /><br />>>> system('touch -t200504072214.12 input/foo.txt')<br />>>> system("hashedassets maps/map.json input/*.txt input/*/*.txt output/")<br /><br />If we change the file's content, it will get a new name:<br /><br />>>> with open("input/foo.txt", "w") as file:<br />... file.write("foofoo")<br /><br />>>> system("hashedassets maps/map.json input/*.txt input/*/*.txt output/")<br />rm 'output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'<br />cp 'input/foo.txt' 'output/QIDaFD7KLKQh0l5O6b8exdew3b0.txt'<br /><br />If you then list the files in the directory, note that the old file ''output/C-7Hteo_D9vJXQ3UfzxbwnXaijM.txt'' is gone:<br /><br />>>> system("ls output/")<br />QIDaFD7KLKQh0l5O6b8exdew3b0.txt<br />Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt<br /><br />If we remove one of the created files, it gets recreated:<br /><br />>>> system("rm output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt")<br />>>> system("hashedassets maps/map.json input/*.txt input/*/*.txt output/")<br />cp 'input/subdir/bar.txt' 'output/Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt'<br /><br />>>> system("ls output/")<br />QIDaFD7KLKQh0l5O6b8exdew3b0.txt<br />Ys23Ag_5IOWqZCw9QGaVDdHwH00.txt<br /><br />If a file that is about to be removed because the original content changed, it isn't recreated:<br /><br />>>> system("rm output/QIDaFD7KLKQh0l5O6b8exdew3b0.txt")<br />>>> with open("input/foo.txt", "w") as file:<br />... file.write("foofoofoo")<br />>>> system("hashedassets maps/map.json input/*.txt input/*/*.txt output/")<br />cp 'input/foo.txt' 'output/NdbmnXyjdY2paFzlDw9aJzCKH9w.txt'<br /><br />Error handling<br /><br />However, if we run this with no arguments, it fails:<br /><br />>>> system("hashedassets", external=True)<br />Usage: hashedassets [ options ] MAPFILE SOURCE [...] DEST<br />< BLANKLINE ><br />hashedassets: error: You need to specify at least MAPFILE SOURCE and DEST

Requirements:	No special requirements
Platforms:	*nix, Linux
Keyword:	Callback,  Files,  Inputfootxt,  Inputsubdirbartxt,  Inputtxt,  Mapsmapjson,  Outputc Hteo Vjxq Ufzxbwnxaijmtxt,  Outputquotcp,  Outputquotgtgtgt,  Outputys Iowqzcw Qgavddhwh Txt Gtgtgt,  Print,  Quotwquot,  Systemquothashedassets
Users rating:	0/10