(Originally published on Reddit, archived here in lightly edited form for posterity.)
A recent question led to a suggestion to use process substitution to emulate the effects of a proper shebang invocation, but the OP eventually discovered that 8 characters of padding had to be added to the substitution’s output to get it working.
Here’s why: SBCL seems to scan the first 8 characters in an input script file to determine what sort of file it is, then rewinds to the beginning and parses the file anew.
But pipes are not seekable, so with a process substitution, this process fails miserably: SBCL fails to rewind, so it starts parsing from the 9th character onwards, leading to very odd errors.
In addition, there’s one broad class of process substitutions that are guaranteed to fail when used: those that input/output ZIP/JAR archives, e.g. streamed via curl from a remote server. The problem stems from the ZIP file structure; since the main file header is at EOF, commands expect to seek backwards through an archive. That’s just not gonna happen on a pipe.
So the next time you think “hey, I can use a process substitution for this”, and find that your chosen command chokes on it, but works just fine on a file with the exact same contents, it’s almost certainly seeking in its input, or doing something else that works with files but not with pipes.
2 replies on “Process substitutions are pipes, NOT files”
What I imagine to be a useful edit to this excellent post.
[Note: “Process Substitution” is identified by the Bash syntax
<(LIST)or>(LIST). That LIST of commands is executed in a Bash subshell and the whole construct is treated like the name of a file.]I consider myself a proficient Bash programmer, but I would wager I am not the only person who reads process substitution and ponders, “Does that mean
$( COMMAND)or<(LIST). Or maybe it’s just me.Interesting point. When writing about bash, my first choice of phrasing is always based on the bash man page, so that anyone who’s confused can find a definitive exposition.
That said, I agree that process (
<(cmd)) may easily be confused with command ($(cmd)) substitution. I’ll ponder how best to clarify the difference, and edit my post accordingly.Thanks much for the feedback!