Talk:Pipeline (Unix)

From Wikipedia, the free encyclopedia
Jump to: navigation, search
WikiProject Computing  
WikiProject icon This article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
 ???  This article has not yet received a rating on the project's quality scale.
 ???  This article has not yet received a rating on the project's importance scale.
 
WikiProject Linux  
WikiProject icon This article is within the scope of WikiProject Linux, a collaborative effort to improve the coverage of Linux on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
 ???  This article has not yet received a rating on the project's quality scale.
 ???  This article has not yet received a rating on the project's importance scale.
 

Pipemill?[edit]

I have never, in 35 years of working with virtually every major variant of UNIX from 5th Edition onwards, encountered the term "pipemill" before. This section should be removed or reworded, or the etymology of the term should be provided. -- Resuna (talk) 19:23, 1 April 2014 (UTC)

removed suggestion that pipes were borrowed[edit]

I removed the suggestion that pipes in Unix were borred from MS-DOS - that history was backwards.

limit to pipelines?[edit]

Is there a limit to the number of commands you can pipe together depending on the shell? -- Joseph Lorenzo Hall 23:42, 22 February 2006 (UTC)

Joseph, wikipedia is not a tech support forum. Anyway the article is about the conceptual use of pipes in UNIX ... covering both the use of pipes in shells and their use inside lower level programming languages (like C). Any limits would be implementation details of the OS and/or the specific shell. For example the OS (version of UNIX) defines a maximum command line length ... which is the longest "vector" that can be passed to the execve() system call can accept. Any particular shell may implement a different maximum command line length which may be (and generally is) larger than the kernel's maximum execve() vector limit. That's because a command line entered to the shell may map into a number of separate execve() system call vectors --- and many commands within any command line may not result in any system calls (built-ins within the same shell process) or may involve system calls other than execve() (for example `echo $foo` ... often also supported as $(echo $foo) would involve a fork() system call --- there is a subshell --- but might not entail any execve() ... when echo is a built-in). 158.140.1.25 23:44, 9 May 2007 (UTC)
There tend to be lots of limits, and they do indeed vary dramatically in type and quantity from one environment to another. Another common limit is the number of simultaneous processes a user can own (a pipeline stage is usually at least one process). As noted, the specific limits aren't germane to this article, but some discussion of the common types of limits might be. RossPatterson 01:33, 10 May 2007 (UTC)

removed Mac os X in history[edit]

Mac os X is an unix so there is no point of telling it has borrowed the "pipe" concept, it was part of the package. Actually Linux is not Unix, so you may add Linux in this field

It would be interesting to add something about C/PM and to see if someone can talk about graphical pipe à la smalltalk


Everything is a File Link[edit]

There does not seem to be a page for the link "everything is a file" under the Network Pipes section.

Afaik. the pipe symbol used to be a caret ^ on some older systems instead of the vertical line. Maybe someone has more informations about that fact? The description of the "tee" command (for fitting together two pipes) sounds mighty wrong to me, it's more exactly the opposite (or my knowledge of the English language fails there) --Volker

I noted the bad tee(1) description (where did the author get that from?) and fixed it. JöG (talk) 10:32, 21 July 2009 (UTC)
There were several notations for pipes, or "hoses". It was around 4th Edition when '|' was settled on, so I don't think the earlier syntaxes were seen much outside the labs. [1] 70.225.161.132 (talk) 11:09, 14 April 2011 (UTC)
WWIW, I distinctly remember that on the early Sun workstations the University of Nijmegen were using in the mid-80s, which were running some BSD, both | and ^ were supported pipe symbols, ^ being the older, deprecated one. Not all terminals had a | symbol back then. Rp (talk) 22:41, 28 September 2011 (UTC)

Fair use rationale for Image:Automator Icon.png[edit]

Nuvola apps important.svg

Image:Automator Icon.png is being used on this article. I notice the image page specifies that the image is being used under fair use but there is no explanation or rationale as to why its use in Wikipedia articles constitutes fair use. In addition to the boilerplate fair use template, you must also write out on the image description page a specific explanation or rationale for why using this image in each article is consistent with fair use.

Please go to the image description page and edit it to include a fair use rationale.

If there is other other fair use media, consider checking that you have specified the fair use rationale on the other images used on this page. Note that any fair use images uploaded after 4 May, 2006, and lacking such an explanation will be deleted one week after they have been uploaded, as described on criteria for speedy deletion. If you have any questions please ask them at the Media copyright questions page. Thank you.BetacommandBot 21:56, 31 May 2007 (UTC)

pipes = awesome —Preceding unsigned comment added by ReluctantPhilosopher (talkcontribs) 09:35, 21 October 2007 (UTC)

A less(1) example[edit]

I added a simple ls | less example before the complicated spellcheck-an-URL example. Honestly, I found all of the existing examples fairly daunting – and I use complex pipelines daily. If anyone feels like adding examples showing off other aspects of pipelines, I suggest (a) pipelines involving infinite data streams like yes(1) or tail -f; (b) pipelines which parallelizes work (e.g. a CPU-bound data producer feeding into gzip); (c) pipelines across networks (e.g. foo | ssh somewhere bar -o baz). JöG (talk) 22:18, 18 March 2008 (UTC)

I agree a simpler example was desperately needed; it could be even simpler if the more common (excuse the pun) more was used instead of less—I agree less is more useful, but more is more widespread! Particularly for those used to a standard Unix environment rather than a Linux or GNU one. MeekMark (talk) 03:42, 19 March 2008 (UTC)

I chose less because I feel it's widely spread these days (people either work on free Unixes, or on systems where much of the free goodies are installed) and because it is more useful in pipelines (last time I used more, you could not scroll up when reading from stdin). And by the way, the article seems to have improved in general since late March. Thanks! JöG (talk) 19:42, 16 April 2008 (UTC)

Newlines in filenames[edit]

I would like to point out that the first example under Example of Pipemill will not work properly for any filename containing a newline ('\n'). I didn't want to alter the article due to the simplicity of the example, but this is worth mentioning for correctness. Use of find ... -print0 | xargs -0 ... handles any filename. DRA (talk) 18:44, 5 November 2008 (UTC)

Complex example fails[edit]

The complex example, which is very nice, does not work, because the file /usr/share/dict/words is not sorted properly (check "sort -c /usr/share/dict/words"). I don't know how to fix the example. —Preceding unsigned comment added by 94.208.225.204 (talk) 13:21, 16 January 2009 (UTC)

Add info about bash pipefail[edit]

Please add information about the bash pipefail, because it is important for this article, but my knowledge of English do not allow this. --Важнов Алексей Геннадьевич (talk) 04:31, 28 April 2011 (UTC)

"Pipeline"?[edit]

I have been using Unix for 25 years, and I have only seen the name "pipe", never "pipeline". What is the justification for this article being called "Pipeline"? Rp (talk) 07:41, 28 April 2011 (UTC)

Technically, A pipeline is part of the shell grammar, and the pipe itself is an (optional) operator that's part of it's syntax. This usage of the term is consistent amongst the documentation of most bourne-derived shells that I'm aware of, as well as POSIX. One error here is that the pipeline and syntax as described in the article is a feature of the command shell, not the operating system. Though most shells use pipe(2) to set up communication between the subshells, that isn't technically a requirement AFAICT. Honestly, this article is extremely bad and needs a complete rewrite. Unfortunately, I don't have time. See the spec: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_02 Ormaaj (talk) 12:18, 24 October 2011 (UTC)
I notice that the Bash shell's manpage confirms this usage. Rp (talk) 15:39, 31 October 2011 (UTC)

parallel[edit]

Hi. What about GNU Parallel ? Should be described her ? --Adam majewski (talk) 06:06, 25 February 2012 (UTC)