Software Pipeline
In software engineering, a pipeline consists of a chain of processing elements (processes, threads, coroutines, functions, etc.), arranged so that the output of each element is the input of the next; the name is by analogy to a physical pipeline. Usually some amount of buffering is provided between consecutive elements. The information that flows in these pipelines is often a stream of records, bytes, or bits, and the elements of a pipeline may be called filters; this is also called the pipes and filters design pattern. Connecting elements into a pipeline is analogous to function composition.
Narrowly speaking, a pipeline is linear and one-directional, though sometimes the term is applied to more general flows. For example, a primarily one-directional pipeline may have some communication in the other direction, known as a return channel or backchannel, as in the lexer hack, or a pipeline may be fully bi-directional. Flows with one-directional tree and directed acyclic graph topologies behave similarly to (linear) pipelines – the lack of cycles makes them simple – and thus may be loosely referred to as “pipelines”.
UNIX Pipeline (Pipes):
In Unix-like computer operating systems, a pipeline is a mechanism for inter-process communication using message passing. A pipeline is a set of processes chained together by their standard streams, so that the output text of each process (stdout) is passed directly as input (stdin) to the next one. The second process is started as the first process is still executing, and they are executed concurrently. The concept of pipelines was championed by Douglas McIlroy at Unix’s ancestral home of Bell Labs, during the development of Unix, shaping its toolbox philosophy. It is named by analogy to a physical pipeline. A key feature of these pipelines is their “hiding of internals” (Ritchie & Thompson, 1974). This in turn allows for more clarity and simplicity in the system.
This article is about anonymous pipes, where data written by one process is buffered by the operating system until it is read by the next process, and this uni-directional channel disappears when the processes are completed. This differs from named pipes, where messages are passed to or from a pipe that is named by making it a file, and remains after the processes are completed. The standard shell syntax for anonymous pipes is to list multiple commands, separated by vertical bars (“pipes” in common Unix verbiage):
Flow diagram view:
VisualSim can be used to construct software pipelines using SystemResource blocks; where priority and preemption can be factored into the shared resource.
In VisualSim, data structures represent the common file concept in UNIX pipes; time to execute the task or thread is typically a field of the data structure, which represents N cycles of execution. VisualSim can use a relative time, assuming exact cycles are unknown; or exact cycles at a speed in MHz, once detailed cycle information is known about the task or thread. Buffering between tasks or threads might be added as queue blocks. The UNIX message equivalent is the VisualSim data structure. Finally, VisualSim SystemResources can also represent hardware accelerators with proper assumptions.
Web Reference 1: https://en.wikipedia.org/wiki/Pipeline_(software)
Web Reference 2: https://en.wikipedia.org/wiki/Pipeline_(Unix)