Pipelines is a utility that allows you to change the contents of a text file or files, quickly and easily. You can specify that only certain sections of a file are to be changed; you can confine those changes to a column, word or field range, translate words and phrases, discard or insert new lines of text. You can perform a whole range of operations on a file or files, using only a simple set of commands.
Pipelines build's on the concept of directing the output of one process to the input of another; commonly known as pipelining.
However, Pipelines takes an extra step; allowing you to build multi-stream pipelines, where the topology is no longer horizontal and linear, but two-dimensional; where the records travel up and down the pipeline chain through intersecting joints which control the flow of data.
With standard linear pipelines the data flows through each filter or stage, passing into the next and so on until it reaches a sink. Multi-stream pipelines on the other hand allow you to select and operate on specific sets of records; routing unselected records through a joint into and out of other sections of the pipeline. This allows you to join multiple pipelines together in configurations that address a whole range of transformation problems.
Pipelines comprises 42 input, output, selection and transformation stages which cover a broad range of manipulation functions; splitting records, stripping characters, joining records, collating and sorting and more.
On the whole, similar operations are performed by a single stage; which means that you do not have to remember the names of an unnecessarily lengthy list of stages. For example; stripping characters from a record, Pipelines provides a single stage called STRIP which removes characters from the beginning and/or the end of a record.
The Pipelines syntax is very simple; it does not employ lists of terse /switches, but rather, an English-like syntax which is straight forward to read.
With Pipelines, the pipeline can be specified on the system command-line, in a batch file or in a Pipelines file, ext (.PPL). You design the pipeline in your favourite editor and save it; to execute the pipeline you simply double click the file icon and Pipelines will launch it.
You can specify pipelines which accept arguments which substitute stage operands and even stage names and coupled with the capability to connect pipelines together this allows you build a range of utility pipelines that can be called upon whenever you need them.
You may find Pipelines of use in many cases where you might otherwise have to write a program to solve the problem and it may well save you some time and effort that could be better spent on other tasks.
Here are some key features of "Pipelines":· Multiple Pipelines instances may execute concurrently.
· Pipelines dispatches the stages in the order in which they appear in the pipeline, however; any stage may be the first to begin processing records.
· The relative order of the records flowing through a pipeline can be predicted; as long as the stage path only comprises stages' that do not delay the records.
· Unless the pipeline comprises a stage or stages' that accumulate records; for example the SORT stage, and, that the input records are not excessively long, Pipelines requires only a small amount of memory to process input files of any size; as only a handful of records will be in the pipeline at any one time.
· Pipelines is not pre-emptive. When a stage reports an initialisation or runtime error; Pipelines begins terminating the pipeline by instructing all active stages to quiesce. When all active stages in the pipeline chain have responded to the quiesce command and have terminated; Pipelines terminates.
· Pipelines does not verify that a pipeline is semantically correct, only that it is syntactically correct. This means that you may construct a pipeline that does not execute in the way that you expect it to. It may produce output records in a format or an order that you did not intend or it may not produce any output records at all. In view of this; when developing a pipeline that replaces the contents of a disk file, it is particularly prudent to test the pipeline against a copy of that file. Pipelines does not issue "are you sure?" messages!
· Pipelines does not work with records containing MBCS or UNCODE data, only the single-byte ASCII character set is supported. As a consequence, you should ensure that only ASCII type input files are selected for modification. Pipelines cannot determine the format of an input file; it simply executes the pipeline that you specify.
· Pipelines comprises a stall detection mechanism that determines when a pipeline is stalled; A stall occurs when Pipelines determines that every stage is either waiting to read a record or write a record. That is, there is no stage that is currently processing a record; all stages are either read-pending or write-pending. Pipelines writes the current status of each stage in the pipeline to a dump-file which can be inspected to determine the combination of stream connections that caused the stall.
· When a stage does not specifically limit the number of input and/or output streams, the stage may process up to 4096 input streams and the unsigned integer value MAX_INT output streams.
· The pipeline is not interpreted, Pipelines performs a single-pass parse of the pipeline; allocating the resources required by each stage and then it begins dispatching them.
· A pipeline can be specified as an element in a system command-line or a batch-file pipeline, for example; the following pipeline sorts its input data on three key fields (1-20, 30-40 and 80-100); 30-40 is the sub-sort field of 1-20 and 80-100 is the sub-sort field of 30-40.
Requirements:· NET Framework 2.0
What's New in This Release
· This version of Pipelines is currently only available for the .NET 2.0 platform.
· A Pipelines Visual Studio/VC++ API mechanism (phase 1: VS8 - VC++ 9).
· The API provides all the initialisation and runtime routines that support the current builtin stage set; comprising stage command parsing, runtime extraction and stream connection routines and a stage DLL project wizard which creates a fully functional skeletal Pipelines stage DLL.
· In order to keep the Pipelines download package to a reasonable and manageable size and at the same time provide me with the flexibility to publish Pipelines features which may have already been developed but not yet documented, as and when they are deemed fit for use; the Pipelines documentation is now available online and no longer provided with the download package.
· FILELIST stage command: the file date and time values have been changed to represent ‘local-time’; this brings the field’s in line with the DOS DIR and Windows file date and time property...
|
Developer:
License / Price:
Size / OS:
Last Updated:
| TenFiftyTwo
GPL / FREE
19.5 MB / Windows NT / XP
August 19th, 2010 |
