A contemporary sourcecode site consists of a very large number of files, some of which are text files and some of which have binary formats. It is the mandate of this system that it be able to process an existing sourcecode site and to then transform it into a new sourcecode site with different files, different structure, and even with different internal file format. In addition, this system itself has its own sourcecode site which must be organized and often reconfigured, so that it can be moved to new sites. The term "Deployment" is used to refer to the various operations required to gather large numbers of files into a few bundles and then to redistribute those bundles in a new location.
Though there are many binary compaction formats that could be used to gather files into a single file, this system uses a simple text-based format that is compatible with the UNIX shell system. The details of this format will be discussed later. The advantages of using a text-based format are two-fold. First, different operating systems use different internal file formats for text files, which form the bulk of the files in the site. This system knows these formats and can both recognize and produce them as it performs the various deployment operations. Binary
zip formats cannot change the formats as they gather and then distribute files; thus, the user is required to use a separate process to do the text-line format conversions. Second, the authoring facilities of this system produce outputs which contain many files. It makes this operation easier to manage if the authors can produce single files using this text-based format that can then be deployed once they have been produced and accepted.
This page describes how to start the Deployment tool on the command line. The syntax of the command line is as follows
Deploy filename[.bnd | .dir] [switches]
The actual task performed by the tool depends upon the filename extension. The following different file types are recognized:
Ext | Description of task |
bnd | The specified file contains a bundled multi-file output of the tool. At this point, the files in the bundle are unbundled and deployed to their final target destinations. |
dir | The specified file contains a list of filenames that are to be bundled into a single file for easy backup, later redeployment, or transfer to another environment. |
The following command line switches are recognized:
Switch | Description |
---|
ES | Echoes the input to the tool as it reads it |
UNIX | Requests that text files use UNIX linefeed conventions |
QUIET | Requests that no process messages be displayed |
VERBOSE | Requests that progress messages be displayed |
REPLACE | Requests that existing files be replaced |
DEPLOY | Requests that file management commands be executed |
NOHEADER | Requests that no header be displayed |
FULL | Retain full file path when bundling a list.dir file |
Xname | Specifies an extraction location |
Mname | Specifies the name of the file to be made |
Pname | Specifies an input filename prefix |
To better understand the need for this deployment capability and the basic
bundling and
unbundling operations that it performs, the following is a simple example. This work is being performed in a Microsoft command window. Keeping things simple, the editor being used is
notepad. A user of this system has submitted a zip-file that was produced on a Linux machine that contains a large number of individual source files along with a copy of the script used to do a build and the log of that build. That log, called
buildads.log, shows errors which need to be corrected.
After unzipping the files, there are 182 of them, the first step is to use notepad to look at buildads.log to see what the problems are. Unfortunately, notepad reports that this file contains 15 lines, each 1022 characters long that contain a large number of undisplayable characters. In fact, all of the files show this sort of thing. Though some Microsoft tools can work with them, most cannot. The problem of course is that Linux uses simple line-feeds to mark the end of text-lines; while Microsoft expects carriage-return, line-feed pairs at the end of each line (except the last one sometimes -- really!). Before this set of files can be used, they must be converted into Microsoft form. This can be done by bundling them and then unbundling them using the deployment capability. Of course, if the user had used this tool to do the bundling in the first place, then this second step would not be necessary. The tools were not available on the Linux machine, which is often the case.
To bundle a set of files, a list of those files is needed in a separate file whose extension is
dir. The easiest way to do this in a Microsoft command window is with the
dir command. For example
Code Block |
---|
language | none |
---|
theme | Eclipse |
---|
linenumbers | true |
---|
|
> dir/b *.log,*.sh
buildads.log
buildads.sh
|
the above is a "brief" listing that produces the names of the .log and .sh files in simple form -- without additional size, type, or date information. Similar commands exist on all operating systems. Saying instead
Code Block |
---|
language | none |
---|
theme | Eclipse |
---|
linenumbers | true |
---|
|
> dir/b *.log,*.sh >test.dir
|
produces a file test.dir which contains the names of the two files. Obviously the same can be done for large numbers of files. Once the .dir file exists, the simple command
Code Block |
---|
language | none |
---|
theme | Eclipse |
---|
linenumbers | true |
---|
|
> Deploy test.dir
|
produces a text file bundle
test.bnd that contains the two files. There are two very important points. First, the deployment logic automatically recognized that the input files were in UNIX format and converted them as it read them. There was no need to tell the tool to do this. It always converts to the format of the platform that it is on when it reads a file to be bundled. Second, the bundled file is now a simple text file in the local Microsoft format and can be viewed/edited with notepad. It looks generally as follows, showing the start of each file
Code Block |
---|
language | none |
---|
theme | Eclipse |
---|
linenumbers | true |
---|
|
cat >buildads.log <<'!)(!'
mkdir -p ./lnx
echo Making imslib.a ...
Making imslib.a ...
cd imslib
rm -r -f *.o
cc -c -DAN6PLAT -I./ -I../include *.c
rdqlocal.c: In function `RdqPushSymbol':
.....
!)(!
cat >buildads.sh <<'!)(!'
mkdir -p ./lnx
echo Making imslib.a ...
cd imslib
.....
!)(!
|
Note that each file is introduced by a
cat command and is terminated by a
!)(! marker. The tool understands these commands but so do the various shell programs originally developed for UNIX, but now available for all platforms. This deployment capability is NOT needed to deploy these bundled files.
The bundled file could now be worked with directly, but to show the entire process, the tool can now be told to unbundle the
test.bnd file
Code Block |
---|
language | none |
---|
theme | Eclipse |
---|
linenumbers | true |
---|
|
> Deploy test.bnd verbose
deploy V9.82 (BETA.001) System Build(12/08/09 16:05:15)
The file already exists: buildads.log
The file already exists: buildads.sh
|
The
bnd extension tells the tool that an unbundling operation is to be performed. One of the problems with the deployment process is that it is easy to unintentionally overwrite existing files. The tool will not overwrite an existing file unless it is explicitly told to do so, using the REPLACE option,
Code Block |
---|
language | none |
---|
theme | Eclipse |
---|
linenumbers | true |
---|
|
> Deploy test.bnd verbose REPLACE
deploy V9.82 (BETA.001) System Build(12/08/09 16:05:15)
Unbundling: buildads.log
Unbundling: buildads.sh
|
Now, the files that were originally in UNIX format have been replaced by files that are in native format and can be worked on with all the native tools.
For deployment on UNIX platforms, text files must use line-feed (LF) end-of-line conventions. If you use the UNIX command-line option, the deploy utility will do this automatically on the Windows text files listed in
test.dir
Code Block |
---|
language | none |
---|
theme | Eclipse |
---|
linenumbers | true |
---|
|
> deploy test.dir UNIX VERBOSE
deploy V9.82 (BETA.001) System Build(12/08/09 16:05:15)
Processing: buildads.log
Processing: buildads.sh
|
The
test.bnd bundle file produced now uses UNIX end-of-line conventions, and it can be shelled directly on a UNIX Linux machine, using for example the UNIX command sh
Code Block |
---|
language | none |
---|
theme | Eclipse |
---|
linenumbers | true |
---|
|
> sh test.bnd
|
On Windows, though it is now in UNIX format, test.bnd will unbundle into files that use Windows end-of-line format conventions
Code Block |
---|
language | none |
---|
theme | Eclipse |
---|
linenumbers | true |
---|
|
> deploy test.bnd verbose REPLACE
deploy V9.82 (BETA.001) System Build(12/08/09 16:05:15)
Unbundling: buildads.log
Unbundling: buildads.sh
|