DOCLIB Beta Synchronized Release Procedures
Last modified: 3/26/2007
Author : Timothy M. Meyers
Kevin Chen
Overview
DOCLIB is a C++ toolkit that provides document/image analysis capabilities through a thoroughly documented, easy to use interface. This library seeks to provide a functional, stable, and robust environment that supports a plethora of image types whereby applications (research or production) can be built on top of.
Procedures have been identified and documented to assure consistency in
delivery and behavior across different versions of DOCLIB. Use this document to
expedite release proceedings for DOCLIB.
Release Notes
DOCLIB development occurs simultaneously in both Unclassified and Classified environments. Having two separate development environments can and does lead to changes on both sides that must be merged before a new release. This procedure is a difficult and complicated task that is outlined below.
Additionally, the classified network now uses Subversion instead of CVS for content management.. In the future, we would ideally like to move the unclassified repository to Subversion as well and remove all references to CVS in this document.
In the examples below, where applicable, the CVS example is predicated with a ‘$’ symbol representing the prompt, whereas the SVN example is predicated with a ‘#’ prompt.
Classified
Environment: Pre Sync
1) Create
a directory for preparation. “~/syncRelease” works well
2) Check out the Latest Version of
Doclib to syncRelease
$: cvs
co –d doclib_current doclib
#: svn checkout $svnurl/DOCLIB/DOCLIB_<version>/trunk
<syncRelease>/doclib_current
3) Validate performance and behavior
i) Verify successful compilation under Redhat linux 9.0.
a) Run regression test and make sure all test cases pass. Rectify any failures within the regression test.
ii) Verify successful compilation under SunOS
a) Run regression test and make sure all test cases pass. Rectify any failures within the regression test.
4) Tag DOCLIB and Add-ons within CVS/SVN
i) Tag versions of the DOCLIB module (contains add-ons) within CVS/SVN using standard naming convention (DOCLIB_BETA_GOV_X_XX_PRESYNC) where X_XX is the version number
$: cvs
tag DOCLIB_BETA_GOV_X_XX_PRESYNC
#: svn copy –m “DOCLIB Beta Gov X_XX Presync” $svnurl/DOCLIB/DOCLIB_<version>/trunk $svnurl/DOCLIB/DOCLIB_<version>/tags/pre-sync
5) Check out the Latest Release of
Doclib to syncRelease
$: cvs
co –d DOCLIB_BETA_X_XX –r DOCLIB_BETA_X_XX
doclib
#: svn checkout $svnurl/DOCLIB/DOCLIB_<previous-version>/trunk
<syncRelease>/DOCLIB_<previous-version>
If there is not a tagged last release, do a ‘cvs log’ or ‘svn list $svnurl/DOCLIB/DOCLIB_<version>/tags’
on any of the files in the module and there should be a listing of tags. The last on the list should be the imported
release. If there is not a list of tags,
then attempt to check out based the date (using the “–d DATE” option in place
of the “-r REVISION” option) of the last import.
6)
Perform
a CVS DIFF (or regular DIFF) on the two modules
From the ‘syncRelease’
directory
$: cvs
diff –r DOCLIB_BETA_GOV_X_XX_PRESYNC –r DOCLIB_BETA_Y_YY
#: diff DOCLIB_<previous-version> doclib_current
where X_XX is the version you will be releasing
and Y_YY is the previous released version. Again, if there is no tag for the previous
version of DOCLIB, use the “–d DATE” option.
You may want to redirect the output of CVS DIFF to a file for future revision
$: cvs
diff –r DOCLIB_BETA_GOV_X_XX_PRESYNC –r DOCLIB_BETA_Y_YY > diff.out
#: diff DOCLIB_<previous-version> doclib_current
> diff.out
7)
Decision
Time
This is where the process gets tricky. You must now decide if the DIFF is
substantial enough to warrant an export of the files that are different (i.e.
how much code are you willing to retype).
If the DIFF is large, gather up the files that are different together in a
single folder and begin the NTA export process (this
has never been done for a DOCLIB Release.
Usually the DIFF is small, e.g.
less than 200 lines).
If the DIFF is small:
i) Review the different files line by line for Classified content (GREP works nicely if you know what you are looking for).
a) Replace or remove content based on necessity
ii) Print all source code from files that are different (a2ps saves on paper) after verifying there is no sensitive information in them.
iii) Print a copy of the DIFF file after verifying there is no sensitive information in it.
8) Bring all changed source code and copy of the DIFF file to the Unclassified
side.
Unclassified
Environment: Sync Process
After the Classified changes are brought out, they need to be merged with the current Unclassified version of DOCLIB
1) Validate performance and behavior
i) Verify successfully compilation under Redhat linux 9.0.
ii) Run
regression test and make sure all test cases pass. Rectify any failures within
the regression test.
2) Tag DOCLIB and Add-ons within CVS/SVN
Tag versions of the DOCLIB module (and any Add-ons that will be synched) within
CVS/SVN using standard naming convention (DOCLIB_BETA_X_XX_PRESYNC)
where X_XX is the version number
The process of synching the Classified and Unclassified environments can be time-consuming. There are programs available to assist in merging source code if the Classified code was exported. If hard copies (paper) of the Classified code was exported, then the changes will need to be re-implemented on the Unclassified side line-by-line.
Unclassified
Environment: Release Process
After the synchronizing process, DOCLIB is ready to be released.
1) Validate performance and behavior
i) Verify successfully compilation under Redhat linux 9.0.
ii) Run
regression test and make sure all test cases pass. Rectify any failures within
the regression test.
2) Review code written by foreign nationals
i) All code written by foreign nationals that are being imported into Government spaces are subject to review and approval.
ii) Download and install CSDiff freeware package from http://www.download.com
iii) Download prior and most recent release onto directory structure
iv) Using CSDiff tool, compare and report all differences to html files.
v) Post
all differences onto the mediaproc server at
/var/www/html/doclibDeveloper/doclib_screen
vi) Assimilate review and approval of all new and/or modified files written by non-Government related folks (Look at log files at end of each file for authorship)
3) Update Documentation
i) All DOCLIB-related documents should be modified to reflect features supported within each release. Experience has shown that users of the doclib software packages perceive value and/or decide whether or not to upgrade based upon the information contained within these documents.
ii) Update doxyfile and run doxygen against all src/include directories
iii) Capture and fix all doxygen warning and error messages to ensure robust documentation output
iv) Update the DOCLIBCommandLine document with any new features added to the standalone interface.
v) Update the FeatureList document to include the latest version number and any new macro-level functions added (use the bugtracker tool for assistance)
vi) Update LIBTIFF_BUILD.txt, readme,txt, testcases.txt, and execsummary.txt to reflect the latest version number
vii)Update src and documentation within each doclib add-on (e.g.: Logo Detect, Mach Hand detection)
4) Tag DOCLIB and Add-ons within CVS/SVN
i) OPTIONAL: Before tagging the release, if
code is particular messy, one may run the source through the emacs auto-format script provided in the ReleaseDocs directory.
$: autoformat.sh
X.cpp
$: autoformat.sh X.h
ii) Tag
versions of DOCLIB and add-ons within CVS using standard naming convention
(DOCLIB_BETA_0-10, LOGODETECT_BETA_0-08, etc).
From module root:
$: cvs tag “TAG_BETA_X-XX”
‘User’ and ‘developer’ versions of DOCLIB are created and maintained for each release. The ‘user’ release contains documentation and libraries for the DOCLIB library. Additionally, the users release contains built binaries. Alternatively, the ‘developer’ release contains libraries, documentation, and source code DOCLIB and supported add-ons. The steps to create each release are described below.
5) DOCLIB package – Developer’s Release
i) Check out tagged version of Doclib and add-ons under Linux in ‘Export’ mode
ii)
Remove the CVS logs from all source code using
the doclibStrip java program.
The process will resemble the following:
$: cd
~/DoclibStrip/classes/
$: java doclibstrip/DoclibStrip <source
directory>
iii) Remove the “resource” directory from root of the doclib directory
iv) Individually tar the doclib and add-ons within a separate directory
v) Copy the files located within the ../doclib/ReleaseDocs/ files into this directory
vi) Copy the tar files and the release docs to the /home/shared/doclibRelease/DOCLIB_BETA_X-XX
vii) Update the Developer’s corner website with the new version of doclib
viii) Burn CD with new version of doclib (it is best to burn Linux formatted tar files as opposed to Windows formatted zips).
6) DOCLIB package – User’s Release
i) Check out tagged version of Doclib under windows and Linux
ii) Build Doclib and the DOCLIBCommandLine under windows and Linux
iii) From the windows version of Doclib, remove
a) ..\doclib\windows\bin\*.ilk
b) ..\doclib\windows\bin\DoclibRegressionTesting.exe
c) ..\doclib\windows\bin\DoclibProtypeCommandLine.exe
d) ..\ doclib\windows\projectfiles\*.* (all files and sub directories)
e) ..\doclib\data (including all sub directories)
f) ..\doclib\ReleaseDocs\LIBTIFF_BUILD.txt, readme.txt, testcases.txt, make.pl, params, and “DOCLIB Beta Release Procedures.doc”
g) ..\doclib\resource (all files and sub directories)
h) ..\doclib\scripts (all files and sub directories)
i) ..\doclib\*.* (all files, leave sub directories)
j) ..\doclib\src\*.* (all files, leave sub directories)
iv) From the Linux build, copy the “linux24” directory to the Windows build, under “doclib” (Only copy the “DoclibCommandLine” file in the “bin” directory.
v) Zip the doclib directory with the name new version of doclib. doclib_beta_X.XX_lib.zip
vi) Make sure the dependencies are up-to-date. Take the dependencies from the last release and add/delete doclib dependencies.
vii)Copy the dependencies and doclib to Mediaproc: /home/shared/doclibRelease/DOCLIB_BETA_X-XX
viii) Make a new directory for the new release of Doclib in /var/www/html/documents/doclib/release/
ix) Copy these files to the directory just created
a) doclib_beta_X.XX_lib.zip
b) doclib_Dependencies_linux.tar
c) doclib_Dependencies_windows.zip
d) DoclibCommandLine.txt
e) FeatureList.txt
f) LIBTIFF_BUILD.txt
g) readme.txt
h) testcases.txt
x) Add a new page in the Doclib user’s corner pointing to the new version of DOCLIB.
Classified Environment:
Release Process (TBD)
The Classified Environment will receive the Developers Release CD-ROM. Put in an import ticket for the CD-ROM contents. It is probably best to have them placed in a directory in your home directory. In the past, ~/new_release/ has been used. After the CD-ROM contents have been imported
1) Unzip and untar all files and add-ons
i) Untar the main tarball
$: tar xvvf
DOCLIB_PLUS_ADDONS_BETA-X.XX.tar
ii) Untar the Doclib module and each of the Add-on modules
2) Update the params
file
(use current Classified version as a template, or simply merge both params files).
Keep in mind that the Government Proprietary Image Format Add-on Module will
need to be integrated. See relevant Readmes in Classified Doclib module root directory.
3) Import DOCLIB and all Add-Ons to CVS/SVN Repository, and update Repository links
i) Import
DOCLIB and each Add-On
$: cvs
import –m “Initial Import” DOCLIB_BETA_X-XX bah-r61
initial-import
#: svn import –m “Initial Import” .
$svnurl/DOCLIB/DOCLIB_<new-version>trunk
ii) Update
CVS links if using CVS – Example
$:
cd $CVSROOT
$: rm doclib (this should only be a symbolic link!)
$: ln –s DOCLIB_BETA_X-XX
doclib (creates
new symbolic link)
4) Validate performance and behavior
i) Verify successfully compilation under Redhat linux 9.0
a) Run regression test and make sure all test cases pass.
b) Test that the Gov’t Prop Image format can be loaded and saved
ii) Verify successful compilation under SunOS
a) Run regression test and make sure all test cases pass.
b) Test
that the Gov’t Prop Image format can be loaded and
saved
5) Install Doclib Module
i) Do not install DLPrototypeCommandLine or DLRegressionTest
ii) OPTIONAL: Install two different versions of doclib library for users. (This has never been done before)
a) libDoclib.a – standard DOCLIB library
b) libDoclib<X>.a
– include support for Client proprietary format.
6) Announce Release to R61