DOCLIB Beta Synchronized Release Procedures

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Last modified:          3/26/2007

Author :                     Timothy M. Meyers

                                    Kevin Chen

                                    Summit Sampat

           

 

 

Overview

 

DOCLIB is a C++ toolkit that provides document/image analysis capabilities through a thoroughly documented, easy to use interface. This library seeks to provide a functional, stable, and robust environment that supports a plethora of image types whereby applications (research or production) can be built on top of.


Procedures have been identified and documented to assure consistency in delivery and behavior across different versions of DOCLIB. Use this document to expedite release proceedings for DOCLIB.

 

Release Notes

 

DOCLIB development occurs simultaneously in both Unclassified and Classified environments.  Having two separate development environments can and does lead to changes on both sides that must be merged before a new release.  This procedure is a difficult and complicated task that is outlined below.

 

Additionally, the classified network now uses Subversion instead of CVS for content management..  In the future, we would ideally like to move the unclassified repository to Subversion as well and remove all references to CVS in this document.

 

In the examples below, where applicable, the CVS example is predicated with a ‘$’ symbol representing the prompt, whereas the SVN example is predicated with a ‘#’ prompt.



Classified Environment: Pre Sync

1)      Create a directory for preparation.  “~/syncRelease” works well


2)      Check out the Latest Version of Doclib to syncRelease
$: cvs co –d doclib_current doclib

#: svn checkout $svnurl/DOCLIB/DOCLIB_<version>/trunk <syncRelease>/doclib_current



3)      Validate performance and behavior

i)       Verify successful compilation under Redhat linux 9.0.

a)      Run regression test and make sure all test cases pass. Rectify any failures within the regression test.

ii)     Verify successful compilation under SunOS

a)      Run regression test and make sure all test cases pass. Rectify any failures within the regression test.



4)      Tag DOCLIB and Add-ons within CVS/SVN

i)       Tag versions of the DOCLIB module (contains add-ons) within CVS/SVN using standard naming convention (DOCLIB_BETA_GOV_X_XX_PRESYNC) where X_XX is the version number


$: cvs tag DOCLIB_BETA_GOV_X_XX_PRESYNC

#: svn copy –m “DOCLIB Beta Gov X_XX Presync” $svnurl/DOCLIB/DOCLIB_<version>/trunk $svnurl/DOCLIB/DOCLIB_<version>/tags/pre-sync

 

5)      Check out the Latest Release of Doclib to syncRelease
$: cvs co –d DOCLIB_BETA_X_XX –r DOCLIB_BETA_X_XX doclib

#: svn checkout $svnurl/DOCLIB/DOCLIB_<previous-version>/trunk <syncRelease>/DOCLIB_<previous-version>


If there is not a tagged last release, do a ‘cvs log’ or ‘svn list $svnurl/DOCLIB/DOCLIB_<version>/tags’ on any of the files in the module and there should be a listing of tags.  The last on the list should be the imported release.  If there is not a list of tags, then attempt to check out based the date (using the “–d DATE” option in place of the “-r REVISION” option) of the last import.


6)      Perform a CVS DIFF (or regular DIFF) on the two modules
From the ‘syncRelease’ directory

$: cvs diff –r DOCLIB_BETA_GOV_X_XX_PRESYNC –r DOCLIB_BETA_Y_YY

#: diff DOCLIB_<previous-version> doclib_current


where X_XX is the version you will be releasing and Y_YY is the previous released version.  Again, if there is no tag for the previous version of DOCLIB, use the “–d DATE” option.

You may want to redirect the output of CVS DIFF to a file for future revision
$: cvs diff –r DOCLIB_BETA_GOV_X_XX_PRESYNC –r DOCLIB_BETA_Y_YY > diff.out

#: diff DOCLIB_<previous-version> doclib_current > diff.out



7)      Decision Time
This is where the process gets tricky.  You must now decide if the DIFF is substantial enough to warrant an export of the files that are different (i.e. how much code are you willing to retype).

If the DIFF is large, gather up the files that are different together in a single folder and begin the NTA export process (this has never been done for a DOCLIB Release.  Usually the DIFF is small, e.g. less than 200 lines).

If the DIFF is small:

i)       Review the different files line by line for Classified content (GREP works nicely if you know what you are looking for).

a)      Replace or remove content based on necessity

ii)     Print all source code from files that are different (a2ps saves on paper) after verifying there is no sensitive information in them.

iii)  Print a copy of the DIFF file after verifying there is no sensitive information in it.



8)      Bring all changed source code and copy of the DIFF file to the Unclassified side.



Unclassified Environment: Sync Process

After the Classified changes are brought out, they need to be merged with the current Unclassified version of DOCLIB

 

1)      Validate performance and behavior

i)       Verify successfully compilation under Redhat linux 9.0.

ii)     Run regression test and make sure all test cases pass. Rectify any failures within the regression test.


2)      Tag DOCLIB and Add-ons within CVS/SVN
Tag versions of the DOCLIB module (and any Add-ons that will be synched) within CVS/SVN using standard naming convention (DOCLIB_BETA_X_XX_PRESYNC) where X_XX is the version number

 

The process of synching the Classified and Unclassified environments can be time-consuming.  There are programs available to assist in merging source code if the Classified code was exported.  If hard copies (paper) of the Classified code was exported, then the changes will need to be re-implemented on the Unclassified side line-by-line.

 



Unclassified Environment: Release Process

After the synchronizing process, DOCLIB is ready to be released.

 

1)      Validate performance and behavior

i)       Verify successfully compilation under Redhat linux 9.0.

ii)     Run regression test and make sure all test cases pass. Rectify any failures within the regression test.

 

2)      Review code written by foreign nationals

i)       All code written by foreign nationals that are being imported into Government spaces are subject to review and approval.

ii)     Download and install CSDiff freeware package from http://www.download.com

iii)  Download prior and most recent release onto directory structure

iv)   Using CSDiff tool, compare and report all differences to html files.

v)     Post all differences onto the mediaproc server at
/var/www/html/doclibDeveloper/doclib_screen

vi)   Assimilate review and approval of all new and/or modified files written by non-Government related folks (Look at log files at end of each file for authorship)

 

 

3)      Update Documentation

i)       All DOCLIB-related documents should be modified to reflect features supported within each release. Experience has shown that users of the doclib software packages perceive value and/or decide whether or not to upgrade based upon the information contained within these documents.

ii)     Update doxyfile and run doxygen against all src/include directories

iii)  Capture and fix all doxygen warning and error messages to ensure robust documentation output

iv)   Update the DOCLIBCommandLine document with any new features added to the standalone interface.

v)     Update the FeatureList document to include the latest version number and any new macro-level functions added (use the bugtracker tool for assistance)

vi)   Update LIBTIFF_BUILD.txt, readme,txt, testcases.txt, and execsummary.txt to reflect the latest version number

vii)Update src and documentation within each doclib add-on (e.g.: Logo Detect, Mach Hand detection)

 

 

4)      Tag DOCLIB and Add-ons within CVS/SVN

i)       OPTIONAL: Before tagging the release, if code is particular messy, one may run the source through the emacs auto-format script provided in the ReleaseDocs directory.

$: autoformat.sh X.cpp
$: autoformat.sh X.h


ii)     Tag versions of DOCLIB and add-ons within CVS using standard naming convention (DOCLIB_BETA_0-10, LOGODETECT_BETA_0-08, etc).

From module root:
$: cvs tag “TAG_BETA_X-XX”


‘User’ and ‘developer’ versions of DOCLIB are created and maintained for each release. The ‘user’ release contains documentation and libraries for the DOCLIB library. Additionally, the users release contains built binaries. Alternatively, the ‘developer’ release contains libraries, documentation, and source code DOCLIB and supported add-ons. The steps to create each release are described below.

 

5)      DOCLIB package – Developer’s Release

i)             Check out tagged version of Doclib and add-ons under Linux in ‘Export’ mode

ii)           Remove the CVS logs from all source code using the doclibStrip java program.

The process will resemble the following:
$: cd ~/DoclibStrip/classes/
$: java doclibstrip/DoclibStrip <source directory>


iii)        Remove the  “resource” directory from root of the doclib directory

iv)         Individually tar the doclib and add-ons within a separate directory

v)           Copy the files located within the ../doclib/ReleaseDocs/ files into this directory

vi)         Copy the tar files and the release docs to the /home/shared/doclibRelease/DOCLIB_BETA_X-XX

vii)       Update the Developer’s corner website with the new version of doclib

viii)    Burn CD with new version of doclib (it is best to burn Linux formatted tar files as opposed to Windows formatted zips).



6)      DOCLIB package – User’s Release

i)       Check out tagged version of Doclib under windows and Linux

ii)     Build Doclib and the DOCLIBCommandLine under windows and Linux

iii)  From the windows version of Doclib,  remove

a)      ..\doclib\windows\bin\*.ilk

b)     ..\doclib\windows\bin\DoclibRegressionTesting.exe

c)      ..\doclib\windows\bin\DoclibProtypeCommandLine.exe

d)     ..\ doclib\windows\projectfiles\*.* (all files and sub directories)

e)      ..\doclib\data  (including all sub directories)

f)       ..\doclib\ReleaseDocs\LIBTIFF_BUILD.txt, readme.txt, testcases.txt, make.pl, params, and “DOCLIB Beta Release Procedures.doc”

g)     ..\doclib\resource (all files and sub directories)

h)     ..\doclib\scripts (all files and sub directories)

i)       ..\doclib\*.*  (all files, leave sub directories)

j)        ..\doclib\src\*.*  (all files, leave sub directories)

 

iv)   From the Linux build, copy the “linux24” directory to the Windows build, under “doclib  (Only copy the “DoclibCommandLine” file in the “bin” directory.

v)     Zip the doclib directory with the name new version of doclib. doclib_beta_X.XX_lib.zip

vi)   Make sure the dependencies are up-to-date.  Take the dependencies from the last release and add/delete doclib dependencies.

vii)Copy the dependencies and doclib to Mediaproc: /home/shared/doclibRelease/DOCLIB_BETA_X-XX

viii)                      Make a new directory for the new release of Doclib in /var/www/html/documents/doclib/release/

ix)   Copy these files to the directory just created

a)      doclib_beta_X.XX_lib.zip

b)     doclib_Dependencies_linux.tar

c)      doclib_Dependencies_windows.zip

d)     DoclibCommandLine.txt

e)      FeatureList.txt

f)       LIBTIFF_BUILD.txt

g)     readme.txt

h)     testcases.txt

x)      Add a new page in the Doclib user’s corner pointing to the new version of DOCLIB.




Classified Environment: Release Process (TBD)

The Classified Environment will receive the Developers Release CD-ROM.  Put in an import ticket for the CD-ROM contents.  It is probably best to have them placed in a directory in your home directory.  In the past, ~/new_release/ has been used.  After the CD-ROM contents have been imported

 

1)      Unzip and untar all files and add-ons

i)       Untar the main tarball
$: tar xvvf DOCLIB_PLUS_ADDONS_BETA-X.XX.tar

ii)     Untar the Doclib module and each of the Add-on modules

2)      Update the params file
(use current Classified version as a template, or simply merge both params files).
Keep in mind that the Government Proprietary Image Format Add-on Module will need to be integrated.  See relevant Readmes in Classified Doclib module root directory.

3)      Import DOCLIB and all Add-Ons to CVS/SVN Repository, and update Repository links

i)       Import DOCLIB and each Add-On

$: cvs import –m “Initial Import” DOCLIB_BETA_X-XX bah-r61 initial-import

#: svn import –m “Initial Import” . $svnurl/DOCLIB/DOCLIB_<new-version>trunk


ii)     Update CVS links if using CVS – Example

$: cd $CVSROOT
$: rm doclib
(this should only be a symbolic link!)
$: ln –s DOCLIB_BETA_X-XX doclib
(creates new symbolic link)

4)      Validate performance and behavior

i)       Verify successfully compilation under Redhat linux 9.0

a)      Run regression test and make sure all test cases pass.

b)     Test that the Gov’t Prop Image format can be loaded and saved

ii)     Verify successful compilation under SunOS

a)      Run regression test and make sure all test cases pass.

b)     Test that the Gov’t Prop Image format can be loaded and saved

5)      Install Doclib Module

i)       Do not install DLPrototypeCommandLine or DLRegressionTest

ii)     OPTIONAL: Install two different versions of doclib library for users.  (This has never been done before)

a)      libDoclib.a – standard DOCLIB library

b)     libDoclib<X>.a – include support for Client proprietary format.

6)      Announce Release to R61