CPAM with TWW/Porting TWW HPMS to other OS

Porting TWW HPMS to other OS
There are two tasks need to be done when porting TWW HPMS tools to other OS.
 * Porting the TWW HPMS tool itself
 * Porting the package sources.
 * The makefile to automate the TWW tools porting.

Linksys NSLU2
{}
 * A Makefile to build TWW tools for nslu2.
 * compile sbtuils tool to generate sb
 * use sb to compile and build pb and pkgutils
 * Porting sbtools
 * Porting pbtools
 * Porting pkgutils

Mac OS X
Doing our homework, following are some good documents about software packaging on Mac OS X. The package management system is getting better but still fall short of a basic functionality that a PMS shoule have. Ex, you can't list out current installed package using command line tool. you can't remove a package using a command or handle package dependency automatically.
 * PackageMaker Howto
 * Local PMS

/Developer/Applications/Utilities/PackageMaker.app/Contents/MacOS/PackageMaker is getting better in 10.3.8. we can now create a .pkg with command line. Be aware to supply absolute path after -p or -f options.

Following is an partial makefile to create TWW CPAM tools in Mac OSX's local PMS format.


 * 1) A simple layout for .pkg format
 * 2) Test.pkg
 * 3)    Contents
 * 4)        Archive.bom
 * 5)        Archive.pax.gz
 * 6)        Info.plist
 * 7)        PkgInfo
 * 8)        Resources
 * 9)            Description.plist
 * 10)            postflight      -1-
 * 11)            postinstall     -2-
 * 12)            postupgrade     -3-
 * 13)            preflight       -4-
 * 14)            preinstall      -5-
 * 15)            preupgrade      -6-

PKGMK=/Developer/Applications/Utilities/PackageMaker.app/Contents/MacOS/PackageMaker SB_PKG_DIR=/opt/TWWfsw/sbutils124 SB_PKGNAME=sbutils-1.2.4 PB_PKGNAME=pbutils-1.1.12 PKGUTILS_PKGNAME=pkgutils-1.5.19 SB_PKG_HOME=/opt/build/sbutils-1.2.4 PB_PKG_HOME=${SB_PKG_HOME} PKGUTILS_PKG_HOME=${SB_PKG_HOME} SB_PKG_STG=${SB_PKG_HOME}/sbutils-stage PB_PKG_STG=${SB_PKG_HOME}/pbutils-stage PKGUTILS_PKG_STG=${SB_PKG_HOME}/pkgutils-stage PB_INSTALL_PREFIX=/opt/TWWfsw/pbutils11 PKGUTILS_INSTALL_PREFIX=/opt/TWWfsw/pkgutils15 sb-pkg: sb-pkg-prep sb-pkg-pkgmk sb-pkg-clean: rm -rf ${SB_PKG_HOME}/${SB_PKGNAME} ${SB_PKG_STG} sb-pkg-prep: rm -rf $(SB_PKG_STG) mkdir -p $(SB_PKG_STG)/Contents/Resources touch $(SB_PKG_STG)/Contents/Resources/postflight touch $(SB_PKG_STG)/Contents/Resources/postinstall touch $(SB_PKG_STG)/Contents/Resources/postupgrade touch $(SB_PKG_STG)/Contents/Resources/preflight touch $(SB_PKG_STG)/Contents/Resources/preinstall touch $(SB_PKG_STG)/Contents/Resources/preupgrade sb-pkg-pkgmk: $(PKGMK) -build -p ${SB_PKG_HOME}/${SB_PKGNAME}.pkg -f $(SB_INSTALL_PREFIX) -i ${SB_PKG_HOME}/src/macosx/Info.p\list.sb -d ${SB_PKG_HOME}/src/macosx/Description.plist.sb

sb-pkg-pkgmk-clean: rm -rf ${SB_PKG_HOME}/${SB_PKGNAME} sb-pkg-install: /usr/sbin/installer -pkg sbutils-1.2.4-1.pkg -target / sbutils-1.2.4 compiled ok on Mac OS X 10.3.8. We also can use packagemaker to generate sbutils-1.2.4 into Mac OS X's native package format. Following quick notes need to be documented later.
 * Porting sbtools

conchi:~/tww-tools/sbutils-1.2.4 tjyang$ sudo make sb-pkg rm -rf /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage mkdir -p /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/postflight touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/postinstall touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/postupgrade touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/preflight touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/preinstall touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/preupgrade cp -rp /opt/TWWfsw/sbutils124/* /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage ^Cmake: *** [sb-pkg-prep] Interrupt

conchi:~/tww-tools/sbutils-1.2.4 tjyang$ sudo make sb-pkg rm -rf /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage mkdir -p /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/postflight touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/postinstall touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/postupgrade touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/preflight touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/preinstall touch /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Resources/preupgrade /usr/bin/mkbom -s /opt/TWWfsw/sbutils124 /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Archive.bom (pax -w -x cpio /opt/TWWfsw/sbutils124 |gzip - > /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents/Archive.pax.gz) /Developer/Applications/Utilities/PackageMaker.app/Contents/MacOS/PackageMaker -build -p /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-1.2.4-1.pkg  -f /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-stage/Contents -i //Users/tjyang/tww-tools/sbutils-1.2.4/Info.plist -d //Users/tjyang/tww-tools/sbutils-1.2.4/Description.plist 2005-02-27 23:29:24.664 PackageMaker[19034] ** building a package at /Users/tjyang/tww-tools/sbutils-1.2.4/sbutils-1.2.4-1.pkg ** 2005-02-27 23:29:32.961 PackageMaker[19034] done (0) conchi:~/tww-tools/sbutils-1.2.4 tjyang$ ls -lrt total 216 -rwx--  1 tjyang  tjyang  42318 23 Feb 15:04 systype -rw-r--r--  1 tjyang  tjyang   1069 27 Feb 15:49 cipo.error.txt drwx-- 43 tjyang  tjyang   1462 27 Feb 15:59 src -rwxr-xr-x  1 tjyang  tjyang   7796 27 Feb 22:23 genpkg.sh -rw-r--r--   1 tjyang  tjyang    459 27 Feb 23:13 Description.plist -rw-r--r--  1 tjyang  tjyang   1050 27 Feb 23:17 Info.plist -rwx--  1 tjyang  tjyang  44915 27 Feb 23:28 Makefile drwxr-xr-x  3 root    tjyang    102 27 Feb 23:28 sbutils-stage drwxr-xr-x  3 root    tjyang    102 27 Feb 23:29 sbutils-1.2.4-1.pkg conchi:~/tww-tools/sbutils-1.2.4 tjyang$ ls sbutils-1.2.4-1.pkg/ Contents conchi:~/tww-tools/sbutils-1.2.4 tjyang$ ls sbutils-1.2.4-1.pkg/Contents/ Archive.bom    Archive.pax.gz  Info.plist      PkgInfo         Resources conchi:~/tww-tools/sbutils-1.2.4 tjyang$ ls -l sbutils-1.2.4-1.pkg/Contents/ total 32312 -r--r--r-- 1 root  tjyang     35525 27 Feb 23:29 Archive.bom -r--r--r-- 1 root  tjyang  16496572 27 Feb 23:29 Archive.pax.gz -r--r--r--  1 root  tjyang      1050 27 Feb 23:29 Info.plist -r--r--r-- 1 root  tjyang         8 27 Feb 23:29 PkgInfo drwxr-xr-x 7 root  tjyang       238 27 Feb 23:29 Resources conchi:~/tww-tools/sbutils-1.2.4 tjyang$ ls -l sbutils-1.2.4-1.pkg/Contents/Resources/ total 40 -r--r--r-- 1 root  tjyang  459 27 Feb 23:29 Description.plist lrwxr-xr-x 1 root  tjyang   14 27 Feb 23:29 sbutils-1.2.4-1.bom -> ../Archive.bom -r--r--r-- 1 root  tjyang  275 27 Feb 23:29 sbutils-1.2.4-1.info lrwxr-xr-x 1 root  tjyang   17 27 Feb 23:29 sbutils-1.2.4-1.pax.gz -> ../Archive.pax.gz -r--r--r--  1 root  tjyang   46 27 Feb 23:29 sbutils-1.2.4-1.sizes conchi:~/tww-tools/sbutils-1.2.4 tjyang$ vi sbutils-1.2.4-1.pkg/Contents/Resources/sbutils-1.2.4-1.info conchi:~/tww-tools/sbutils-1.2.4 tjyang$

Microsoft Windows
The first problem of porting TWW HPMS tool to Win32 platformat is which package creation software and Unix foundation software to base upon.

"aux" need to be renamed to "auxx". perl -pi -e 's!\/aux\/!/aux!' Makfile http://cygwin.com/setup.html is the place to understand cygwin package format. 
 * Unix foundation software
 * cygwin notes for sbutils package source.
 * mingw
 * SFU
 * Package Creation tools
 * MakeMSI (see http://users.cyberone.com.au/dbareis/makemsi.htm) automatically builds (or updates) MSI files (or any other type of Windows Installer databases). It uses an XML like scripting language which can be invoked via explorer or the command line. It also builds MSI documentation which includes file (MD5 checks installed files against expected etc.) and product verification buttons. It includes a very powerful preprocessor.

Dialogs can be created and easily formatted without a dialog editor. There are a number of "frameworks" for different uses and user skills. Custom action code can be integrated and has some ADSI/COMPLUS/scheduling and other support.

The following is a small sample script (one file, one shortcut, one smallish VBSCRIPT based Custom Action). 
 * --- Load MAKEMSI (via wrapper which configures MAKEMSI to work how I like it) ---
 * 1) include "DENNIS.MMH"

<$DirectoryTree Key="INSTALLDIR" Dir="[ProgramFilesFolder]\TryMe" CHANGE="\" PrimaryFolder="Y">
 * --- Create INSTALLDIR (we wish to install file(s) here) ---

<$Component "TryMe" Create="Y" Directory_="INSTALLDIR">  ;;In this case GUID autogenerated (at least 1st time) ;--- Add the files to the "TryMe" component - <$File SOURCE="TryMeWithAdvShortCut.MM" KeyPath="Y">
 * --- Add "TRY.MM" and shortcut to same component (file must be keypath) ---

;--- Add the shortcut (advertise the "complete" feature) #(--- Advertised shortcut in "Start->programs" ---       <$Shortcut                   Dir="ProgramMenuFolder"               Feature="\"                        ;;root feature is my "complete" one                 Title="Open TryMeWithAdvShortCut.MM (via default association)"           Description=^Simply opens "TryMeWithAdvShortCut.MM" in associated handler (probably notepad)!^                  Icon="@.\TryMe.ico"               WorkDir="INSTALLDIR"        >    #) <$/Component>

;--- Some data we'd like to pass to the CA --- "InstallationDirectory" "[INSTALLDIR]" "SystemFolder" "[SystemFolder]" <$VbsCa Binary="Popup.vbs" DATA=^PopupCaData^> <$VbsCaEntry "HiThere"> ;--- If the VBSCRIPT "MsgBox" or similar function was used this would prevent a silent install --- #(           CaMsgBox "I",                     "Hi this is a message from '<?InputFile>'! This was displayed via a VBSCRIPT MSI custom action." &                     vbCRLF & vbCRLF &                     "Note that I have enlarged this message box so that when an error occurs you will see the whole text! " &                     "This and most tweeking MAKEMSI does to the user interface is optional (most performed by ""UISAMPLE.MMH"")." &                     vbCRLF & vbCRLF &                     "The installation directory is """ & VbsCaCadGet("InstallationDirectory") & """. " &                     "The system directory is """       & VbsCaCadGet("SystemFolder")          & """."       #) <$/VbsCaEntry> <$/VbsCa> <$VbsCaSetup Binary="Popup.vbs" Entry="HiThere" Seq="StartServices-" CONDITION=^<$CONDITION_INSTALL_ONLY>^ DATA=^PopupCaData^>
 * --- Display a message during a non-silent install (logs if silent) -
 * 1) data "PopupCaData"
 * 1) data

candle and light are a pair of commandline package creation tool similar to rpmbuild in redhat, packagemaker in Max OS X or pkgmk in solaris. F:\src\wix\tutorial\Sample1>cat batch11.bat del Sample11.msi candle -nologo Sample11.wxs light -nologo Sample11.wixobj msiexec /i Sample11.msi /l* Sample11.log rem msiexec /x Sample11.msi /qb F:\src\wix\tutorial\Sample1> F:\src\wix\tutorial\Sample1>cat Sample11.wxs   
 * NSIS http://nsis.sourceforge.net/
 * Inno
 * Wix at http://sourceforge.net/projects/wix, it has XML,scripting and command line support to generate a MSI package.
 * 1) The challenge of using Wix tool set to generate a Unix package is the GUID. Fortunately mallow.exe come to the rescue. it can scan a src directory and generate GUID for each component(directory or file).
 * 2) We need to figure out how to make pb to generate wxs xml file and communicate with candle and light to gernate msi pacakge with the help of mallow.
 * 1) Following is an small batch file to illustrate creating a msi package, installing and remove this package.
 * 1) The wix package source is display in following text, following are todos.
 * 2) Converting pbuitils' .pb xml file to wix's wxs XML.
 * 3) Figuring out msiexec to delete a package using a package ID without the presence of actual .msi package.





   

     </Component>

<Component Id='Manual' Guid='CE36F399-574D-4A9A-A266-5B5EC2C022A4'> <File Id='Manual' Name='Manual.pdf' DiskId='1' src='Manual.pdf' />

<Shortcut Id="startmenuManual" Directory="ProgramMenuDir" Name="Manual" LongName="Instruction Manual" Target="Documentation" /> </Component>

</Directory> </Directory> </Directory>

<Directory Id="ProgramMenuFolder" Name="PMenu" LongName="Programs"> <Directory Id="ProgramMenuDir" Name='Foobar10' LongName="Foobar 1.0" /> </Directory>

<Directory Id="DesktopFolder" Name="Desktop" /> </Directory>

<Feature Id='Complete' Level='1'> <ComponentRef Id='MainExecutable' /> <ComponentRef Id='Manual' /> </Feature>

</Product> </Wix>

F:\src\wix\tutorial\Sample1> http://www.tramontana.co.hu/wix/ http://www.ryanlowe.ca/blog/archives/001167_learning_wix_part_one.php http://blogs.msdn.com/robmen/
 * References of WiX and MS installer.
 * Porting sbtools
 * Porting pbtools
 * Porting pkgutils

FreeBSD

 * Porting sbtools
 * Porting pbtools
 * Porting pkgutils

Debian

 * Porting sbtools
 * Porting pbtools
 * Porting pkgutils

Package Management Process
Software engineering process include design and implement software via software developers to generate source codes. The source codes will then need to be built, packaged and be managed. We can divide this process into three general steps.


 * Software build process(SB) and SB can be divided into following processes on a build machine
 * source unpack
 * source patch
 * source configure
 * source build to generate binary and documentation
 * binary test
 * binary install
 * binary uninstall
 * Package build (PB) process is much straight forward
 * prepare the configuration script
 * prepare init script to start/stop application service
 * call up local PMS to pack the binary,documentation and scripts into local PMS format.
 * Package Management (PKGUTIL) Process
 * upload and administrate packages on package depot server.
 * install
 * remove
 * upgrade/patch
 * auto installation upon package dependence.

TWW HPMS introduction
TWW Inc. has three tools to match the package management processes.

[root] ls -l /opt/TWWfsw/sbutils12/bin/ total 164 -rwxr-xr-x  1 root     root       38551 Dec  1 15:17 gen-sb-db -rwxr-xr-x  1 root     root       44418 Dec  1 15:17 sb [root] [root] ls -l /opt/TWWfsw/pbutils11/bin/ total 186 -rwxr-xr-x  1 root     root       29975 Apr 30  2004 gen-pb-db -rwxr-xr-x  1 root     root       29169 Apr 30  2004 gen-pkg-db -rwxr-xr-x  1 root     root       34023 Apr 30  2004 pb [root]
 * Softeware build
 * sbutils
 * Package build
 * pbutils

[root] ls -l /opt/TWWfsw/pkgutils15/bin/ total 542 -rwxr-xr-x  1 root     root       32434 Jan 25  2004 chk-pkg-updates -rwxr-xr-x  1 root     root       18751 Jan 25  2004 pkg-config -rwxr-xr-x  1 root     root       48281 Jan 25  2004 pkg-info -rwxr-xr-x  1 root     root       70570 Jan 25  2004 pkg-inst -rwxr-xr-x  1 root     other      68887 May 18  2003 pkg-inst~ -rwxr-xr-x  1 root     root       35692 Jan 25  2004 pkg-rm [root]
 * Package Management
 * pkgutils
 * How to get TWW tools ?

TWW Inc.
All the TWW tools and sources of their supported packages are GPL compliant. They are freely available at their ftp site ftp://support.thewrittenword.com. But their support labor and binary packages are not free of charge. We are glad TWW Inc. find a way to do open-source works and still can make a living out of it. Please support TWW Inc. if your workplace need such kind of service ( http://shopping.netsuite.com/tww ).