ROSE Compiler Framework/Jenkins Failures

Misc. failures
Problem Failure in astFileIOTests: filename = /getcwd: : No such file or directory ... Error preDeleteTests: classType->get_declaration = 0x25492af0 = SgClassDeclaration make[5]: *** [test-read-medium] Aborted make[4]: *** [check-local] Error 2

Solution: This is usually due to a system error so the solution is to simply restart the failed job.

Problem Failure in git-clone: ERROR: Error cloning remote repo 'origin' : Could not clone rose-dev@rosecompiler.llnl.gov:rose/edg4x/rose-sh.git hudson.plugins.git.GitException: Could not clone rose-dev@rosecompiler.llnl.gov:rose/edg4x/rose-sh.git at hudson.plugins.git.GitAPI.clone(GitAPI.java:271) at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:1040) at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:982) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2273) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:270) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) ...

Solution: This is usually due to a system error so the solution is to simply restart the failed job.

Problem Failure in git-clone or ssh: ControlSocket /tmp/rose-dev@rosecompiler.llnl.gov:22 already exists fatal: The remote end hung up unexpectedly

at hudson.plugins.git.GitAPI.launchCommandIn(GitAPI.java:863) at hudson.plugins.git.GitAPI.access$000(GitAPI.java:40) at hudson.plugins.git.GitAPI$1.invoke(GitAPI.java:267) at hudson.plugins.git.GitAPI$1.invoke(GitAPI.java:246) at hudson.FilePath.act(FilePath.java:852) at hudson.FilePath.act(FilePath.java:825) at hudson.plugins.git.GitAPI.clone(GitAPI.java:246) ... 12 more Trying next repository ERROR: Could not clone repository FATAL: Could not clone

Solution: The error here has to do with the SSH configuration (~/.ssh/config), specifically the ControlSocket setting. The solution could be to remove this optional (convenience) configuration.

Problem Failure in git-submodule-update: Commencing build of Revision 1d4fd18f6d17dd66553a248496026ceb88773fac (detached) Merging Revision 1d4fd18f6d17dd66553a248496026ceb88773fac (detached) onto master FATAL: Command "git submodule update" returned status code 1: fatal: reference is not a tree: d19d03c5d63860f0e9cc1c35afc4f8b8ca682afd Unable to checkout 'd19d03c5d63860f0e9cc1c35afc4f8b8ca682afd' in submodule path 'src/frontend/CxxFrontend/EDG'

Solution: Ensure that you have pushed your accompanying submodule commits.

Problem Input/Output error while executing a script /tmp/hudson2706138848682410422.sh: line 29: /home/hudson-rose/Applications/jenkins/scripts/tests/rose-test.sh: Input/output error Build step 'Execute shell' marked build as failure

Solution: Restart the job; this is an I/O error that could be related to NFS.

Problem' Problem fetching from origin / origin - could be unavailable. Continuing anyway. Fetching upstream changes from rose-dev@rosecompiler.llnl.gov:rose/edg4x/rose-sh.git ERROR: Problem fetching from origin / origin - could be unavailable. Continuing anyway. hudson.plugins.git.GitException: Command "git fetch -t rose-dev@rosecompiler.llnl.gov:rose/edg4x/rose-sh.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: ssh_exchange_identification: Connection closed by remote host fatal: The remote end hung up unexpectedly

at hudson.plugins.git.GitAPI.launchCommandIn(GitAPI.java:863)

Solution: Restart the job; this is due to ?.

Problem' $ROSE_INSTALL does not exist $ROSE_INSTALL does not exist: '/home/hudson-rose/opt/rose/edg4x//frontend/EDG_4_4/gcc/4.2.4' + exit 1

Solution: Ensure that the make-install job has completed first for the given GIT_COMMIT and GCC Version. Then, you may restart the job.

Problem rsync error: some files/attrs were not transferred rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]

Solution: Restarting the job could solve it. This is usually due to some NFS, SSH, or permission problem while a ROSE workspace is being copied.

Problem ../../scripts/dotgenLetterPDF inputCode_wholeAST_1_WholeAST.dot inputCode_ASTWholeGraphGenerator_small.pdf Can't find GraphViz dot program in user's path, ignoring conversion to postscript file (not a problem) ... if ! test -e inputCode_ASTWholeGraphGenerator_small.pdf; then exit 1; fi

C0-Start
http://hudson-rose-30:8080/job/C0-Start/

1. Problem: can't merge your branch cleanly

Solution: Run <tt>git-merge</tt> on your local machine to resolve the merge conflicts with master.

For example,

2. Problem: <tt>FAILED Has new commits?</tt>

Solution: Make sure you've committed your changes and then push your changesets again. Otherwise, ignore the message, maybe your already-integrated branch simply got tested by mistake.

3. Problem: <tt>Errors detected by the no tab characters policy. Aborting now.</tt>

The following C/C++ source files contain TAB characters, which can be fixed in a variety of ways. Perhaps the easiest is to run this script again with the "--fix" switch (see "./scripts/policies/NoTabCharacters.pl --help" for details). src/midend/programAnalysis/genericDataflow/analysis/analysis.C       src/midend/programAnalysis/genericDataflow/analysis/dataflow.C        src/midend/programAnalysis/genericDataflow/simpleAnalyses/liveDeadVarAnalysis.C Errors detected by the no tab characters policy. Aborting now. Build step 'Execute shell' marked build as failure Project C0-Start still waiting for 12 builds to complete Notifying upstream projects of job completion Finished: FAILURE

Solution: Follow the instructions to remove the TAB characters from your source files:

$ cd $ROSE_SOURCE $ ./scripts/policies/NoTabCharacters.pl --fix

And next time, please run the tests locally before you push! (The policy checks are a part of <tt>make all</tt>. You can also run <tt>./scripts/policies-checker.sh</tt> separately from the top-level of the ROSE source tree.)

4. Problem: <tt>Errors detected by the unix line termination policy. Aborting now.</tt>

POLICY Checking unix line termination policy (./scripts/policies/UnixLineTermination.pl) ROSE developers predominantly use Unix-like systems for development. These systems use the ASCII LF (line feed, '\n') character for line termination. Some other operating systems insert a CR (carriage return, '\r') in front of   the LF. Insertion of the extra CR causes problems for revision management software and some Unix-based text editors which developers then have to work around and/or correct. To convert a file from any mixture of CR+LF or   LF-only line endings to LF-only line endings, use: * the unix command "dos2unix" * vim 7.0 and later commands ':update', ':setlocal ff=unix'. * in XEmacs use the "M-x set-buffer-file-coding-system RET" and enter "undecided-unix". The following source files have CR+LF line termination: 3 ( 100%) lines in ./tests/CompileTests/Fortran_tests/test2011_Rice_function-no-result-1.f90 6 ( 100%) lines in ./tests/CompileTests/Fortran_tests/test2011_Rice_function-no-result-2.f90 3 ( 100%) lines in ./tests/CompileTests/Fortran_tests/test2011_Rice_function-result-1.f90 Errors detected by the unix line termination policy. Aborting now. ... Finished: FAILURE

Solution: Follow the instructions to remove the CR+LF line termination characters from your source files:

$ man dos2unix

$ dos2unix ${ROSE_SOURCE}/path/to/culprit/file_1 $ dos2unix ${ROSE_SOURCE}/path/to/culprit/file_2 ... $ dos2unix ${ROSE_SOURCE}/path/to/culprit/file_N

And next time, please run the tests locally before you push! (The policy checks are a part of <tt>make all</tt>. You can also run <tt>./scripts/policies-checker.sh</tt> separately from the top-level of the ROSE source tree.)

5. Problem: <tt>Errors detected by the unused sources policy. Aborting now.</tt>

Number of remaining unused source files: 1482. Errors detected by the unused sources policy. Aborting now. ... Finished: FAILURE

The "Unused Sources Policy" will pass with "<tt>Number of remaining unused source files: 1481.</tt>". Therefore, you will see at least 1481 lines of output source files. No, they are not your problem. Simply, when this policy was first put into place, there were culprit files already. The point of this policy is simply not to lower the bar, i.e. to not go above 1481 unused sources. (Maybe one day we'll actually fix this).

Solution: Make sure your source file is mentioned in some sort of Makefile that matches this regular expression: <tt>/\/(Makefile\.am|Makefile_variables|CMakeList\.txt|make_rule\.inc|Makefile-.*\.inc)$/</tt> (as of commit <tt>213fd5d72</tt>, July 26, 2012).

And next time, please run the tests locally before you push! (The policy checks are a part of <tt>make all</tt>. You can also run <tt>./scripts/policies-checker.sh</tt> separately from the top-level of the ROSE source tree.)

C6-ROSE-distcheck
http://hudson-rose-30:8080/job/C6-ROSE-distcheck/

Problem: make[3]: *** No rule to make target `README', needed by `distdir'. Stop. ... make: *** [distdir] Error 1


 * FAILED make distcheck step
 * FAILED make distcheck step

The problem is that some files are not automatically added into the software distribution during "<tt>make dist</tt>". For example, input test code files, <tt>README</tt> files, and configuration files.

Prognosis:

Overview of "<tt>make distcheck</tt>":

1. "<tt>make dist</tt>": ensure that you can successfully create a tarball distribution of the software package.

2. The tarball distribution is un-tarred (unpackaged) and the tests are then carried out on this distribution source tree.

3. "<tt>$ROSE/configure</tt>": configure the software package with GNU Autoconf.

4. "<tt>make all</tt>": build the software package GNU Automake.

5. "<tt>make check</tt>": execute the software package's regression tests using GNU Automake <tt>check-local</tt> rules.

Files are not always automatically added into the distribution tarball (during "<tt>make dist</tt>"). However, most source and header files will be automatically added if they are used to build libraries and executables (specified within Automake makefiles, <tt>Makefile.am</tt>).

Solution:
 * Add the culprit files into the <tt>EXTRA_DIST</tt> Automake variable in the appropriate <tt>Makefile.am</tt> file.
 * Test "<tt>make distcheck</tt>" on your local machine before you push next time!

Reference:
 * Basics of distribution: http://www.delorie.com/gnu/docs/automake/automake_91.html

Problem: <tt>cannot find input file Makefile.in</tt>

config.status: error: cannot find input file: projects/RTC/docs/Makefile.in make: *** [distcheck] Error 1

The problem is that the <tt>Makefile.am</tt> does not list all of the project's sub-directories. The result is that <tt>make dist</tt> is unable to create a complete distribution since it does not know to descend into the project's sub-directories since they are not listed.

The <tt>$ROSE/projects/RTC/Makefile.am</tt> looked like this:

SUBDIRS = src

However, there are more sub-directories than just <tt>src/</tt>:

$ ls projects/RTC/ docs/ include/  libs/  Makefile.am  Makefile.in  Makefile.in~  RTC.C  src/  tests/

Solution:

Add ALL sub-directories to <tt>$ROSE/projects/RTC/Makefile.am</tt>:

SUBDIRS = \ docs \ include \ libs \ src \ tests

Problem:No rule for librose.la make[4]: *** No rule to make target `../../../src/librose.la', needed by `vectorization'. Stop. make[4]: *** Waiting for unfinished jobs.... ... make: *** [distdir] Error 1


 * FAILED make distcheck step
 * FAILED make distcheck step

The following Makefile.am is used in the tests directory of a project. The translator simply takes the code in <tt>TESTCODES</tt> as input and performs the translation.

The problem is that <tt>TESTCODES</tt> is listed in <tt>EXTRA_DIST</tt> for distribution and also served as target rule in the Makefile. It's still unclear how Make distdir handles this case. But avoiding using the variable in <tt>EXTRA_DIST</tt> as Makefile rule can eliminate this issue.

Solution:
 * Add another variable for the makefile rule.

Problem Operation not permitted

C6-ROSE-distcheck

********************************************************************************       *** Starting ROSE-build setup step ********************************************************************************       chmod: changing permissions of `./ROSE-build/rose-0.9.5a': Operation not permitted chmod: changing permissions of `./ROSE-build/rose-0.9.5a/docs': Operation not permitted chmod: cannot read directory `./ROSE-build/rose-0.9.5a/docs': Permission denied chmod: changing permissions of `./ROSE-build/rose-0.9.5a/rose_config.h': Operation not permitted chmod: changing permissions of `./ROSE-build/rose-0.9.5a/rose_config.h.in': Operation not permitted chmod: changing permissions of `./ROSE-build/rose-0.9.5a/ROSE_ResearchPapers': Operation not permitted chmod: changing permissions of `./ROSE-build/rose-0.9.5a/ROSE_ResearchPapers/2008-09-SignatureVisualizationOfSoftwareBinaries-SoftVis.pdf': Operation not permitted chmod: changing permissions of `./ROSE-build/rose-0.9.5a/ROSE_ResearchPapers/2009-TechniquesForSoftwareQualityAnalysisOfBinaries-ISTA.pdf': Operation not permitted chmod: changing permissions of `./ROSE-build/rose-0.9.5a/ROSE_ResearchPapers/lcpc03_qsys_final.tgz': Operation not permitted chmod: changing permissions of `./ROSE-build/rose-0.9.5a/ROSE_ResearchPapers/2007-CommunicatingSoftwareArchitectureUsingAUnifiedSingle-ViewVisualization-ICECCS.pdf': Operation not permitted chmod: changing permissions of `./ROSE-build/rose-0.9.5a/ROSE_ResearchPapers/2003-ParallelObject-OrientedFrameworkOptimization-CPC.pdf': Operation not permitted chmod: changing permissions of `./ROSE-build/rose-0.9.5a/ROSE_ResearchPapers/2007-AnalyzingAndVisualizingWholeProgramArchitectures-Aerose-ICSE.pdf': Operation not permitted [many more similar lines omitted...]

C2-ROSE-language-matrix-linux
http://hudson-rose-30:8080/job/C2-ROSE-language-matrix-linux/ http://hudson-rose-30:8080/job/C2-ROSE-language-matrix-osx/

Problem: ROSE has configuration options (e.g., <tt>--enable-only-java</tt>) to toggle (compiler frontend) language support (see <tt>$ROSE/configure --help</tt> for more information).

Many new developers are not aware of this so their work often fails on these tests due to invalid assumptions about which frontends are enabled, or disabled.

Solution: To pass these tests, make sure you have conditional blocks in your GNU Automake <tt>Makefile.am</tt> to turn on (compiler frontend) language specific logic.

Example: In your Fortran-specific work, use the following conditional:

if ROSE_BUILD_FORTRAN_LANGUAGE_SUPPORT check-local: echo "Fortran (compiler frontend) language support is ENABLED!" else echo "Fortran (compiler frontend) language support is DISABLED!" endif

Language independent test codes can be found in <tt>${ROSE}tests/input_codes/Makefile.for.input.codes</tt>:

A full example of these conditionals are given in this file:


 * 1) TODO: add CUDA and OpenCL


 * 1) Minimal input codes (just enough to be 'compiled')
 * 1) Minimal input codes (just enough to be 'compiled')

if ROSE_BUILD_C_LANGUAGE_SUPPORT __minimal_input_code = $(top_srcdir)/tests/input_codes/minimal/minimal.cpp else if ROSE_BUILD_FORTRAN_LANGUAGE_SUPPORT __minimal_input_code = $(top_srcdir)/tests/input_codes/minimal/minimal.f90 else if ROSE_BUILD_JAVA_LANGUAGE_SUPPORT __minimal_input_code = $(top_srcdir)/tests/input_codes/minimal/minimal.java else if ROSE_BUILD_PHP_LANGUAGE_SUPPORT __minimal_input_code = $(top_srcdir)/tests/input_codes/minimal/minimal.php else if ROSE_BUILD_PYTHON_LANGUAGE_SUPPORT __minimal_input_code = $(top_srcdir)/tests/input_codes/minimal/minimal.py else if ROSE_BUILD_BINARY_ANALYSIS_SUPPORT __minimal_input_code = $(top_srcdir)/tests/input_codes/minimal/minimal.binary endif # Binary analysis endif # Python endif # PHP endif # Java endif # Fortran endif # C
 * 1) ROSE_BUILD_C_LANGUAGE_SUPPORT and ROSE_BUILD_CXX_LANGUAGE_SUPPORT are equivalent in ROSE due to the same EDG C++ frontend used.

Java unexpected error
../../testTranslator -rose:java:d javac-output-classes -rose:verbose 0 ../../../../tests/CompileTests/Java_tests/test2011_02.java make[5]: *** [test2011_02.o] Error 1 make[5]: *** Waiting for unfinished jobs.... Test using Java 1.4 version support.
 * 1) An unexpected error has been detected by Java Runtime Environment:
 * 2)  SIGSEGV (0xb) at pc=0x00002b2e79c63961, pid=24881, tid=47481965754688
 * 3) Java VM: Java HotSpot(TM) 64-Bit Server VM (11.0-b16 mixed mode linux-amd64)
 * 4) Problematic frame:
 * 5) V  [libjvm.so+0x484961]
 * 6) An error report file with more information is saved as:
 * 7) /export/tmp.hudson-rose/hudson/workspace/C1-ROSE-from-scratch-linux/ROSE_TEST_BUILD_STYLE/make-check-install-minimal/ROSE_TEST_GCC_VERSION/4.0.4/label/amd64-linux/ROSE-build/tests/CompileTests/Java_tests/hs_err_pid24881.log
 * 8) If you would like to submit a bug report, please visit:
 * 9)   http://java.sun.com/webapps/bugreport/crash.jsp
 * 10) The crash happened outside the Java Virtual Machine in native code.
 * 11) See problematic frame for where to report the bug.
 * 1) If you would like to submit a bug report, please visit:
 * 2)   http://java.sun.com/webapps/bugreport/crash.jsp
 * 3) The crash happened outside the Java Virtual Machine in native code.
 * 4) See problematic frame for where to report the bug.
 * 1) See problematic frame for where to report the bug.

../../testTranslator -rose:verbose 0 -rose:detect_dangling_pointers 2 -I../../../../tests/CompileTests/Fortran_tests -rose:f90 -c ../../../../tests/CompileTests/Fortran_tests/test2011_Rice_A-cant-assert-keyword-nonnull.f90

Note: astNode = 0x98dc790 = SgBasicBlock parent = 0x98fdad8 Note: parentNode = 0x98dc790 = SgBasicBlock <font style="color:red">ERROR: building test2011_Rice_A-cant-assert-keyword-nonnull.o
 * 1) An unexpected error has been detected by Java Runtime Environment:
 * 2)  SIGSEGV (0xb) at pc=0x00d38750, pid=12219, tid=3012402064
 * 3) Java VM: Java HotSpot(TM) Server VM (11.0-b16 mixed mode linux-x86)
 * 4) Problematic frame:
 * 5) C  [libstdc++.so.6+0xb5750]  __dynamic_cast+0x20
 * 6) An error report file with more information is saved as:
 * 7) /export/tmp.hudson-rose/hudson/workspace/C1-ROSE-from-scratch-linux/ROSE_TEST_BUILD_STYLE/make-check-install-minimal/ROSE_TEST_GCC_VERSION/4.1.2/label/i686-linux/ROSE-build/tests/CompileTests/Fortran_tests/hs_err_pid12219.log
 * 8) If you would like to submit a bug report, please visit:
 * 9)   http://java.sun.com/webapps/bugreport/crash.jsp
 * 10) The crash happened outside the Java Virtual Machine in native code.
 * 11) See problematic frame for where to report the bug.
 * 1) If you would like to submit a bug report, please visit:
 * 2)   http://java.sun.com/webapps/bugreport/crash.jsp
 * 3) The crash happened outside the Java Virtual Machine in native code.
 * 4) See problematic frame for where to report the bug.
 * 1) See problematic frame for where to report the bug.

Hide Details make[6]: *** [test2011_Rice_A-cant-assert-keyword-nonnull.o] Aborted make[6]: *** Waiting for unfinished jobs....