ROSE Compiler Framework/OpenK

Overview
An ongoing project to explore knowledge-driven HPC analysis and optimization. We use the standard and toolchain used by OWL to formally model the concepts and relations in HPC domains, including programs, hardware, analysis and optimization, etc.

The framework has several parts:
 * One is the C program parser to generate database.
 * The source code is under OpenK/tools/rosePrgKnowledgeBuilder, it extracts the knowledge from the input program and store it as ontology in turtle format.
 * The output knowledge base can be parsed with SWI-Prolog semweb library.
 * The second part is the canonical loop analysis part written in Prolog.
 * The third part is a demo of using Prolog C++ interface to query hardware ontology

OpenK has been integrated into ROSE under
 * https://github.com/rose-compiler/rose/tree/master/projects/OpenK

The documentation below is a bit out of date and needs to be updated.

Directory Layout
List
 * ontology/ - owl files.
 * projects/ - contains independent projects or programs: canonical loop detection, pointer analysis, etc.
 * canonicalloop/ - using Prolog query to find canonical loops
 * cfg_test/ - using Prolog to generate control flow graph
 * rose_canonicalloop/ - classic AST-based canonical loop detection, used for comparison
 * staticCFG_tests/- classic AST-based control flow graph generation, used for comparison
 * test/ - test benchmarks and scripts
 * tools/ - the knowledge generator: use ROSE frontend to parse the C code and build a knowledge base.

Software Dependences
You need to install
 * ROSE
 * SWI-Prolog

User Guide
To build the translator:


 * 1) Set ROSE environment by  . Note the set.rose file should be modified according to your machine.
 * 2) Or just manually export ROSE_INS=/home/liao6/workspace/masterDevClean/install
 * 3) export BOOST_INS=/nfs/casc/overture/ROSE/opt/rhel7/x86_64/boost/1_54_0/gcc/4.8.3
 * 4) cd to openk/tools/rosePrgKnowledgeBuilder folder.
 * 5) You can run   to make the rosePrgKnowledgeBuilder.exe and   to check it against test file under the same directory.

rosePrgKnowledgeBuilder.exe
./rosePrgKnowledgeBuilder.exe -emit-ttl ut.cc.ttl -c ./ut.cc

openk.git/tools/rosePrgKnowledgeBuilder]./rosePrgKnowledgeBuilder.exe --help -Tool-Specific Help--- This is a source translator to build knowledge base from your C/C++ code. Usage: ./rosePrgKnowledgeBuilder.exe[options][-o output]-c input.c

Options: --help             This help message. -emit-pl output.pl      ontologies in prolog file. -emit-owl output.owl    ontologies in OWL format. -emit-ttl output.ttl    ontologies in Turtle. -alive true/false       embedding Prolog engine.

Declarative canonical loop analysis
A declarative analysis tool is in openk/projects/canonicalloop


 * 1) The openk/csem is the prolog modules that implement C program fundamental analysis including searching different kinds of program constructs. They can be seen as the library modules. Feel free to add more modules under csem to extend the library.
 * 2) Every prolog-based c program analysis should have its own directory with a run.pl as the driver program. They use modules in csem.

For example, the canonical loop analysis is under openk/projects/canonicalloop.

Run

A reference rose native C++ implementation of the canonical loop is under openk/projects/rose_canonicalloop.

Run  to make roseNativeCl.exe

To test it against testbench:


 * 1) There is a script for invoking the test. The script uses the tools rosePrgKnowledgeBuilder.exe and roseNativeCl.exe and the prolog program. You don't need to do anything, the script uses the relative path to these tools.
 * 2)   to run the native Rose canonical loop execution. The output logs will be generated under current folder.
 * 3)    to run the parser to generate the database, then   to run the prolog-based canonical loop on all test files.
 * 4) Or simple, you can just run   to run all the tests.
 * 1)    to run the parser to generate the database, then   to run the prolog-based canonical loop on all test files.
 * 2) Or simple, you can just run   to run all the tests.

declarative control flow graph
cd projects/cfg_test

To generate a CFG report from a ttl file
 * swipl --nosignal --quiet run.pl sp_single.c.ttl sp_single.c.ttl_openk_cfg_report.txt

hardware ontology demo
openk.git/projects/owl_hw_demo
 * hpchardware-rdf.owl HPC hardware ontology
 * load2.pl prolog predicates to read ontology via prolog semantic web library
 * main.cpp C++ program interacting with the ontology via SWI-Prolog C++ interface

testing
We provide a python script to run various tests: cd /test/npb2.3-test

./run_test.py

Option can be one of the following: clean: clean the result files. build-kb: build knowledge base from the input codes openk-cl: run the openk canonical loop analysis. Must after the build-kb step rose-cl: run the rose native canonical loop analysis openk-cfg: openk cfg

key source files
List
 * openk.git/tools/rosePrgKnowledgeBuilder/
 * rosePrgKnowledgeBuilder.cpp
 * OntoGeneratorInterface.hpp

Command line:
 * swipl --nosignal --quiet /export/liao6/openk.git/projects/canonicalloop/run.pl bt.c.ttl bt.c.pl_openk_cl_report.txt

./projects/canonicalloop/run.pl
 * - use_module(library(semweb/rdf_db)).
 * - use_module(library(semweb/turtle)).
 * - use_module('../csem/canonical_loop.pl').
 * - use_module('../csem/url_process.pl').

./projects/csem/canonical_loop.pl /* CanonicalLoop */
 * - module(canonical_loop, [canonicalLoop/1]).
 * - use_module(c_construct).

%%	The top level specification canonicalLoop(LoopURL) :- forLoop(LoopURL), hasForInit(LoopURL, Init), isCanonicalInit(Init, LoopVar),

hasForTest(LoopURL, Test), isCanonicalTest(Test, LoopVar),

hasForIncr(LoopURL, Incr), isCanonicalIncr(Incr, LoopVar).

isCanonicalInit(InitURL, LoopVar) :- %% forLoop(LoopURL), hasForInit(LoopURL, InitURL), roseCanonicalInit(InitURL, LoopVar, _), hasType(LoopVar, Type), (intType(Type); Type == pointer_type).

%% normal style roseCanonicalInit(InitURL, LoopVar, LB) :- hasChild(InitURL, AssignOpURL), c_is_a(AssignOpURL, 'AssignOp'), !, leftOperand(AssignOpURL, VarRefURL), get_varDecl(VarRefURL, LoopVar), rightOperand(AssignOpURL, LB).

%% C99 style %% In this case, rose AST nodes have the same location roseCanonicalInit(InitURL, LoopVar, LB) :- %% hasChild(InitURL, VarURL), c_is_a(InitURL, 'VariableDecl'), c_is_a(InitURL, 'Variable'), !, \+ (hasChild(InitURL, VarDecl), c_is_a(VarDecl, 'VariableDecl')), roseHasInitValue(InitURL, LB), LoopVar = InitURL. ...