Commit 46302e73 authored by Kadir's avatar Kadir

a case study for sparse linear algebra has been added

parent f1382370

Too many changes to show.

To preserve performance only 1000 of 1000+ files are displayed.
CodeVault: Case Studies for Sparse Linear Algebra
================
# Overview
In this case studies folder, there are sample codes for beginners, as well as advanced codes including important algorithm, which include sparse linear algebra
# Contributors & Maintainers
- Cevdet Aykanat (aykanat@cs.bilkent.edu.tr)
- Kadir Akbudak (kadir.cs@gmail.com)
- Reha Oguz Selvitopi(reha@cs.bilkent.edu.tr)
# Contents
- pagerank: calculates ranking of pages. Sparse linear algebra is implemented via PETSc.
=======
README
=======
- 1. Code sample name
pagerank
- 2. Description of the code sample package
pagerank code calculates ranking of pages. The link information of the pages is given as a graph. The original code is taken from [1].
This package contains the following important directories:
1) pagerank: Implementation of the PageRank algorithm. This code is developed by D. Gleich, L. Zhukov, and P. Berkhin. No license information is given on the download page [1]. C programming language is used.
2) petsc-2.3.2-p10: PETSc version 2.3.2-p10, which is used to implement the PageRank algorithm.
3) pagerank/boost: The Boost library. Some of its utilities are used in the pagerank code.
4) pagerank/data: The input files are stored in this directory.
The implementation of the kernel algorithm is located at line 278 of ppagerank.cc file. This file is found under the directory pagerank.
This is one of the code sample from the PRACE CodeVault. You can find more code samples available for download from the PRACE CodeVault here: https://gitlab.com/PRACE-4IP/CodeVault
- 3. Release date
08 April 2016
- 4. Version history
1.0: initial version
- 5. Contributor (s) / Maintainer(s)
Kadir Akbudak, Bilkent University <kadir.cs@gmail.com>
Cevdet Aykanat, Bilkent University <aykanat@cs.bilkent.edu.tr>
- 6. Copyright / License of the code sample
Apache 2.0
- 7. Language(s)
C
- 8. Parallelisation Implementation(s)
MPI
- 9. Level of the code sample complexity
advanced
- 10. Instructions on how to compile the code
Pre-requisities are gcc, make, Bash, and an MPI library.
Compilation of the package contains two steps:
1) For compilation of PETSc, run the following commands in Bash:
cd petsc-2.3.2-p10
./config/configure
PETSC_DIR= <workingdir>/petsc-2.3.2-p10
export PETSC_DIR
PETSC_ARCH=linux-gnu-c-debug
export PETSC_ARCH
make
cd ..
2) For compilation of pagerank, run the following commands in Bash:
cd pagerank
Set the variable PETSC_DIR in the makefile as ../petsc-2.3.2-p10.
make
- 11. Instructions on how to run the code
Run the script named run.sh. The script contains the following line:
mpirun -np 4 ./ppagerank -m data/example-tiny.graph
The only parameter is the input data.
- 12. Sample input(s)
Data files are located under the folder named data. There is only one test data in this folder. More files can be obtained from [2].
- 13. Sample output(s)
The program prints the PageRank vector. This vector contains the rank of each page.
REFERENCES:
[1] https://github.com/dgleich/ppagerank
[2] https://www.cs.purdue.edu/homes/dgleich/codes/innout/innout-web.tar.gz
pagerank @ 67ee5994
Subproject commit 67ee5994a8c05aaeab0dbd1878f12877528e839a
syntax: glob
RDict.db
RDict.log
TAGS*
bmake/*/*
bmake/petscconf
configure.log*
externalpackages/*
*/ftn-auto/*
lib/*
make_log_*
test_log_*
src/python/*
*.pyc
*~
*.orig
#*#
ex[0-9]
ex[0-9]f
ex[0-9]f90
ex[0-9][0-9]
ex[0-9][0-9]f
ex[0-9][0-9]f90
*.m.html
index.html
*.F.html
makefile.html
*.c.html
*.h.html
*.h90.html
*.cxx.html
*.hh.html
*.sidl.html
docs/manualpages/*
docs/changes/*
docs/bugreporting.html
docs/codemanagement.html
docs/copyright.html
docs/exampleconcepts
docs/faq.html
docs/installation.html
docs/linearsolvertable.html
docs/manconcepts
docs/manual.pdf
docs/troubleshooting.html
src/docs/tex/manual/manual1.tex
1aa4d6336f2765a087177bad4ef722ba8204f02b release-2.3.2
#!/usr/bin/env python
"""
Generates the code needed to connect an application code
to the TOPS Solver Components
"""
from TOPSInstaller import *
if __name__ == '__main__':
args = []
title = "TOPS Code Generator"
message1 = """SciDAC TOPS Code Generator"""
message2 = """The DOE Mathematics SciDAC TOPS ISIC develops software for the
scalable solution of optimizations, eigenvalues and algebraic systems. More
information on TOPS may be found at http://www.tops-scidac.org. \n\n
This tool allows you to define the class of algebraic problem you are solving
and generates the appropriate "glue" code needed to use the TOPS Solver Components"""
result = buttonbox(message=message1, title=title, choices = ["Cancel", "Continue"],fontSize = 20,message2=message2)
if result == "Cancel": sys.exit()
dim = int(buttonbox(message="Dimension of the grid?", title=title, choices = ["1", "2", "3"],fontSize = 20))
lin = buttonbox(message="Type of algebraic problem?", title=title, choices = ["linear", "nonlinear"],fontSize = 20)
if lin == "nonlinear":
jac = buttonbox(message="Will you provide analytic Jacobian?", title=title, choices = ["No", "Yes"],fontSize = 20,message2="Otherwise it will be computed via finite differencing")
ig = buttonbox(message="Will you provide an initial guess?", title=title, choices = ["No","Yes"],fontSize = 20)
dof = int(enterbox("Number of degrees of freedom per grid point?",title,argDefaultText="1"))
grid = buttonbox(message="Type of grid?", title=title, choices = ["logically rectangular", "unstructured"],fontSize = 20)
if grid == "logically rectangular":
uns = "S"
staggered = buttonbox(message="Are you using a staggered grid?", title=title, choices = ["No","Yes"],fontSize = 20)
else:
uns = "Uns"
app = enterbox("Name of application?",title)
# generate the sidl
import os.path
import sys
if not os.path.isdir(app): os.mkdir(app)
f = file(os.path.join(app,app+'.sidl'), 'w')
f.write('package '+app+' version 0.0.0 {\n')
f.write(' class System implements-all TOPS.System.System,gov.cca.Component, gov.cca.ports.GoPort,\n')
f.write('TOPS.System.Initialize.Once, TOPS.System.Initialize.EverySolve,\n')
if lin == 'nonlinear':
f.write('TOPS.System.Compute.Residual')
if jac == 'yes':
f.write(',TOPS.System.Compute.Jacobian')
else:
f.write('TOPS.System.Compute.RightHandSide,')
f.write('TOPS.System.Compute.Matrix')
if ig == 'yes':
f.write(',TOPS.System.Compute.InitialGuess')
f.write('{}\n}\n')
f.close()
f = file(os.path.join(app,'makefile'),'w')
f.write('EXNAME = '+app+'\n')
f.write('SIDL = ${EXNAME}.sidl\n')
f.write('TLIBNAME = lib${EXNAME}\n')
f.write('SIDLEXCLUDE = -e TOPS[\.a-zA-Z_]* -e gov.cca[\._a-zA-Z_]*\n')
f.write('TOPSCLIENT_LIB = -L${PETSC_LIB_DIR} ${CC_LINKER_SLFLAG}${PETSC_LIB_DIR} -ltopsclient-c++\n')
f.write('DIRS = server/c++\n')
f.write('include ${PETSC_DIR}/src/tops/makefile.rules\n')
f.close()
result = buttonbox(message="Ready to generate code", title=title, choices = ["Continue"],fontSize = 20,message2="This may take several minutes.")
import commands
(status,out) = commands.getstatusoutput('cd '+app+';make server/c++/obj/makefile')
if status:
result = buttonbox(message="SIDL code generation failed", title=title, choices = ["Ok"],fontSize = 20,message2=out)
# Add the common code
f = file(os.path.join(app,'server','c++',app+'_System_Impl.cc'),'r')
text = f.read()
f.close()
bscode = 'this->solver = (TOPS::'+uns+'tructured::Solver)solver;'
text = text.replace('begin('+app+'.System.setSolver)','begin('+app+'.System.setSolver)\n'+bscode)
if dof > 1:
bscode = ' this->solver.setBlockSize('+str(dof)+');'
text = text.replace('begin('+app+'.System.initializeOnce)','begin('+app+'.System.initializeOnce)\n'+bscode)
bscode = '''TOPS::'''+uns+'''tructured::Solver solver = this->solver;'''
if grid == 'logically rectangular':
if dof > 1: sct = 1
else: sct = 0
bscode = bscode + '''
int xs = x.lower('''+str(sct)+''');
int xm = x.length('''+str(sct)+''') - 1;'''
if dim > 1:
bscode = bscode + '''
int ys = x.lower('''+str(sct+1)+''');
int ym = x.length('''+str(sct+1)+''') - 1;\n'''
if dim > 2:
bscode = bscode + '''
int zs = x.lower('''+str(sct+2)+''');
int zm = x.length('''+str(sct+2)+''') - 1;\n'''
if dim > 2:
bscode = bscode + '''for (int k=zs; k<zs+km; k++) {\n'''
if dim > 1:
bscode = bscode + '''for (int j=ys; j<ys+ym; j++) {\n'''
bscode = bscode + '''
for (int i=xs; i<xs+xm; i++) {
}'''
if dim > 2:
bscode = bscode + '''}\n'''
if dim > 1:
bscode = bscode + '''}\n'''
if lin == 'nonlinear':
text = text.replace('begin('+app+'.System.computeResidual)','begin('+app+'.System.computeResidual)\n'+bscode)
if jac == 'Yes':
text = text.replace('begin('+app+'.System.Jacobian)','begin('+app+'.System.Jacobian)\n'+bscode)
else:
text = text.replace('begin('+app+'.System.computeRightHandSide)','begin('+app+'.System.computeRightHandSide)\n'+bscode)
text = text.replace('begin('+app+'.System.computeMatrix)','begin('+app+'.System.computeMatrix)\n'+bscode)
if ig:
text = text.replace('begin('+app+'.System.initalGuess)','begin('+app+'.System.initialGuess)\n'+bscode)
bscode = '''
myServices = services;
gov::cca::TypeMap tm = services.createTypeMap();
if(tm._is_nil()) {
fprintf(stderr, "Error:: %s:%d: gov::cca::TypeMap is nil\\n",__FILE__, __LINE__);
exit(1);
}
gov::cca::Port p = self; // Babel required casting
if(p._is_nil()) {
fprintf(stderr, "Error:: %s:%d: Error casting self to gov::cca::Port \\n",__FILE__, __LINE__);
exit(1);
}
myServices.addProvidesPort(p,"TOPS.System","TOPS.System", tm);
myServices.addProvidesPort(p,"TOPS.System.Initialize.Once","TOPS.System.Initialize.Once", tm);
myServices.addProvidesPort(p,"TOPS.System.Initialize.EverySolve","TOPS.System.Initialize.EverySolve", tm);
// GoPort (instead of main)
myServices.addProvidesPort(p,"DoSolve","gov.cca.ports.GoPort",myServices.createTypeMap());
myServices.registerUsesPort("TOPS.'''+uns+'''tructured.Solver","TOPS.'''+uns+'''tructured.Solver", tm);'''
if ig:
bscode = bscode + '''myServices.addProvidesPort(p,"TOPS.System.Compute.InitialGuess","TOPS.System.Compute.InitialGuess", tm);'''
if lin == 'nonlinear':
bscode = bscode + '''myServices.addProvidesPort(p,"TOPS.System.Compute.Residual","TOPS.System.Compute.Residual", tm);'''
if jac == 'Yes':
bscode = bscode + '''myServices.addProvidesPort(p,"TOPS.System.Compute.Jacobian","TOPS.System.Compute.Jacobian", tm);'''
else:
bscode = bscode + '''myServices.addProvidesPort(p,"TOPS.System.Compute.RightHandSize","TOPS.System.Compute.RightHandSide", tm);'''
bscode = bscode + '''myServices.addProvidesPort(p,"TOPS.System.Compute.Matrix","TOPS.System.Compute.Matrix", tm);'''
text = text.replace('begin('+app+'.System.setServices)','begin('+app+'.System.setServices)\n'+bscode)
bscode = ''' int argc = 1;
char *argv[1];
argv[0] = (char*) malloc(10*sizeof(char));
strcpy(argv[0],"'''+app+'''");
this->solver = myServices.getPort("TOPS.'''+uns+'''tructured.Solver");
this->solver.Initialize(sidl::array<std::string>::create1d(argc,(const char**)argv));
this->solver.solve();
myServices.releasePort("TOPS.'''+uns+'''tructuredSolver");'''
text = text.replace('begin('+app+'.System.go)','begin('+app+'.System.go)\n'+bscode)
f = file(os.path.join(app,'server','c++',app+'_System_Impl.cc'),'w')
f.write(text)
f.close()
f = file(os.path.join(app,'server','c++',app+'_System_Impl.hh'),'r')
text = f.read()
f.close()
bscode = '''#include "TOPS.hh"'''
text = text.replace('begin('+app+'.System._includes)','begin('+app+'.System._includes)\n'+bscode)
bscode = ''' TOPS::'''+uns+'''tructured::Solver solver;
gov::cca::Services myServices;'''
text = text.replace('begin('+app+'.System._implementation)','begin('+app+'.System._implementation)\n'+bscode)
f = file(os.path.join(app,'server','c++',app+'_System_Impl.hh'),'w')
f.write(text)
f.close()
text = '''#!ccaffeine bootstrap file.
# ------- don't change anything ABOVE this line.-------------
path set '''+os.path.join('@PETSC_LIB_DIR@','cca')+'''
repository get-global TOPS.StructuredSolver
repository get-global '''+app+'''.System
instantiate TOPS.StructuredSolver solver
instantiate '''+app+'''.System system
connect solver TOPS.System.Initialize.Once system TOPS.System.Initialize.Once
connect solver TOPS.System.Initialize.EverySolve system TOPS.System.Initialize.EverySolve
connect system TOPS.Structured.Solver solver TOPS.Structured.Solver
parameter solver tops_options options "-snes_monitor -ksp_monitor"\n'''
if ig:
text = text + '''connect solver TOPS.System.Compute.InitialGuess system TOPS.System.Compute.InitialGuess\n'''
if lin == 'nonlinear''':
text = text + '''connect solver TOPS.System.Compute.Residual system TOPS.System.Compute.Residual\n'''
text = text + '''go system DoSolve
quit'''
f = file(os.path.join(app,app+'_rc.in'),'w')
f.write(text)
f.close()
result = buttonbox(message="Code generated",title=title,choices = ["Ok"],fontSize = 20,message2="Now edit "+app+"/server/c++/"+app+"_System_Impl.cc")
#!/usr/bin/env python
#!/bin/env python
# $Id: adiforfix.py,v 1.3 2001/08/24 16:32:18 bsmith Exp $
#
# change python to whatever is needed on your system to invoke python
#
# Adds & continuation marker to the end of continued lines
#
# Calling sequence:
# | adiforfix.py
#
import urllib
import os
import ftplib
import httplib
import sys
from exceptions import *
from sys import *
from string import *
def main():
lines = sys.stdin.readlines()
n = len(lines)
for i in range(0,n):
lines[i]=replace(lines[i],"\t"," ")
for i in range(0,n):
# replace tab with 6 spaces
if len(lines[i]) > 0:
if (lines[i][0] == '\t'):
lines[i] = " "+lines[i][1:]
# replace comment indicator with !
if len(lines[i]) > 0:
if (lines[i][0] == 'c') | (lines[i][0] == 'C') | (lines[i][0] == '*'):
lines[i] = "!"+lines[i][1:]
# move number right one position
# if len(lines[i]) > 0:
# if (lines[i][0] != ' ') & (lines[i][0] != '!') & (lines[i][0] != '#'):
# lines[i] = " "+lines[i]
# replace continuation indicator with &
if len(lines[i]) > 6:
if (lines[i][5] != ' ') & (lines[i][5] != '\t') & (lines[i][0] == ' '):
lines[i] = " &"+lines[i][6:]
for i in range(0,n):
if lines[i][0] == ' ':
# is next line a continued line
for j in range(i+1,n):
if len(lines[j]) > 6:
if (lines[j][5] == '&') & (lines[j][0] == ' '):
lines[i] = rstrip(lines[i])+" "
lines[i] = lines[i][0:72]+"&\n"
break
elif (lines[j][0] == ' '):
break
# replace E - with E-
lines[i]=replace(lines[i],"E - ","E-")
sys.stdout.writelines(lines)
#
# The classes in this file can also be used in other python-programs by using 'import'
#
if __name__ == '__main__':
main()
#!/usr/bin/env python
#!/bin/env python
# $Id: adprocess.py,v 1.12 2001/08/24 18:26:15 bsmith Exp $
#
# change python to whatever is needed on your system to invoke python
#
# Processes .c or .f files looking for a particular function.
# Copies that function as well as an struct definitions
# into a new file to be processed with adiC
#
# Calling sequence:
# adprocess.py file1.[cf] functionname functionname2
#
#
# Bugs: Final line of C function must begin with } on first line
# Application context must be called AppCtx
# structs possible format is not completely general
#
import urllib
import os
import ftplib
import httplib
import re
from exceptions import *
from sys import *
from string import *
from parseargs import *
#
# Copies structs from filename to filename.tmp1
def setupfunctionC(filename,g = None):
import re
regtypedef = re.compile('typedef [ ]*struct')
reginclude = re.compile('#include [ ]*"([a-zA-Z_0-9]*.h)"')
regdefine = re.compile('#define')
regdefine__ = re.compile('#define [ ]*__FUNCT__')
regextern = re.compile('extern')
regEXTERN = re.compile('EXTERN')
regif = re.compile('#if')
regendif = re.compile('#endif')
f = open(filename)
if not g:
newfile = filename + ".tmp1"
g = open(newfile,"w")
g.write("#include <math.h>\n")
g.write("#define PetscMin(a,b) (((a)<(b)) ? (a) : (b))\n")
line = f.readline()
while line:
# line = lstrip(line)+" "
fl = regtypedef.search(line)
if fl:
struct = line
while line:
reg = re.compile('^[ ]*}')
fl = reg.search(line)
if fl:
break
line = f.readline()
struct = struct + line
#
# if this is the AppCtx then replace double and Scalar with passive
#
reg = re.compile('^[ ]*}[ ]*AppCtx[ ]*;')
fl = reg.search(line)
if fl:
print "Extracting structure AppCtx"
reg = re.compile('\n[ ]*PetscScalar ')
struct = reg.sub('\nPassiveScalar ',struct)
reg = re.compile('\n[ ]*double ')
struct = reg.sub('\nPassiveReal ',struct)
reg = re.compile('\n[ ]*PetscReal ')
struct = reg.sub('\nPassiveReal ',struct)
else:
reg = re.compile('^[ ]*}[ ]*')
line = reg.sub('',line)
reg = re.compile('[ ]*;\n')
line = reg.sub('',line)
print "Extracting structure "+line
g.write(struct)
# copy over all #define macros
fl = regdefine.search(line)
if fl:
fl = regdefine__.search(line)
if not fl:
g.write(line)
# copy over extern statements
if regextern.search(line) or regEXTERN.search(line):
g.write(line)
if reginclude.search(line):
fname = reginclude.match(line).group(1)
if os.path.exists(fname):
setupfunctionC(fname,g)
fl = regif.search(line)
if fl:
g.write(line)
fl = regendif.search(line)
if fl:
g.write(line)
line = f.readline()
f.close()
return g
#
# Appends function functionname from filename to filename.tmp1
def getfunctionC(g,filename,functionname):
import re
f = open(filename)
g.write("/* Function "+functionname+"*/\n\n")
line = f.readline()
while line:
for i in split('int double PetscReal PetscScalar PassiveReal PassiveScalar PetscErrorCode'," "):
reg = re.compile('^[ ]*'+i+'[ ]*'+functionname+'[ ]*\(')
fl = reg.search(line)
if fl:
print 'Extracting function', functionname