If this is the first build tool you have ever used, then this section is the best place to start. Make sure you have read the Introduction if you haven’t.
In this section we are going to describe the very basics of a build system in order to understand its internal components and how it works at a high level.
Most concepts in pyke are similar to other build tools, but even if you are already experienced using other build tools I think it would be a good idea to read through this section.
A pyke file is no more than a set of rules written in python that tell pyke what, how and when to run something.
Unlike other build systems where you use a custom language to create new rules, define targets, dependencies and commands, pyke uses python to do all of this. The good thing is that it should be really, really easy for you even if you don’t know python language. In pyke, rules are defined as python functions, and the parameters of those functions are used as the inputs and the outputs.
A rule is a way to abstract a system command and its dependencies.
These functions internally handle what to do (the commands/jobs that should be executed), what they need to do it (dependencies), and what files will be generated after the commands are executed (targets).
A job is a system command or a python function that will be executed in order to generate something.
For example, if you want to compile the file “main.cc” and generate the object file “main.o” you will run something like “g++ -c -o main.o main.cc”. That command would be a job, and will be created by running a rule.
Apart from system commands, you can also define your own python functions to run as jobs. Imagine that you want to convert a file from latin-1 to UTF-8. With pyke you can easily define a python function and have it directly integrated as a part of the build system itself (more on this on later chapters).
A target is a file or set of files that are created after running one or more jobs.
“main.o” is the file that will be created after running “g++ -c -o main.o main.cc”, for that reason “main.o” would be considered a target.
Sometimes there can be several targets. Say you run the following command: “g++ -o main main.cc -Wl,-Map,main.map”
Here, you will be generating the application “main” and the map file “main.map”, thus, there would be two targets.
Sources and dependencies, as you might expect, are the input files required by the commands to generate the targets, wether explicit or implicit.
For this command: “g++ -c -o main.o main.cc”
“main.cc” would be the source, and, if a “main.h” exists, “main.h” would be a dependency.
Pyke rules usually handle dependencies automatically, but there are cases where this is not possible or you might want to force a command to be executed after some other command. Pyke has ways of doing this (see _needs parameter on Optional keyword arguments)
Pyke comes with a predefined set of rules for your convenience. You can define your own rules, though, although the easiest way to get started is to make use of the existing rules.
For example, in order to create “main.o” from “main.cc” (including the dependencies) you will create a “master.pyke” file like this:
from pyke.tools.default import *
cc ('main.cc')
And in order to create the executable ‘main’ out of ‘main.o’ you will do:
from pyke.tools.default import *
app ('main', cc ('main.cc'))
Please note that the code above compiles in Windows, in Mac and in Linux if you have the right tools installed (g++ or Visual Studio).
Please note that the return value of “cc (‘main.cc’)” is a list containing the absolute path to “main.o” object file. The output filename is automatically generated inside the ‘cc’ call.
Pyke follows some basic conventions to make it easier to write and understand pyke code:
Each rule always returns a list containing the absolute path of all the target files that are going to be created. This way it is really easy to chain commands in big projects. You’ll see some examples in this documentation.
# chaining: build object files for main.cc then create the application
app ('main', cc ('main.cc'))
Target names are guessed when possible, if they cannot be guessed (say when you specify the application name, or the name of a compressed file you are building), then they should be specified as the first parameter of the rule.
# automatically generate main.o
cc ('main.cc')
# specify the target file
compress ('something.zip', [file1, file2, ..., fileN])
Source and target arguments can be either a path string or a list of path strings, always expressed as absolute paths or relative paths to the current *.pyke* file.
# all are valid calls
cc ('main.cc')
cc (['main.cc', 'hello/hello.cc'])
cc ('*.cc')
cc ('/abs/path/to/file.cc')
You can add _needs parameter to any rule in order to specify a list of dependencies that cannot be guessed or that you want to enforce.
hello = cc ('hello/hello.cc')
# main.o will wait until hello.o has been built
cc ('main.cc', _needs = hello)
You can add _creates parameter to any rule in order to specify a list of targets that cannot be guessed or that you want to enforce.
# we let know pyke that 'myfile.txt' will be created by this command
shell ('touch myfile.txt', _creates = 'myfile.txt')
In your project you can create whatever conventions you want, but these are the standards in pyke.
For every command that is going to be executed, pyke should know all the inputs and all the outputs, that way pyke can schedule each command and parallelize it in the most optimal way.
The scheduler for pyke is quite powerful and very simple conceptually. First, the commands that have no dependencies are executed, then, after a command finishes it unlocks all commands that depended on the previous one, and will continue running this way until all commands are executed.
As in other built systems, pyke only rebuilds a file when any of the sources or dependencies is more recent than the targets. This behaviour can be changed, as most things in pyke, but is an advanced topic and won’t be discussed here.